Mixed type algorithms

Mixed type algorithms are really more like methods that combine classic and machine learning algorithms. No single classic or machine learning algorithm can surpass all other algorithms in terms of accuracy on any given data set. In Demand Guru, you can run a set of algorithms on a specific data set to identify and use the algorithm that performs the best on its test partition. The out-of-sample RMSE values (RMSE calculated over the test data) provides relative test performance of all the algorithms considered.

Tournament

In its default setting, all time series forecasting algorithms except AutoArima, Automatic, and Automatic Plus are run in a tournament mode. The algorithm that yields the lowest out-of-sample RMSE value is selected for predicting over the forecast horizon.

Typically, SVM Linear is selected as the best algorithm because it is robust to outliers, fast, and works well across variety of datasets. However, if demand data is random and/or does not exhibit seasonality and trend patterns, or if the time series length is too short, predictions by this algorithm can stray in the wrong direction because of overfitting. On such random data sets, the Simple Moving Average or Simple Exponential Smoothing algorithm will at least provide predictions that are close to the recent demand history. For this reason, they are included in the tournament.

Additional algorithms can be added to the tournament for more accurate results. However, a trade-off exists between time and accuracy. Other algorithms added to the tournament can produce more accurate results, but the also require additional time to complete the operation.

Ensemble

Similar to the Tournament selection, several algorithms are run in a tournament mode:

• Arima
• Arimax
• Model Average Neural Network
• Stochastic Gradient Boosting
• Intermittent
• Naïve Forecasting
• Quantile Random Forest
• Support Vector Machine (SVM Linear)
• Simple Moving Average
• Exponential Smoothing

However, instead of selecting the algorithm that yields the lowest out-of-sample RMSE value, the Ensemble algorithm assigns a score (weight) to each algorithm based on its performance.

Refer to Comparing Tournament and Ensemble algorithms for more information.

Automatic

The Automatic algorithm is exactly the same as Tournament algorithm when its default settings are used. The three algorithms (SVM Linear, Simple Moving Average, and Simple Exponential Smoothing) are run in a tournament mode, and the best algorithm is selected based on its performance on the test data (out-of-sample RMSE).

This algorithm is designed for those with limited knowledge and/or experience with various forecasting algorithms and causal forecasting. Unlike the Tournament algorithm, it does not include any controllable parameters, and so no additional algorithms can be added into the tournament mode for comparison.

Automatic Plus

The Automatic Plus algorithm combines certain capabilities of the Tournament and Automatic algorithm choices. Initially, it predicts the top three time series forecasting algorithms for each time series, which include at least one machine learning algorithm and one classic algorithm. This prediction is made using historical data and RMSE, with no causal data considered and regardless of the error metric you select. The three algorithms are then run in a fashion similar to the automatic method, using the error metric you have selected and any causal data you have provided, and the algorithm with the lowest RMSE value is selected as the best one.

This choice can provide improved model execution runtime over the Tournament method while also resulting in a lower RMSE than the Automatic method.