Lead Time

Lead time refers to the latency period that exists between a customer placing an order and receiving the order. For manufacturers and retailers, a product must often be manufactured or ordered before it is sold or replenished at downstream facilities, and this process can take anywhere between a few days to a few months. Such delays in fulfillment mean that the forecast for any week must be locked down based on the demand observed days or weeks prior to the most recent time point. This also means that the forecast does not have to necessarily operate long-term, because the modeler can update the forecast as new demand is realized.

Under these circumstances, it is imperative that any demand model providing a forecast is also being validated in a similar fashion as the process followed by the demand planner.

Validation without lead time

Validation of demand models in Demand Guru is catered towards mid- to long-range forecasting and modeling. To validate the demand model, historical data is divided into In Sample (training) and Out of Sample (testing) based on the slice ratio you define. Out of sample errors are calculated based on the assumption that you are interested in the accuracy of the forecast over the entire span of the test data, with a demand model that is trained based solely on the training data points. The validation process occurs in the workflow as shown in the following example:

In this example, out of sample error metrics are calculated on the accuracy of the forecast over 5 test periods (from period 41 through 45), based on a model trained with 40 data points.

Validation using lead time

If you are more interested in error that occurs over a specific future point in time, and you know that you can update the forecast in the future, you might not be concerned about the error metric observed over the entire test data span. Instead, you might want to focus on a specific week of the test data. In our lead time example, a total of 45 historical data points is split into 40 training data points, and 5 test data points. While validating the model, if you are interested in the forecast accuracy on the third week from when the historical data stops (you have a lead time of 2 weeks prior to the product being delivered for the third week), you might want the model to be chosen and/or validated based on the error that occurs over that third week.

The Lead Time feature in Demand Guru enables you to validate your model repeatedly and thus choose the most accurate model corresponding to the lead time you utilize for demand forecasting. To facilitate calculating the accuracy of the model for a specific lead time in a consistent manner, the application uses a rolling horizon approach for validating forecasts. In our example, if you provide a lead time value of 2 weeks, Demand Guru first predicts week 43 for a model trained with 40 data points, then updates the model to include the 41st data point and predicts for week 44, and finally updates the model to include the 42nd time period to predict week 45. The error metrics are then calculated for the specific lead time and reported back to you as out of sample error metrics in the Model Summary table.