Contents
How are forecast errors and residuals different from each other?
Note that forecast errors are different from residuals in two ways. First, residuals are calculated on the training set while forecast errors are calculated on the test set. Second, residuals are based on one-step forecasts while forecast errors can involve multi-step forecasts.
How is the size of a test set used in forecasting?
Because the test data is not used in determining the forecasts, it should provide a reliable indication of how well the model is likely to forecast on new data. The size of the test set is typically about 20% of the total sample, although this value depends on how long the sample is and how far ahead you want to forecast.
How are forecast errors related to scale dependent errors?
Scale-dependent errors The forecast errors are on the same scale as the data. Accuracy measures that are based only on et e t are therefore scale-dependent and cannot be used to make comparisons between series that involve different units.
How is the accuracy of a forecast determined?
It is important to evaluate forecast accuracy using genuine forecasts. Consequently, the size of the residuals is not a reliable indication of how large true forecast errors are likely to be. The accuracy of forecasts can only be determined by considering how well a model performs on new data that were not used when fitting the model.
How are scaled errors used to evaluate forecast accuracy?
Scaled errors were proposed by Hyndman & Koehler (2006) as an alternative to using percentage errors when comparing forecast accuracy across series with different units. They proposed scaling the errors based on the training MAE from a simple forecast method.
How are training and test data used to evaluate forecasts?
When choosing models, it is common practice to separate the available data into two portions, training and test data, where the training data is used to estimate any parameters of a forecasting method and the test data is used to evaluate its accuracy. Because the test data is not used in determining the forecasts,…