Contents
How is AIC used in time series forecasting?
AIC is frequently recommended as criterion to compare models for time series forecasting. See for example this in the context of dynamic regression models: The AIC can be calculated for the final model, and this value can be used to determine the best predictors.
Can you compare AIC to a different model?
You can compare AICs for different models fitted by the same method. So no help there. Looking at rolling out-of-sample forecasts was already exactly the right thing to do. Now you know that each model is best one third of the time.
How to choose a forecast for your time series?
Another approach that is quite popular in research is to avoid selecting a single forecast altogether. We can do this by combining forecasts. Returning to Fig. 1 we can take the values of both forecasts and calculate the arithmetic mean for each period: We can combine the forecasts from as many sources as desirable.
Is the AIC computed using the training data?
Yes, it is computed using the training data. But asymptotically, minimizing the AIC is equivalent to minimizing the leave-one-out cross-validation MSE for cross-sectional data, and equivalent to minimizing the out-of-sample one-step forecast MSE for time series models.
When to use LRT and AIC in forecasting?
LRT tells you whether at a chosen confidence level you can reject the hypothesis that some restrictions on the richer model hold (e.g. some elements in the richer model are redundant). You would use AIC if your goal is model selection for forecasting. You would use likelihood ratio test for significance testing.
How is the AIC of a model determined?
AIC works by evaluating the model’s fit on the training data, and adding a penalty term for the complexity of the model (similar fundamentals to regularization ). The desired result is to find the lowest possible AIC, which indicates the best balance of model fit with generalizability.
What’s the difference between AIC and likelihood ratio test?
AIC and likelihood ratio test (LRT) have different purposes. AIC tells you whether it pays to have a richer model when your goal is approximating the underlying data generating process the best you can in terms of Kullback-Leibler distance.
How is AIC calculated for the final model?
The AIC can be calculated for the final model, and this value can be used to determine the best predictors. That is, the procedure should be repeated for all subsets of predictors to be considered, and the model with the lowest AICc value selected.
When is it appropriate to select models by…?
Rather, all models with reasonably small GCV or AIC values should be considered as potentially appropriate and evaluated according to their simplicity and scientific relevance. It has been suggested (Duong, 1984) that models with AIC values within c of the minimum value should be considered competitive (with c=2 as a typical value).