Building models for prediction · Apple quarterly revenue
How much to hold out?
To test a forecast, we hide the most recent quarters, fit a model to everything before them, and see how well it predicts the quarters it never saw. Choose how many quarters to hold out and which model to fit, and watch the fitted line (solid) and the projection (dashed) change.
Step 1
Fit on the past, project into the holdout
Blue dots are the training quarters. The solid line is the model fit to them. The shaded orange band is the holdout: the dashed line carries the same trend forward into quarters the model never saw, and the orange lines are its misses.
Step 2
The 4 held-out forecasts
RMSPE turns these misses into one number: square each miss, average the squares, take the square root.
| Quarter | Actual ($B) | Forecast ($B) | Miss | Miss squared |
|---|---|---|---|---|
| Mean of squares | ||||
| Square root = RMSPE |
Step 3
Every holdout size at once
Each holdout size gives a different test. Here are all of them from 1 to 20 quarters. Click the chart to jump to one.
- Test error is always bigger. At every holdout size the orange line sits above the blue one. Fit on data the model already saw always looks better than forecasting data it didn't.
- Hold out too little and one quarter decides. With 1 quarter held out, the whole test is 2025 Q4, the holiday quarter, and RMSPE jumps to $29B. With 4 or more, the answer settles between about $16B and $19B.
- Hold out too much and you give up data. Every quarter you hide is one fewer to train on, and the model has to forecast further ahead.
- Which model wins depends on the test. The gray dashed line is the other model. With 4 quarters held out the two are nearly tied ($19.08B for log, $19.28B for linear). The log model wins at some holdout sizes and the linear model at others. Neither one can see seasons, so both miss the same way.
For quarterly data, holding out the last 4 quarters (one year) is a common choice: enough quarters to see every season once, while keeping almost all the history for training. That's what the lecture uses.
In R
Choose a holdout, fit both models, forecast
Revenue in billions of dollars, by calendar quarter, 2011 Q3 to 2025 Q4 (58 quarters). The log model is log(Revenue) ~ Period and the linear model is Revenue ~ Period, each fit by least squares to the training quarters. "Miss on training data" is the root mean squared miss in dollars on the training quarters. Companion lab: Train, Test, Repeat.