Reasons for changing the forecasting model: a response to The Economist

The Economist magazine have published a great article on the difficulties of predicting the next election. It makes lots of good points well and kindly covers my 2015 general election forecasting model. But I was bemused by the line that reads, "Within months he had published a revised model: polls had not adhered to the original one."

I did revise my October 2013 model in February 2014, but not because the polls had not adhered to the original one. They have kindly published a correction. The rest of this post sets out why adherence of polls to the model has be adequate and explains the actual basis for the change in the model, before finally noting that adherence to the polls might weaken close to the European Parliament and local council elections.

As I wrote last week here, there has been very little change in the polls or the forecasts since October, especially relative the to prediction interval for the forecast. Given the polls, this would be true for any model of this kind at this stage in the election cycle. Without dramatic swings in the polls, we are too far from the election and there is too much variation in the historical record to be able to say that deviations from the central estimates of any such forecasting model are so big as to cast doubt on any particular model.

Only today, have I looked at how both models changed between the first published forecast on the website (25th Oct 2013) and last published forecast for the old model before switching to the new one (31st Jan 2014).  While the two models have substantially different predictions the change in the forecast share of the vote for both models over this period was the same: Con +1, Lab -1 and less than a percentage point change for LD. 

So not only was there was no reason to switch models on the basis of adherence to the polls, but the trajectory of the polls has been as close to that predicted by both models as could reasonably be expected.

What prompted the change of the model was explained at the time here, “The change in the methodology is due to a couple of the comments which prompted me to reconsider modelling reversion to the last election, which was an approach I had neglected too quickly.” The justification for the switch of model in the February working paper is better performance at predicting past election results: nothing to do with polls in this cycle. 

If and when I do change the model again (I have said I probably would) it will be an attempt to better reflect the most important aspects of the historical record. Forecasting the current election cycle does require some judgement calls which should depend on current circumstances. But the changes in the polls over the last year do not tell us much as to what kind of model to prefer.

All that said, as we approach the European Parliament and local council elections we are likely to see general election vote intention in the opinion polls change. The Conservatives dropped and UKIP rose in the polls in run up to last year’s local elections. Similarly in the month before the 2009 European and local elections, support for the government dropped while UKIP and other minor party support rose dramatically, not just in European Parliament vote intention but in general election polls too. The MPs’ expenses crisis doubtless contributed to the 2009 changes and this pattern was much more muted in 2004. In all of these cases there was some reversion back to pre-campaign levels afterwards.

This pattern does not always hold, certainly not for all local elections. But if this kind of phenomenon happens again this year then my forecasting model will behave strangely but revert back to normal in a couple of months time.