Unraveling the 2014 local election changes in the share of the vote: who suffered most from UKIP?

A naïve glance at the overall changes in the share of the vote since 2010 in the table below suggests that UKIP’s big gains came as the expense of the biggest losers, the Liberal Democrats.

	Change % vote since 2010	Change % vote since 2012
Con	-6	-0.2
Lab	+5	-8
LD	-14	-3
UKIP	+13	+10
Grn	+4	+2
BNP	-2	-0.2
Oth	+0	-0.6

(Overall changes in the share of the vote across the BBC’s 1008 Keywards.)

This is clearly one of those cases where the net changes in the share of the vote can be misleading.  The opinion polls have commonly told us that Labour have benefitted primarily from the collapse of the Liberal Democrats, whereas the UKIP surge has come mostly from the Tories. 

If all the Lib Dem losses since 2010 were to Labour then they should be up 14 not just 5. The Green 4 point gain is likely to have largely come from the LDs, but that would still leave Labour 5 points short of where they would have been if they were simply benefitting from the Lib Dem losses.

Conversely, on the right of the political spectrum, if, apart from 2 points from the BNP, all of UKIP’s 13 point rise since 2010 were from the Tories, then Conservative would have been down by 11 points, not 6 they were actually down by. This discrepancy of 5 points corresponds to the 5 points Labour seem to have been short by in the above paragraph.

On this rough reckoning then, UKIP took 5 points from Labour, and 6 points from the Conservatives.  So, by comparison with 2010, it looks like Farage hurt both major parties roughly equally, not disproportionately from the Tories.

The story about change since 2012 also in the table is also revealing here.  That shows much more clearly how Labour have fallen and UKIP risen in local election vote shares over the last two years.  What seems to have happened is that between 2010 and 2012 UKIP took votes mainly from the Conservatives, but between 2012 and 2014 they have had more success in attracting Labour voters. The net effect is that the UKIP rise from 2010 to 2014 has been at similar expense to Labour and the Conservatives.

This is certainly a story that fits the main pattern of change in the general election vote intention opinion polls. The narrowing of the Labour lead over the past two years has been mainly due to a Labour fall and UKIP rise.

Another way of looking at this issue is to consider who suffered most in the wards where the UKIP vote share went up most.  In BBC Keywards fought by all four parties, the correlation between the change in the UKIP share of the vote since 2010 and that for the Conservatives was very similar to that between UKIP and Labour: both close to -0.3.  This provides further evidence that UKIP hurt Labour by about the same amount as they hurt the Conservatives.

This pattern is also visible in the main source of regional variation.  Across wards fought by all three main Westminster parties, UKIP were up by just 6 points in London, but by 14 points elsewhere in England: an 8 point difference.  The corresponding (partially) compensating London/elsewhere differences in the performances of the Conservatives, Labour and Liberal Democrats were 2.0, 2.6 and 1.4 points respectively.  These figures not only reinforce the picture of Labour suffering similarly if not more than the Tories from UKIP, but they also show an (albeit smaller) impact on the Liberal Democrats.

So much for votes. What about seats? Among the 53 UKIP gains in the BBC Keywards, 22 came from the Conservatives, 19 from Labour and 12 from the Lib Dems.  So a similar pattern again. 

Acknowledgements: Thanks to the BBC, John Curtice, Rob Ford, Jon Mellon and Rosie Shorrocks for their help with both data and analysis.

What do the 2014 local election results mean for next year’s general election?

The Labour lead of 2 percentage points in the BBC’s projected national share of the local election vote (PNS) is too narrow a lead for Labour to suggest they will be ahead in a general election next year. 

Since it was first calculated (in 1983) oppositions have usually had bigger leads in the PNS in the year before a general election, but they still went on to lose the next election.  The changes of government (in 1997 and 2010) were preceded by very big leads for the opposition in the PNS (16 points for Labour in 1996 and a 15 point lead for the Tories in 2009).

The message from the PNS is in line with general election vote intention in the opinion polls, which have on average also shown a narrow lead for Labour recently.

Even though their PNS is lower than the 23% they scored last year, the UKIP challenge remains significant. But since they seem to be hitting both main parties roughly equally, it is perhaps more important to think about both the impact on the major parties and it’s distribution across the country.

A uniform swing projection of the House of Commons from the PNS suggests that UKIP would not win any seats. This is partly because in 2010 their share was very evenly distributed.

There were some signs last year of UKIP’s vote becoming more concentrated in areas with more older people, fewer graduates and fewer ethnic minorities. 

UKIP share became still more variable this year. They went up more where they started strongest. So they are building up strong support in particular places, which is what they need to do to win seats in Westminster.  Indeed UKIP won the share of the vote across all the wards in the Great Grimsby constituency. This was the seat that Robert Ford and Matthew Goodwin predicted would be the most fertile territory for a UKIP general election victory.

(Rotherham has also been mentioned as an effective UKIP constituency win, but it was not part of the BBC keywards data collection so I’m not in a position to be able to confirm.)

Moreover, there are several seats where, based on the results of this week’s local elections UKIP would not be far behind the winning party. These include Portsmouth North (5.9% behind Con), Portsmouth South (6.1% behind LD), Southport (9.3% behind LD), Bradford South (9.4% behind Lab). 

Just as in local elections, UKIP are more likely to win seats in the Commons where the share of the vote between the three main parties is more evenly divided. In a general election if it becomes apparent that UKIP are a serious challengers this will lead to the kind of attention that might generate strong efforts to encourage tactical voting both for and against UKIP. So it may be that UKIP will find it difficult to divide and conquer.

Acknowledgements: Thanks to the BBC, John Curtice, Rob Ford, Jon Mellon and Rosie Shorrocks for their help with both data and analysis.

BBC Projected National Share of Vote 2014

The BBC Projected National Share of the vote just announced is Con 29, Lab 31, LD 13, UKIP 17, Others 10.

The PNS is an attempt to estimate what the share of the vote would be if the whole of GB had local elections and if the three main Westminster parties had fielded candidates in all wards, as they do in general elections. For more details see here and here.

Graphs below show how it compares with previous figures and polls historically.

Locals seats forecast from national polls 2014

Last year I published details of a basic model for forecasting the number of local council seats won from national opinion polls here. The basic idea is to estimate change in seats since the last time they were fought (four years ago) as a function of change in the polls over that time.  Despite the massive increase in UKIP candidature last year and their remarkable achievement of a Projected National Share (PNS) of the vote of 23%, the model still performed relatively well. In part because the even distribution of their vote meant UKIP only won 147 seats. 

The following table shows the actual number of gains and losses in 2013, and both my model estimates and the Rallings and Thrasher forecast.

	Actual	Estimate	Range	Rallings & Thrasher
Con	-355	-205	-295 to -115	-310
Lab	+291	+260	+160 to +355	+350
LD	-124	-190	-85 to -300	-130

Having updated the model, the estimated gains and losses of seats in tomorrow's local elections are as shown in the table below.

	Estimate	Range	Rallings & Thrasher
Con	-130	-235 to -30	-220
Lab	+130	+30 to +220	+490
LD	-400	-570 to -230	-350

The Rallings and Thrasher forecasts come from their PSA briefing.  The pattern of who's expected to be up and who down is straightforward. The seats up this year were last fought on the same day as the 2010 general election; Labour have recovered since then while the governing parties have gone down in the polls. The Liberal Democrat drop is particularly dramatic.

The difference between my model and the Rallings and Thrasher forecast is that for the two main parties the changes are much more muted in my model, especially for Labour.  This maybe partly because they are also predicting UKIP +80, but that still leaves over 300 seat discrepancy to explain. (My model won't be capable of sensibly estimating UKIP seats till 2016: four years after they first fought a large majority of wards in local elections.)

It also doesn't look like the discrepancy is due to differences in estimates for the overall share of the vote. The Rallings and Thrasher forecast 2014 National Equivalent Vote (NEV, similar to PNS) is Con 30, Lab 33, LD 14, UKIP 16, Other 7.  If anything this shows smaller losses and gains since 2010 than do the opinion polls I'm using.  

The graph below shows both the polls and the PNS over time. The rise of UKIP last year led to bigger drops in the PNS for Labour and the Conservatives than might otherwise be expected.  Given the changes in the polls since 2013 and the concurrent Euro elections, the PNS for Con, Lab and UKIP could be close, with UKIP likely to do better than the 16% Rallings and Thrasher have forecast for them.

Returning to the seats forecasts, the differences between the two are most likely to be due to Rallings & Thrasher more careful ward level estimates. My model is a national level only one, estimating the overall share of seats won by each party. Their approach is more detailed and should work better. 

Acknowledgements: Thanks to David Cowling, John Curtice, and Michael Thrasher for help with the data.

Reasons for changing the forecasting model: a response to The Economist

The Economist magazine have published a great article on the difficulties of predicting the next election. It makes lots of good points well and kindly covers my 2015 general election forecasting model. But I was bemused by the line that reads, "Within months he had published a revised model: polls had not adhered to the original one."

I did revise my October 2013 model in February 2014, but not because the polls had not adhered to the original one. They have kindly published a correction. The rest of this post sets out why adherence of polls to the model has be adequate and explains the actual basis for the change in the model, before finally noting that adherence to the polls might weaken close to the European Parliament and local council elections.

As I wrote last week here, there has been very little change in the polls or the forecasts since October, especially relative the to prediction interval for the forecast. Given the polls, this would be true for any model of this kind at this stage in the election cycle. Without dramatic swings in the polls, we are too far from the election and there is too much variation in the historical record to be able to say that deviations from the central estimates of any such forecasting model are so big as to cast doubt on any particular model.

Only today, have I looked at how both models changed between the first published forecast on the website (25th Oct 2013) and last published forecast for the old model before switching to the new one (31st Jan 2014).  While the two models have substantially different predictions the change in the forecast share of the vote for both models over this period was the same: Con +1, Lab -1 and less than a percentage point change for LD. 

So not only was there was no reason to switch models on the basis of adherence to the polls, but the trajectory of the polls has been as close to that predicted by both models as could reasonably be expected.

What prompted the change of the model was explained at the time here, “The change in the methodology is due to a couple of the comments which prompted me to reconsider modelling reversion to the last election, which was an approach I had neglected too quickly.” The justification for the switch of model in the February working paper is better performance at predicting past election results: nothing to do with polls in this cycle. 

If and when I do change the model again (I have said I probably would) it will be an attempt to better reflect the most important aspects of the historical record. Forecasting the current election cycle does require some judgement calls which should depend on current circumstances. But the changes in the polls over the last year do not tell us much as to what kind of model to prefer.

All that said, as we approach the European Parliament and local council elections we are likely to see general election vote intention in the opinion polls change. The Conservatives dropped and UKIP rose in the polls in run up to last year’s local elections. Similarly in the month before the 2009 European and local elections, support for the government dropped while UKIP and other minor party support rose dramatically, not just in European Parliament vote intention but in general election polls too. The MPs’ expenses crisis doubtless contributed to the 2009 changes and this pattern was much more muted in 2004. In all of these cases there was some reversion back to pre-campaign levels afterwards.

This pattern does not always hold, certainly not for all local elections. But if this kind of phenomenon happens again this year then my forecasting model will behave strangely but revert back to normal in a couple of months time.