Local Elections 2013: UKIP hurt Labour at least as much as the Tories compared with last year

Since the local elections last week discussion in the media has focused mainly on the impact of UKIP success on the Conservatives and how they will react.  But compared with last year's local elections Labour suffered at least as much.

By comparison with the last county council elections in 2009 it does look like UKIP gains were at the expense of the Conservatives. Over 80% of UKIP seats came from the Tories, and the Tories lost a greater proportion of their seats to UKIP than the Liberal Democrats, while Labour lost barely any to them. Also there were many seats and councils where the UKIP advance was enough to help another party topple the Tories.

It also makes sense that UKIP should be taking votes from the Conservatives more than others. UKIP are a party of the right. They typically take policy positions that are currently considered to be right wing, such as being against EU membership, immigration, gay marriage and climate change mitigation. So it is unsurprising that UKIP support in the polls comes much more from former Conservative voters than from Labour or the Liberal Democrats. 

But there are a number of problems with these observations as a way of looking at the results of this year's local elections.  First, UKIP seat gains were very modest because of the even distribution of the vote and effects of the First Past The Post electoral system. Their vote share of 25% where they stood, not the seat tally, is the main mark of their success. Second, the Conservatives did unusually well and Labour especially badly in 2009. The Tories were bound to suffer loses and Labour bound to make some gains, as they did in 2011 and 2012. Third, attitudes on Europe, immigration and moral issues are not highly correlated with economic left-right positions. So there are plenty of eurosceptic, anti-immigrant and socially conservative voters among those with relatively socialist values who might be persuaded to leave Labour for UKIP. Fourth, while the opinion polls tell us about general election vote intention, local elections are a different matter.  UKIP's share of general election vote intention, at 14% last week, is much more muted than the 23% they achieved in the Projected National Share of the vote (PNS) from the local elections.

For these and other reasons, it is more instructive to consider party performance this year by comparison with last year's local elections, rather than those in 2009.  Between 2009 and 2012 we saw a change of government, a recovery for Labour and a collapse of the Liberal Democrats, a decline for the Conservatives, and, significantly, a drop in UKIP support (they were running at over 15% in the polls in the run up to the 2009 European Parliament elections but at just 8% this time last year). There are important political questions about how the trends for the Westminster parties up to last year's local elections have been disrupted by the rise of UKIP since this time last year.

If we look at the opinion polls, while UKIP did seem to benefit from Tory losses in the month after the 2012 'omnishambles' budget, there has been relatively little change between the start of May this year and the same time last year: Con -1, Lab -2, LD no change and UKIP +4.  Given the usual +/- 3% confidence interval, we can only really be sure there has been a rise of UKIP.  If anything it looks like it has come from Labour more than the other two parties.

By contrast, there were dramatic differences in party performance between this year and last. The PNS shows that while all three main parties are down since last year, Labour has dropped more than the other two (Con -6, Lab -9, LD -2). Labour were the biggest losers and this alone suggests that UKIP hurt Labour substantially if not more than the Tories. 

Certainly we can't account for the UKIP rise since last year by reference to Conservative loses alone. Either there was a massive shift of support from Labour to the Tories and also from the Tories to UKIP or considerable direct loses from Labour to UKIP.

The pattern of the results across divisions suggests that relative to last year new UKIP votes have come as much from Labour as from the Tories. As at the general election, the change in the UKIP share of the vote since 2009 is more strongly negatively correlated with the change in the Conservative share than it is with any other party. This is as you would expect if most of the direct switching to UKIP from the Conservatives. However, if we look at the places that had local elections on the same boundaries last year and this year (combining district wards to make county divisions), the UKIP change since 2012 is equally strongly correlated with both the Conservative and Labour change, suggesting that relative to last year both parties suffered equally from UKIP progress. (See footnote.)

The socio-demographic profile of the places where UKIP did well also suggests that it took more new votes from people who would otherwise have voted Labour this year. We are used to seeing the party doing better in places with more routine manual workers who are traditionally part of Labour's base. But the correlation between the UKIP share and the proportion of working class people is stronger this year than it was in 2009 or 2012. Correspondingly, the change in the Labour share since last year is negatively correlated with the percentage of routine manual workers. So it seems that Labour is losing more of its base to UKIP.

Some may argue that Labour can afford to lose some support in its heartlands so long as it does well where it needs to win seats at the next general election. UKIP advanced more in the South and East of England and least in the North West, while there reverse is true for Labour.  This is the opposite of what Labour need to do. 

Moreover, comparison with 2012 is not unfair to Labour. It was not a particularly good set of local elections for them from which they might be expected to fall.  Their lead over the Tories in the PNS then, and more so now, is considerably lower than at any of the local elections while Tony Blair was leader of the opposition. 

One saving grace for Labour is that in divisions where they started second to, but within 20 points of, the Conservatives (i.e. Con-Lab marginals) their vote share was up by 10.5 points since 2009, compared with just 8 points elsewhere.

Otherwise, the distribution of the new UKIP voters this year bodes at least as badly for Labour as for the Conservatives. But as Nigel Farage is fond of pointing out, the main effect of UKIP is probably more psychological than psephological. At the moment it seems that the Conservatives are the most panicked while Labour are rather nonchalant. Whether either reaction is helpful remains to be seen.

***

Thanks to John Curtice and Rob Ford for comments and to them, Jon Mellon and the BBC for help with the data.

Footnote: Admittedly there are only 69 such divisions where the three main parties stood each time and UKIP at least one time and they fought either all or none of the constituent wards in a division in 2012. They are places where Labour are relatively strong, but they are similar to other divisions in their pattern of 2009-13 change and the switch in correlations can be seen within these cases. It is also a phenomenon we have seen before at the 2009 European Parliament elections when the change in the UKIP share of the vote was much more strongly correlated with the Labour change than with that for the Conservatives. Note also that the correlations between Liberal Democrat and UKIP change are just as strong as for the other two parties, but the Lib Dems are reaching the point where it is hard for them to fall much further than they did in 2012. 

Local Elections 2013: Why did UKIP do so well?

With 23% on the PNS, and 147 seats, UKIP did shockingly well in this year's local elections. Why? 

There's lots of good analysis of UKIP voters, especially by Rob Ford.  Factors leading people to vote UKIP include anti-government and anti-main party protest voting, euroscepticism, and anti-immigrant sentiment. There is a lot of public support for the policies that UKIP advocate and emphasise. Polls tell us that there is a clear majority for reducing immigration and close to half the public would vote to leave the EU.

But public concern on these issues has been relatively long standing, and the main reasons for protest voting were there last year as well this year.  True, the economic doldrums have continued for yet another year and faith in the abilities of either the Conservatives or Labour to restore economic health has declined since last year, but not dramatically so. So why did UKIP do so much better this year than in any previous local elections?

One very important reason is that the party fielded more candidates. They contested three quarters of seats this year compared with just a third of seats last year and about a quarter in 2009, when the seats up yesterday were last contested. Looking at the 1550+ BBC keywards, in seats they fought both this year and in 2009, UKIP averaged 27%, up 11 points since 2009. In places they fought for the first time this year, they averaged 24%. So, roughly speaking, we can say that if they had only managed to match their 2009 performance but with 2013 candidature levels they would have averaged 15% instead of the 25% they actually averaged. Roughly 44% of the total UKIP vote tally this year is due to improved performance, but 37% is due to fielding more candidates at 2009 levels of support.  Put another way just under half (45%) of the UKIP increase in overall vote share since 2009 is due to fielding more candidates.

Admittedly this does raise the issue of how UKIP managed to attract so many more candidates. This could be partly due to increased overall popularity, but I'm not best placed to speculate.

So that leaves the question of why UKIP were up 11 points on 2009 in the places they did fight last time. Also in last year's local elections UKIP averaged just 12% in the 311 BBC keywards they contested, 13 points lower than this year's average. This is the same average rise as in the (albeit only 41) places were the county council division boundaries this year are the same as the ward boundaries for last year's elections.  

Polls suggest that UKIP support has risen steadily since the general election from 3% in 2010 to 11.5% now, with a 4% rise since last year.  A 4 point rise in the polls for UKIP provides some explanation for a 13 point increase in local election performance, but still leaves a lot to explain. 

The local election campaign over the last month probably has much to do with it.  UKIP seem to have run a much more effective campaign with many more activists who should help mobilise more voters. But the UKIP share and change in share of the vote is negatively not positively correlated with turnout across the BBC keywards.

Maybe more important received a lot more media coverage than ever before; and the debate over immigration from Bulgaria and Romania, which became the focus of the UKIP campaign, surely helped them.  Protest over wind farms helped mobilise support (but in a limited number of places) but disgruntlement with the planned HS2 route probably didn't make much difference.

What should the main the main parties do in response?  Bonnie Meguid (University of Rochester) tells us that the worst strategy to deal with single-issue or niche parties (like the extreme right and greens) is to try to argue against them. It only makes more people realise they agree with the niche party and not the mainstream parties. As Tim Bale observed after the Eastleigh by-election, by focusing on UKIP issues without addressing them satisfactorily, "rather than shooting UKIP's fox, Cameron has fed it." Better, according the Meguid's analysis, to ignore niche parties or steal their policies (if you can stomach them).

***
Thanks to Rob Ford, Jon Mellon and John Curtice for comments

Why didn't UKIP win so many seats given they got so many votes?

UKIP did extraordinarily well in yesterday's local elections, securing a projected national share of the vote of 23%. At the time of writing they have won over 120 council seats; much more than predicted but relatively few given the number of votes they won.

In the 1550+ BBC keywards Conservatives won 36% of the vote and 53% of the seats.  By comparison, UKIP won 21% of the vote but only 8% of the seats.  

In part this difference is due to a classic First Past The Post electoral system effect, but Labour managed to get 19% of the seats on 20% of the vote, and the LibDems got 14% of the seats on 13% of the vote among the BBC keywards.

The reason UKIP did particularly badly in translating votes to seats is because their vote share is still very evenly distributed by comparison with the three main parties in parliament. The histograms below show how the Conservatives vote shares across different divisions vary much more than UKIP.  More generally, whereas the standard deviation of the share of the vote across divisions for each of the three main parties was 15 percentage points, it was just 8 for UKIP.  This is a pattern that has been true in previous elections.

It seems then that UKIP suffer from what used to be called the 'yellow peril': coming second everywhere and first nowhere.  UKIP came first in 8% of BBC keywards but second 40%, more than any other party. Unless they resolve this problem they will have difficulty winning seats in a general election too. 

 

***

Thanks to BBC, John Curtice, Rob Ford and Jon Mellon for help with the data.

Local Elections 2013: PNS in historical perspective

The graph below shows the 2013 Projected National Share of the vote (PNS) from the local elections together with previous PNS figures.  It shows just how dramatic the change is.  This is the first time none of the three main parties in parliament have achieved more than 30%. The Liberal Democrat PNS is the worst on record and the Conservative PNS is equal to its previous worst showing in 1995. Labour's PNS is worse than at any local elections while they were in opposition in the 1980s and 1990s.

 

The previous UKIP figures are actually the Rallings & Thrasher NEV estimate of 10% for 2009 and the 2010 General Election share of 3%, (no estimate for UKIP in 2011).

****

Thanks to BBC and John Curtice for the data.

  

Local elections vote share (PNS) and general election vote intention

The graph below shows the BBC Projected National Share of the vote (PNS) from local elections together with general election vote intention from the polls for the month before each round of local elections.

There are some signs that they are tracking each other, but not that closely. (The correlations are about 0.7). There are clear systematic differences with Labour doing much better in the polls than in the PNS (with very big gaps during Blair's leadership up to the Iraq war: 1994-2003). Also, the Liberal Democrats have consistently done better in local elections than in the polls, except when local elections are on the same day as general elections.

The poll lines continue for 2013.  They show relatively little change since 2012, so perhaps there won't be much change in the PNS either today? But last year UKIP were only contesting around a third of seats and there wasn't so much hype in the media about them.

Stephen Fisher, 2nd May 2013

***
Thanks to Michael Thrasher and John Curtice for help with the data.

What do the polls tell us about local election results?

Stephen Fisher, 2nd May 2013

While some ask what local election results tell us about the state of the parties and prospects for the next general election, it is also important to consider how national party popularity affects the fortunes of candidates in local government elections.  Local elections are often said to be about local issues but actually most of the changes over time in shares of council seats won the Conservatives, Labour and Liberal Democrats can be accounted for by changes the popularity of these parties at the national level.

Just as in general elections, as the electoral pendulum swings so do the fortunes of councillors and would be councillors. Candidates from parties that are more popular in the opinion polls than they were four years ago are more likely to win than they were the previous time, and vice versa. Councillors that won marginal seats on a tide of national popularity for their party are most likely to lose them when the tide ebbs.

The graphs below show how changes in the shares of council seats won by each party are strongly correlated with changes in general election vote intentions. Indeed, roughly 60% of the fluctuations in local party performance can be explained by the opinion polls.

So what do the opinion polls suggest will happen in today's local elections? The seats being contested today were last up for election in 2009. Since then the Conservatives and the Liberal Democrats have dropped by 8 points each in the polls, while Labour are up 16 points. So we should expect to see Labour gain and coalition party losses.  

The table below shows predictions of the net numbers of council seat gains and losses from a statistical model which ignores a few outliers and adjusts for the worse performance of the Liberal Democrats relative to the polls when local and general elections coincide and since they joined the coalition. Otherwise the models rely solely on changes in opinion polls.

Estimate  Range         Rallings & Thrasher

Con -205 -115 to -295       -310

Lab +260 +160 to +355           +350

LD -190 -85 to -300       -130

The Colin Rallings and Michael Thrasher predictions (from the PSA briefing, 23.4.13) are shown for comparison.  The polls based model is not intended as a rival prediction model. Their method is much more thorough and uses information from local by-elections to estimate the National Equivalent Vote and pays attention to the actual shares of the vote in the different county council divisions last time. The polls model is 'just a bit of fun' as Peter Snow used to say. The prediction ranges (95% confidence intervals) from the polls model includes the Rallings and Thrasher predictions for Labour and the Liberal Democrats, but narrowly misses for the Conservatives.

I suspect both predictions for the Conservative losses are too low. Note from the graph above that 2009 was a year in which the Conservative local election performance was well in excess of what you would have expected given their standing in the opinion polls. On the same day as the 2009 local elections, UKIP were the second largest party in European elections. But the party didn't field many candidates in 2009, probably benefiting the Conservatives. Today UKIP are contesting about three quarters of council seats, and opinion polls suggest they may gain as much as 20% of the vote, mostly at the expense of the Conservatives. So the Tories are defending an unusually strong 2009 result and face a much greater challenge than can be accounted for by the opinion polls statistical model.  Conservative losses in the order of 400 seem more likely.    

TECHNICAL AND STATISTICAL DETAILS

The dependent variable for the models is the change in share of seats at a given election.  Without boundary changes this is simply the share of seats up that were won by the party minus the share they were defending (i.e. won four years ago). The share they were defending is defined as share of all seats that were either up for election or abolished, so the measure takes some account of boundary changes, but assumes that there was no instances of seats being abolished for territories without new elections. One potential problem with the measure may be if there are some councils who dramatically reduce/increase the number of seats while others at the same time don't and the pattern of reduction/increase is correlated with party performance. 

The explanatory variable is the four-year change (i.e. since the previous round of elections) in the opinion poll shares for the three main parties taken in the month before the local elections. There is no attempt to adjust for movement of elections out of the usual cycle (such as today's Anglesey elections, or the 2012 Scottish local elections that would traditionally have been held in 2011 but were postponed because of the Scottish parliament elections). These kind of things will increase the errors in the model.

The data cover the 1988 to 2012 period (with lagged information from 1984). Please excuse the Stata output for presentation of the regression models below (writing in haste!).  Notice that the outliers excluded were 2000 for Con and Lab, 2009 for Con and the first three years for the Liberals. The graphs above indicate that these were serious outliers and Root MSE for the models improves considerably with exclusion.  Also note that the LD model performs relatively poorly for the coalition years and on coincident general elections so these factors are included in that model. The R-squareds being around 0.6 are the basis for the claim above that 60% of the fluctuations in the fortunes of the main parties on council seats are due to changes in national popularity.  The Root MSE tells us that the models are on average 3 to 5 percentage points out on the change in share of the council seats won, which is a big average error for a prediction model when there are thousands of seats up for election.

regress CONchshseats chCONpollsh if !inlist(year,2000, 2009)

      Source |       SS       df       MS              Number of obs =      23

-------------+------------------------------           F(  1,    21) =   31.91

       Model |  1015.25331     1  1015.25331           Prob > F      =  0.0000

    Residual |  668.202418    21  31.8191628           R-squared     =  0.6031

-------------+------------------------------           Adj R-squared =  0.5842

       Total |  1683.45572    22  76.5207148           Root MSE      =  5.6408

------------------------------------------------------------------------------

CONchshseats |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]

-------------+----------------------------------------------------------------

 chCONpollsh |   1.129207   .1999083     5.65   0.000     .7134753     1.54494

       _cons |   .8228613   1.182931     0.70   0.494    -1.637178    3.282901

------------------------------------------------------------------------------

regress LABchshseats chLABpollsh if !inlist(year,2000)

     Source |       SS       df       MS              Number of obs =      24

-------------+------------------------------           F(  1,    22) =   39.38

       Model |  1157.81174     1  1157.81174           Prob > F      =  0.0000

    Residual |  646.766002    22  29.3984546           R-squared     =  0.6416

-------------+------------------------------           Adj R-squared =  0.6253

       Total |  1804.57774    23  78.4599017           Root MSE      =   5.422

------------------------------------------------------------------------------

LABchshseats |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]

-------------+----------------------------------------------------------------

 chLABpollsh |   .7133597   .1136716     6.28   0.000     .4776193    .9491001

       _cons |  -.4511028   1.106813    -0.41   0.688    -2.746492    1.844286

------------------------------------------------------------------------------

regress LDchshseats chLDpollsh coalition genelec if !inlist(year,1988,1989,1990)

      Source |       SS       df       MS              Number of obs =      22

-------------+------------------------------           F(  3,    18) =    9.30

       Model |  297.878906     3  99.2929686           Prob > F      =  0.0006

    Residual |  192.253791    18  10.6807662           R-squared     =  0.6078

-------------+------------------------------           Adj R-squared =  0.5424

       Total |  490.132697    21  23.3396523           Root MSE      =  3.2681

------------------------------------------------------------------------------

 LDchshseats |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]

-------------+----------------------------------------------------------------

  chLDpollsh |   .3009987   .1205108     2.50   0.022     .0478148    .5541825

   coalition |  -7.105657   2.750894    -2.58   0.019    -12.88507   -1.326244

     genelec |   -5.02947   1.828205    -2.75   0.013    -8.870386   -1.188553

       _cons |   1.405078   .8358295     1.68   0.110    -.3509345    3.161091

------------------------------------------------------------------------------

Predicted seats and actual seats won graphs are below.  These look much better because the dependent variable seems to be adjusting relatively well for changes in the number of seats. But notice that the gaps are in the hundreds sometimes. This is salutary reminder that the political significance of numbers of gains and losses depends very much on the numbers of seats up for election. The median absolute error on seats is 150 for Con and Lab and 97 for LD. There are some signs of better fit in recent years, but there are some spectacular failures with some fitted values being over 500 seats out.

***

Thanks to Michael Thrasher and David Cowling for help with the data, and to Michael Thrasher and John Curtice for comments on the models.

Local Elections Vote Shares: the BBC PNS and Rallings & Thrasher NEV compared

After repeating the mantra that local elections are local, the overall performances of the main parties at local elections are often discussed by journalists and others.  This commentary tends to focus on the numbers of seats and councils won and lost by each party. But these can be hard to interpret given the very different numbers of seats up for grabs each year and their different profiles at different points in the cycle. (See this article by Chris Game for an excellent discussion of the local elections cycle and this year's elections.) This year's local elections are overwhelmingly for county councils in England. Since these are the Conservative heartlands, the Tories could do very badly and still win the most seats and the most votes.  Indeed, in 2005, the Conservatives topped the shire council elections poll despite Labour winning the general election on the same day.

So perhaps the best indicators for what the local elections tell us about how the parties are faring generally are what the BBC calls the Projected National Share of the vote (PNS) and what Colin Rallings and Michael Thrasher in The Sunday Times call the National Equivalent share of the Vote (NEV).  Both of these essentially aim to be the same thing, an estimate of what the share of the vote would have been if (1) all parts of Great Britain had local elections and (2) the Conservatives, Labour and Liberal Democrats fielded candidates in all wards/divisions.  The idea of these conditions is to provide statistics that are broadly comparable with the circumstances in general elections. The graph shows how the PNS and NEV have changed each year since the early 1980s.

They basically track each other very closely and so tell the same story of major party performance in British local elections.  For the Conservatives and Labour it is possible to link some of the highs and lows with their popularity as measured by general election vote intention in the opinion polls, but there are also periods when the patterns diverge. For instance, in the late 1990s and early 2000s Labour's lead over the Tories in the polls did not produce dominance in the local elections. (They did better in the polls than at the polls.) Meanwhile, since the late 1980s the Liberal Democrats (and predecessors) have consistently done better in local elections than at general elections or in the opinion polls, even with their recent downturn. 

The rise of other parties over the past decade has been a key development, and this year much of the attention of commentators will be paid to UKIP. The BBC hasn't published a PNS for UKIP before but will estimate one this year. The key methodological issue with other parties and the PNS/NEV is what to do about their level of candidature. In 2009 UKIP came second in the European elections held on the same day as in local elections, but they only fielded candidates in a quarter of seats where the three main parties also stood. This year their contestation rate, in places where the Conservatives, Labour and LibDems are also competing, is over 80%, much closer to the 88% candidature rate they had in the 2010 general election. Looking back to 2009, Rallings and Thrasher now estimate that UKIP were on 10% in the NEV. 

The main method for estimating the PNS is to look at change since one or more previous elections in wards fought by all three main parties both times.  For good measure various different baseline years can be used (both the previous times the wards were fought and previous times the particular seats were contested). Various different levels of aggregation are considered and it is wise to check whether there are systematic differences between different kinds of places in the pattern of change and perhaps weight wards differently to achieve redress imbalances. When local elections coincide with general elections it is possible to compare the local election results with the corresponding (groups of) constituency results. These provide the most reliable baselines, and so can be used to revise historical figures. Social scientists often talk about the virtues of triangulation and robustness checks but I've rarely seen any research do as much as typically goes into the PNS estimates.

Given the lack of a single definitive consistently applied method for calculating the PNS/NEV it is hardly surprising that there should be differences between the two series from time to time. Although politicians on election night may be tempted to pick the historical comparisons that are most flattering to their parties, there are two main things to bare in mind: the PNS and NEV rarely differ by much, and if you want to understand whether the parties are doing better or worse than in previous years, compare PNS with PNS or NEV with NEV, but not PNS with NEV or vice versa. 

by Stephen Fisher, 1st May 2013

***

Thanks to John Curtice, Michael Thrasher and Colin Rallings for comments and help with the data.