The Ontarian election, the death of the Likely voters models?
Coup d'oeil sur cet article
A little bit over a year ago, the polling industry in this country was coming off two huge failures in Alberta and BC. While some of them went for the cop-out excuse of saying that people changed their mind, many actually decided to improve their methodology. The BC election in particular seemed to indicate that with low turnout, you couldn't simply take the poll numbers at face value. We needed what we call "likely voters" models, similar to what is done in the States. Since some people are more likely and more committed to vote, it made sense to account for this. Think of it of weighting more the people more likely to vote (older, more educated, more motivated, etc)
And then we got the Ontario election. While most pollsters agreed that PC voters were more committed (and therefore the likely voters adjustments would help the Tories), we got the exact opposite. The Liberals performed mostly in line with expectations while the Conservatives barely received 31% of the vote. Much lower than the 35-36% from the polls among likely voters.
The table below shows all the polls released during the last week. For firms with a LV model (Ekos, Abacus, Ipsos and Angus-Reid), I included both sets of numbers. For the calculations of the margins of error, I kept the same sample sizes even though technically, the likely voters poll is less (but not all pollsters are transparent as to exactly how many respondents they lose while moving from decided to likely voters). I use two measures to evaluate the polls. The first one is the average absolute deviation. So if the PC was in 36% and got 31%, it's a difference of 5 points. This measure is very common but potentially depends on luck. We shouldn't necessarily expect pollsters to nail all four numbers, but they should have them within their margins of error. This is why our second measure counts how many parties were within the MoE for each poll.
|Firms||Method||Likely Voters?||Sample size||LIB||PC||NDP||Green||Average absolute deviation||# Party within MoE|
|Too Close To Call (adjusted average)||37.3||36.0||22.1||3.7||2.2|
Overall, no pollster did really well. In particular, nobody has more than 2 parties within its margins of error. It was like the ones who got the Liberals right then got another one completely wrong.
The best pollster was Angus-Reid, but not their LV numbers. Then we have Abacus (non-LV as well, althought the LV numbers weren't too bad). I said that to me, Abacus has the best LV model and I was right (although here, it's more a question of being less wrong). Ekos has the merit of being the only one to have the main two parties within its MOE (for its non-LV numbers) but its LV numbers missed all four (at least the LV model had that the Liberals wouldn't go down). Worse one is Ipsos (LV) which means this firm has finished last in two consecutive elections (they were last in Quebec). As for my own average, beside the fact that I shouldn't have used the LV numbers, I adjusted the Green down too much. Probably because since the Green were already lower because of the use of LV models, I didn't need to bring them down even more. I'll remember that.
I'll take a look at my projections in the coming weeks. What I know is that if you had entered the correct percentages in the simulator, you'd had got the results almost perfectly (56-30-21 to be exact). But pollsters will have to provide answer to some questions because this election wasn't particularly good for them and their likely voters models.
Mon simulateur, avec les résultats de l'élection, m'aurait donné:
57 OLP 31 PC 19 NDP
Ce qui n'est pas si mal. J'ai découvert que certaines des erreurs sont dues à des circonscription ayant eu des élections partielles depuis la dernière élection générale et que je n'avais pas inclus ces chiffres. Je devrai donc corriger cette situation et commencer à inclure ces résultats.