Earlier this year, the polls in England all but assured that the referendum on leaving the European Union, dubbed “Brexit”, would be defeated. Similarly, almost all U.S. election polls predicted a Hillary Clinton victory in the race for President. Both were staggeringly wrong.
Why Polls Fail
Most people don’t understand how polls actually work. Statistically, counting a sample of a larger collection is a valid way of counting the whole, provided both the following are accurate:
- The size of the sample relative to the size of the total
- The sample is a suitably representative selection of the total
The problem is that while both items can be known for fixed sets of data, in a variable data set, such as U.S. voters, both the above must be estimated in ways that can be incredibly inaccurate.
The size of the electorate depends on how many people actually end up voting. Polls try to account for this by surveying different kinds of voters. Some polls survey “registered voters,” while others survey “likely voters.” While registered voters is a real fact, likely voters requires some sort of guess, such as, having voted in the last three elections.
Getting a suitably representative sample is also difficult. It would be relatively trivial to ask each of your neighbors who they were going to vote for, but that sample wouldn’t be representative of the whole. You all live in the same area, and would ostensibly all be of the same socioeconomic background. Likewise, calling 500 voters in rural Nebraska would give a sample that was too conservative and rural in composition.
Deciding what makes a representative sample is where many polling errors happen. In general, pollsters work from the last election, and try and then try and “correct” for changes in the electorate. So, you would want to poll people from all 50 states, where you call some conservatives, some liberals, and some independents. Make enough calls, from the right pool, and you come close to predicting the election, or so the thinking goes.
Polling In Modern Elections
There are several issues that are creating inaccuracy in recent polls. In both the Brexit vote, and the U.S. Presidential election, it appears that pollsters underestimated the number of rural voters that would participate in the vote. In the U.S., this is unsurprising, considering that turnout was a record.
The other issue is that fewer people tend to speak with pollsters. In fact, polls are becoming more expensive, and taking longer, because so many people won’t participate.
In addition to the number of people who decline to answer questions, is the even larger number of people who won’t even answer the phone. Some voters, like me, never answer the phone from an unknown number. Instead, I let those calls go to voicemail and then return them if I think it warrants it. A pollster, then, would find it impossible to count my vote. Some polls do not call cell phone numbers, again potentially removing and entire group of people from the possible poll. Many Americans, again like me, don’t have landlines, inadvertently removing them from the pool.
There is the very real possibility that this lack of participation is creating a sample that is something less than random. Instead of a random survey of all U.S. voters, it is a random survey of all U.S. voters that answer calls from unknown numbers on landlines. The less random the constructed pool is, the less accurate the poll will be.
Future of Election Polls
Polls are not likely going away any time soon. They give newspapers and news networks a way to fill precious air time, and people love following along to see how their candidate is doing.
However, it is time to face the reality that the standard construction and execution of election polls is becoming less accurate. To be anything that well respected crystal balls, election surveys are going to need to develop new tactics.