politics

A Poll of Polls

2020/11/19

As a company that exists to make the uncertain a little more certain, we rely heavily upon survey research (e.g., Trendlines) to make sense of the world. Our perceptions of others’ thoughts and opinions serve as benchmarks in forming our own.

So, when results from Florida on election night revealed that Floridians had no plans to follow polling projections, the science of polling suffered a bruising trust fall. And with good reason: 50% of Americans closely followed the election polls. A mere 5% didn’t pay any attention.

 

Yet even after 2016’s polling oopsie, only 19% of Americans deeply questioned the accuracy of the pre-election 2020 polls. Given the high stakes in ensuring public trust in research of any kind — but specifically survey research — we wanted to offer a few “Consumer Protection Warnings” for those anxious poll addicts out there.

  • Polls are not created equal. There are highly reputable, deeply experienced pollsters that draw random samples of likely voters from across the country. These happen to be incredibly expensive and used sparingly, especially in state-level election polling. Most election polls rely on demographic quota-based sampling methods that assume everyone has the same probability of being selected — but this is not the case. Just like millennials who scrutinize every gluten-free, dairy-free, taste-free label at the grocery store, we should be scrutinizing the methodology of each poll.
  • Predicting an American election is hard. Doing it during an out-of-control pandemic is even harder. With 50 states, we’ve got a lot of middle children who each want to do their own thing. And it doesn’t help that voter registration guidelines change at the whim of a judge. Unlike the surveys we feature in Trendlines (which target all American adults), the target population of election polls – likely voters — cannot be known before Election Week. So a small error in sampling the “right” likely voters can have an outsized effect on the projection.
  • Not everyone replies to polls. Even if you think you’ve designed a way to capture every type of diversity and experience in your sample of likely voters, some are going to slam the proverbial door in your face right away. When this happens systematically — meaning people with similar characteristics slam doors at consistently higher rates than others — the poll will under represent their views. We find that voters who have strong party allegiance are the most likely to follow polls (74% of self-identified 'strong democrats' and 62% of 'strong republicans'), suggesting they may be more willing to participate in polls when asked.

 

Election polling is a lot of science with a sprinkle of art. That leaves poll consumers with the challenge of examining the data under a microscope, yet embracing the complexity. Still confused? Reach out, we've got your back.