As we enter the final week of the election, a slew of last minute polls that will emerge. This is a good time to remind ourselves that we should not put too much stock in what they say. As I said in an earlier post, pollsters have to make adjustments to the raw data and this introduces systematic uncertainties so that the actual margin of error could be about double the statistical one.

Josh Clinton has done an interesting analysis to try and get a better idea of how much these adjustments can affect the results.

He says that pollsters have to address four questions.

After poll data are collected, pollsters must assess whether they need to adjust or “weight” the data to address the very real possibility that the people who took the poll differ from those who did not. This involves answering four questions:

1. Do respondents match the electorate demographically in terms of sex, age, education, race, etc.? (This was a problem in 2016.)
2. Do respondents match the electorate politically after the sample is adjusted by demographic factors? (This was the problem in 2020.)
3. Which respondents will vote?
4. Should the pollster trust the data?

To show how the answers to these questions can affect poll results, I use a national survey conducted from October 7 – 14, 2024. The sample included 1,924 self-reported registered voters drawn from an online, high-quality panel commonly used in academic and commercial work.

He shows that depending on the kinds of adjustments that the pollsters make to that same set of raw data, that can result in large swings in the results. He concludes:

Even though many people complain about how inaccurate preelection polls can be, it is actually astounding that the polls are as accurate as they are given how many choices a pollster must make.

I’ve shown that reasonable choices about how to weight a poll can produce up to an 8-point shift in the Harris-Trump margin. That’s a larger number than the “margin of error” and the expected margin in most battleground states. In a close election like this one, a pollster’s choices can radically alter a poll’s results.

Yet another reminder that we should not be getting agitated over small changes in polls over time or small differences between polls.