Sunday, September 19, 2004
That's why they actually play the games
Do presidential polls give us meaningful information?
[Politics]
We’ve all heard this one. The underdog sports team pulls off a huge upset victory, defying all the predictions. The laconic coach says, "That’s why we actually play the games."
According to the recent headline-grabbing Gallup poll, completed on September 15, Bush leads Kerry by 13 points, with 55 to 42 %. We might as well not bother to hold the election.
However, the fact is that other polls in the same time frame show the race much closer. Democracy Corps (completed one day before Gallup) shows Bush up by only 1 point (49-48) and IBD/CSM/TIPP (3 days older than Gallup) shows the race dead even at 47. Pew (two days older) shows Bush up by one point and Harris (two days older) shows Kerry up by one point. These are all polls of "likely voters" and ask the same question, "If the election were being held today..."
What explains this divergence of 14 points one way or the other? What's going on? I talked to a couple of our sociology colleagues and heard some interesting theories. Apparently, the screening criteria for likely voters varies considerably from poll to poll. Criteria include the respondent's own report of likelihood of voting, whether they voted in the past X number of presidential elections and other criteria. Apparently, it is the more restrictive criteria that result in polls favoring Bush; these therefore tend to undercount the preferences of new or newly energized voters. Other theories suggest that polling accuracy today is increasingly undermined by telephone behavior changes, such as people being more likely to reject telemarketing-type phone calls and the emergence of primary reliance on cell phones, which presumably are not called by pollsters.
I wondered whether vastly diverging polls are unusual, and so I checked a couple of web sites containing a few presidential polls from 2000. We all know what happened then: Gore won by a razor thin margin, and became president... Well, you know what I mean. A quick snapshot suggests that different polls on the same dates diverged in 2000, but by about half as much as this year's polls. In mid October 2000, most polls showed Bush with 48%, and leading Gore by 4-6 points. Democracy Corps showed Gore ahead by 2 points, and CBS showed the race dead even. So it seems that there is more "interpoll"divergence in the 2004 race (13 points) than the 2000 race (8 points).
What is also interesting (and I did not remember this) is how much swing there was in polls throughout October. Indeed, check this out:
The Gallup/CNN/USA Today poll on October 10/27/2000, less than a week before the election, reported: Bush 52, Gore 39 !!!
CNN/Time Polls of likely voters, on 10/27/2000: Bush 49, Gore 43 (!!!)
Election eve polls in 2000 showed that the race had tightened up a great deal, yet I do not recall any major event that changed the election dynamics against Bush in 2000. By the way, 7 of the 10 major election eve polls predicted that Bush would win the popular vote by 2-5 points.
Another factor that is underplayed in the reports on polls that are scaring Kerry supporters is the persuadability of poll respondents. Here's an interesting 2000 poll result, also from CNN/Time:
Q: Is your mind made up?
A: October 25-26: Yes 87%, No 12%
October 4-5: Yes 83%, No 17%
September 5-6: Yes 77%, No 21%
That's a lot of persuadability one week before the election, and a ton of persuadability in September, particularly in a close race. This year more minds are made up, but there is still a significant amount of persuadability -- 16 % of registered voters, which may well be enough to tip the election. I believe that many "if election were held today" polls include "leaners." I believe that Gallup counts voters "leaning" toward Bush as Bush votes. Yet a recent poll by Annenberg Center for Public Policy shows that among "persuadable" voters (people who report that they are either undecided or likely to change their mind by election day), Bush has lost ground since August.
Other studies of presidential vote patterns have shown that undecideds tend to break against the incumbent by about 2-1.The media loves to report "trial heat" polls (who would you vote for today), because they sell papers, but do they really give us any meaningful information? News outlets report the polls, and commentators wring their hands over what "causes" poll numbers to be the way they are. But none of our pundits tries to analyze how closely correlated a verbal poll response is to actual voting behavior. Remember how Nader in 2000 ended up with less than 3% (2.74) of the vote when we all expected him to do better? Election eve polls showed him getting 3-7%; no poll understated Nader's vote, and nearly all overstated it. One theory I heard is that voters in "battleground states," many votes who said all along they would vote for Nader got into the voting booth and just couldn't bring themselves to help Bush in that way -- and switched to Gore. Might there be such a phenomenon in which Bush leaners switch to Kerry at the last minute?
What conclusions are to be drawn from this? Are there demographic shifts that have undermined polling reliability in a way that polling techniques have yet to adjust to? Are presidential opinion polls farther out in time than a week away simply a poor predictor of the outcome in a close race? Are they merely suggestive tools that require a great deal more sophistication in interpretation than the media gives them? Are they pseudo-social science?
Remember Dewey v. Truman?
That’s why they actually hold the elections.
[Politics]
We’ve all heard this one. The underdog sports team pulls off a huge upset victory, defying all the predictions. The laconic coach says, "That’s why we actually play the games."
According to the recent headline-grabbing Gallup poll, completed on September 15, Bush leads Kerry by 13 points, with 55 to 42 %. We might as well not bother to hold the election.
However, the fact is that other polls in the same time frame show the race much closer. Democracy Corps (completed one day before Gallup) shows Bush up by only 1 point (49-48) and IBD/CSM/TIPP (3 days older than Gallup) shows the race dead even at 47. Pew (two days older) shows Bush up by one point and Harris (two days older) shows Kerry up by one point. These are all polls of "likely voters" and ask the same question, "If the election were being held today..."
What explains this divergence of 14 points one way or the other? What's going on? I talked to a couple of our sociology colleagues and heard some interesting theories. Apparently, the screening criteria for likely voters varies considerably from poll to poll. Criteria include the respondent's own report of likelihood of voting, whether they voted in the past X number of presidential elections and other criteria. Apparently, it is the more restrictive criteria that result in polls favoring Bush; these therefore tend to undercount the preferences of new or newly energized voters. Other theories suggest that polling accuracy today is increasingly undermined by telephone behavior changes, such as people being more likely to reject telemarketing-type phone calls and the emergence of primary reliance on cell phones, which presumably are not called by pollsters.
I wondered whether vastly diverging polls are unusual, and so I checked a couple of web sites containing a few presidential polls from 2000. We all know what happened then: Gore won by a razor thin margin, and became president... Well, you know what I mean. A quick snapshot suggests that different polls on the same dates diverged in 2000, but by about half as much as this year's polls. In mid October 2000, most polls showed Bush with 48%, and leading Gore by 4-6 points. Democracy Corps showed Gore ahead by 2 points, and CBS showed the race dead even. So it seems that there is more "interpoll"divergence in the 2004 race (13 points) than the 2000 race (8 points).
What is also interesting (and I did not remember this) is how much swing there was in polls throughout October. Indeed, check this out:
The Gallup/CNN/USA Today poll on October 10/27/2000, less than a week before the election, reported: Bush 52, Gore 39 !!!
CNN/Time Polls of likely voters, on 10/27/2000: Bush 49, Gore 43 (!!!)
Election eve polls in 2000 showed that the race had tightened up a great deal, yet I do not recall any major event that changed the election dynamics against Bush in 2000. By the way, 7 of the 10 major election eve polls predicted that Bush would win the popular vote by 2-5 points.
Another factor that is underplayed in the reports on polls that are scaring Kerry supporters is the persuadability of poll respondents. Here's an interesting 2000 poll result, also from CNN/Time:
Q: Is your mind made up?
A: October 25-26: Yes 87%, No 12%
October 4-5: Yes 83%, No 17%
September 5-6: Yes 77%, No 21%
That's a lot of persuadability one week before the election, and a ton of persuadability in September, particularly in a close race. This year more minds are made up, but there is still a significant amount of persuadability -- 16 % of registered voters, which may well be enough to tip the election. I believe that many "if election were held today" polls include "leaners." I believe that Gallup counts voters "leaning" toward Bush as Bush votes. Yet a recent poll by Annenberg Center for Public Policy shows that among "persuadable" voters (people who report that they are either undecided or likely to change their mind by election day), Bush has lost ground since August.
Other studies of presidential vote patterns have shown that undecideds tend to break against the incumbent by about 2-1.The media loves to report "trial heat" polls (who would you vote for today), because they sell papers, but do they really give us any meaningful information? News outlets report the polls, and commentators wring their hands over what "causes" poll numbers to be the way they are. But none of our pundits tries to analyze how closely correlated a verbal poll response is to actual voting behavior. Remember how Nader in 2000 ended up with less than 3% (2.74) of the vote when we all expected him to do better? Election eve polls showed him getting 3-7%; no poll understated Nader's vote, and nearly all overstated it. One theory I heard is that voters in "battleground states," many votes who said all along they would vote for Nader got into the voting booth and just couldn't bring themselves to help Bush in that way -- and switched to Gore. Might there be such a phenomenon in which Bush leaners switch to Kerry at the last minute?
What conclusions are to be drawn from this? Are there demographic shifts that have undermined polling reliability in a way that polling techniques have yet to adjust to? Are presidential opinion polls farther out in time than a week away simply a poor predictor of the outcome in a close race? Are they merely suggestive tools that require a great deal more sophistication in interpretation than the media gives them? Are they pseudo-social science?
Remember Dewey v. Truman?
That’s why they actually hold the elections.
Subscribe to Posts [Atom]
