Pollster.com's Mark Blumenthal: Polling on the Dark Side of the Moon (re the ARG poll)

[Pirate Smile]

December 26, 2007
Polling on the Dark Side of the Moon

-snip-
Right now, with interest peaking in Iowa and New Hampshire polls, we have entered into our own dark side of the moon period. Unlike the moon missions, however, we are not in a total blackout. At least one pollster has released an Iowa survey conducted over the weekend, and others will surely follow later this week. However, with so many Americans traveling away from home for holiday travel, those surveys will be of unknown reliability, at best. The worst-case scenario (for the pollsters) will be if these surveys indicate a false trend, a shift that is less an indicator of real change than an artifact of respondents missing due to holiday travel.

-snip-
The reason that pollsters typically avoid polling around the holidays is the assumption that a big chunk of the population is away from home and unavailable for survey calls. As with any sort of "non-response" problem, we risk getting skewed or biased results if the missing respondents are both numerous and different in terms of their political views from those at home when we call.


What kinds of voters might be missing right now? Three years ago, in a survey concluded a full week before the holidays (12/15-17/2004), the Gallup Organization asked a national sample of 1,002 adults whether they planned "to travel more than fifty miles from home this holiday season." Twenty-eight percent (28%) said "yes." More important, as the table below indicates (based on data drawn from the Roper Archives), those planning holiday travel had a very distinctive demographic profile. Holiday travelers were much more likely to be younger and better educated. Notice also that holiday travelers were not just college students. Adults between the ages of 30 and 44 are twice as likely to travel for the holidays than those over 65. (Also, while I do not show it here, the pattern in these results by age, education and income was nearly identical for Democratic and Republican identifiers).



So pollsters fielding surveys this week are going to have a harder time finding younger, better educated respondents. Why is that important? Because in the Democratic race, at least, there are huge differences in vote preference by age and education: In virtually every survey, including those in Iowa and New Hampshire, Barack Obama does best among younger, better educated voters while Hillary Clinton's base of support is older and less well educated. Consider the data from the Boston Globe/University of New Hampshire survey conducted last week: Obama wins the support of 47% of voters age 18 to 34, but only 22% of those over 65; 40% of those with graduate degrees but only 25% of those without a college degree.




-snip-
Do we have any evidence that holiday travel bias might affect vote preference? Maybe. Consider the data below reported by Rasmussen Reports just before and after the Thanksgiving holiday.

Rasmussen runs a daily, rolling-average national survey that tracks presidential primary vote preference. Rasmussen uses an automated interactive-voice-response (IVR) methodology in which respondents answer recorded questions by pressing keys on their touch-tone phones. Each night, they call roughly 175-200 likely Democratic primary voters and roughly 150-160 likely Republican voters, then roll together and report a rolling average of the last four nights of interviews. Thus, each of the points on the chart below represents 750-800 likely Democratic voters with a reported margin of sampling error of +/-4%. The trend line is a regression estimate that Charles Franklin created for me using the Rasmussen data.



Notice what happens to the Obama trend line just before and just after Rasmussen took a five-day break from interviewing, Wednesday through Sunday, over the Thanksgiving weekend (Thanksgiving was November 22). The regression trend line essentially splits the difference between the 17% low points immediately before and after the Thanksgiving break and the much higher 26% results that came just before and just after that. What one makes of the variation may be in the eye of the beholder. Either the Obama trend got unusually erratic in both directions during the last two weeks of November, or there was a very unusual and precipitous plunge from what should have been a plateau around 24-26 to 17% centered on the period of Sunday, Monday and Tuesday before Thanksgiving and the Monday just after. Either way, the Obama variation around Thanksgiving was highly statistically significant. It did not occur by chance alone. Either there was a see-saw in Obama's real world support that week, or something changed in the kinds of voters Rasmussen sampled.


Given what we know about the demographics of holiday travelers and Obama's supporters, I'd bet on the latter.

-snip-
The hard truth is that we are behind the dark side of the moon this week, and we may not know much with certainty until next Wednesday night.

-- Mark Blumenthal

http://www.pollster.com/blogs/polling_on_the_dark_side_of_th.php

[Pirate Smile]

1. kick