Author Topic: Estimation of the cell phone effect on polls: 2.2 percent  (Read 401 times)

240 is Back

  • Getbig V
  • *****
  • Posts: 102396
  • Complete website for only $300- www.300website.com
Estimation of the cell phone effect on polls: 2.2 percent
« on: September 20, 2008, 08:52:44 AM »
Mark Blumenthal has a rundown of the pollsters that are including cellphone numbers in their samples. Apparently, Pew, Gallup, USA Today/Gallup (which I consider a separate survey), CBS/NYT and Time/SRBI have been polling cellphones all year. NBC/WSJ, ABC/Washington Post and the AP/GfK poll have also recently initiated the practice. So too does the Field Poll in California, PPIC, also based in California, and Ann Selzer. There may be some others too but those are the ones that I am aware of.

Let's look at the house effects for these polls -- that is, how much the polls have tended to lean toward one candidate or another. These are fairly straightforward to calculate, via the process described here. Essentially, we take the average result from the poll and compare it to other polls of that state (treating the US as a 'state') after adjusting the result based on the national trendline.

Since ABC, NBC/WSJ and AP/GfK all just recently began using cellphones, we will ignore their data for now. We will also throw out the data from three Internet-based pollsters, Zogby Interactive, Economist/YouGov, and Harris Interactive. This leaves us with a control group of 36 pollsters that have conducted at least three general election polls this year, either at the state or national level.

Pollster n Lean
========= ====
Selzer 5 D +7.8
CBS/NYT 14 D +3.7
Pew 7 D +3.4
Field Poll 4 D +2.8
Time/SRBI 3 D +2.4
USA Today/Gallup 11 D +0.4

Gallup 184 R +0.6
PPIC 4 R +1.3

AVERAGE D +2.3

CONTROL GROUP (36 Pollsters) D +0.1

Six of the eight cellphone-friendly pollsters have had a Democratic (Obama) lean, and in several cases it has been substantial. On average, they had a house effect of Obama +2.3. By comparison, the control group had a house effect of Obama +0.1 (**), so this would imply that including a cellphone sample improves Obama's numbers by 2.2 points. (Or, framed more properly, failing to include cellphones hurts Obama's numbers by approximately 2 points).

The difference is statistically significant at the 95 percent confidence level. Perhaps not coincidentally, Gallup, Pew and ABC/WaPo have each found a cellphone effect of between 1-3 points when they have conducted experiments involving polling with and without a cellphone supplement.

A difference of 2 points may not be a big deal in certain survey applications such as market research, but in polling a tight presidential race it makes a big difference. If I re-run today's numbers but add 2.2 points to Obama's margin in each non-cellphone poll, his win percentage shoots up from 71.5 percent to 78.5 percent, and he goes from 303.1 electoral votes to 318.5. (The difference would be more pronounced still if Obama hadn't already moved ahead of McCain by a decent margin on our projections).



So this is my plea to pollsters: let's get it right. Perhaps the cellphone effect will prove to be a mirage after all, but that's something for the data to determine on its own, rather than the pollster.



(**) Keen observers will wonder why the average house effect is greater than zero. This is because in determining our house effect coefficients, we weight based on how many polls each pollster has conducted. A couple of pollsters that account for a large proportion of our data, like Rasmussen and ARG, have had slight (very slight, but enough to skew the numbers) GOP leans.

http://www.fivethirtyeight.com /

That is big news. If he is right and pollsters are underestimating Obama's actual standings in the polls by 2 percent, that is huge if it shows up on Election Day.