I don't believe that Opinion Polls are Scientifically Valid

[LosinIt]

OK, hear me out. I know that statistically you can take a representative sample of widgets and measure a property of theirs, say length or weight, and then extrapolate out to a larger sample. That is a proven fact. But, I think with people's opinions, it just doesn't hold true. There are just too many factors that go into the way someone feels about something. As an example, my mother is a Republican, as is my brother. My dad died a long time ago, so I am not really sure what his politics were, but I would guess Repug because I remember he thought that Nelson Rockefeller walked on water. So, demographically, I should have the same opinion as these three people but, voila, here I am a rabid member of DU.

In fact, I think that it may sometimes work the other way. A poll comes out and says that 90% of Americans think that black socks are just the coolest. So, those that want to be cool, don the black socks. That isn't a very good example, but do you get what I am saying?

[bowens43]

1. they were pretty close last Tuesday.

They picked all the winners.

[LosinIt]

2. Or did they tell everyone who voted how to be a winner?

that is my point.

[muriel_volestrangler]

3. Polls don't work by assuming that everyone with the same background

votes the same way. They have to sample enough that they catch all the significant variations - in effect, they shouldn't be extrapolating, but interpolating. So I don't think your example of your family would be relevant - the polls should ask enough families that for every 3 people they find like your relatives, they find one like you.

Yes, some people may be influenced by polls. These are also the people who'll be influenced by a newspaper article telling them what's 'cool' - probably more dangerous, in voting terms, than polls. I don't know how you'd prove where the influences come from, though.

One problem the polls in the UK had in the early 90s was shame. People were ashamed to admit, even to strangers over the phone, that they'd vote Conservative. So all the polls underestimated the vote they actually got in the real election. Some say this wasn't conscious dishonesty - that people, when asked, really did think they'd vote Labour (or whatever), but just got selfish when faced with the voting slip in the privacy of the booth.

[LosinIt]

4. Good point, probably why the caucus system isn't the best

people can't be true to themselves when their choice may set them up to ridicule from others. The privacy of a voting booth is a different story.

[buckeye1]

5. Pollsters agree.

With so many cell phones,answering machines,caller ID and the universal disdain for unwanted callers, polls have suffered. Think for yourself.

The voters decide.

[SheilaT]

6. Done right, with the right kind of sampling,

polls are extremely accurate. When they are wrong, it will usually be because they didn't capture the right sample. And there are lots of different reasons why that can happen, and the reasons change over time.

The classic is the 1936 Literary Digest presidential poll. It showed that Alf Landon would win the election a huge majority. He didn't. The problem with the poll was that the Literary Digest had mailed out some 10 million questionnaires to people who owned cars or had telephones, and more than 2 million mailed them back. Even with such a huge sample, the poll was way off because far fewer people owned cars or had telephones back then, and those people were more likely to vote Republican.

Interestingly enough, a man named George Gallup polled only 50,000 people and correctly predicted FDR would be re-elected. And he was, with 62% of the vote. http://aurora.wells.edu/~srs/Math151-Fall02/Litdigest.htm

Right now some of the polls are getting it wrong because of the growing usage of cell phones, and the no-call lists, and caller ID, all of which change the ability of pollsters to get the sample they need.

Another issue is so-called push polling, and other dishonest means of influencing the results. Even in a good poll, not enough alternatives may be offered to assess opinions accurately. I often participate in consumer polls, including taste or product testing, and sometimes I'm stymied by not having a check off option that is my real opinion.

[Peregrine]

7. I question their validity based on the sample sizes

they are using these days. Most polls I see have sample sizes around 700. I think a sample size of 700 from a target demographic of about 50 to 70 million is a bit small and would be prone to sampling error -- I laugh at their margin of error "+-3%."

I just wonder who decided that less than 1000 was a good number.

[ProfessorGAC]

8. LaPlace Decided It

The sampling is based upon Laplacian sampling equations. If the population were truly homogenous, then the pure Laplacian values would provide whatever margin of error was picked ahead of time. (One has to decide the confidence interval, or risk of being wrong, ahead of calculating the sample size.)

The problem is not just the sample size, however. The problem is in the determination of the representative strata for the stratified random sampling that is done.

I know people in the social statistics field and we've had these discussions on a theoretical level. Most of them realize that the sample size within each strata is too small to smooth out the possibility that one person lies about preference, another about affiliation, another about being a "likely voter", etc. They know it, but the larger the size of each stratum, the larger the overall k, which means more money to conduct each poll.

So, the polls are best taken as an overall average of several polls. This, by mathematical definition, does two things: It enlarges the overall k, (5 polls times 1000 people/poll = k = 5000 people) and it reduces the probable error of each poll by the square root of the number of polls. So, in my case, i use the values from 6 different polls for approval ratings and re-elect numbers. The overall error is 1/2.44 that of any individual poll, and the overall sample size averages 6 times (obviously) that of the average of all 6, and about 4.7 times that of the largest single sample size.

The problem isn't the mathematics. It's the implementation of the stratification techniques given the cost limitations of the companies running the polls.
The Professor