Statisticians??? How can 787 be an adequate sample???

[CatWoman]

Explain, please!!!!

Who the hell are they sampling???????

The poll of 787 registered voters used a model for probable voters that assumes a relatively high 50 percent turnout among the state's voting age population. It had a margin of error of plus or minus 4 percentage points, and was conducted Thursday through Saturday, CNN said.


http://www.salon.com/news/wire/2003/09/28/calif_poll/index.html

[arwalden]

1. You're On To Something, Cat. --- This is yet ANOTHER reason...

... why even the "scientific" polls are complete bullshit.

[CatWoman]

10. LONG LIVE THE DUCK

:hi:

:loveya:

[papau]

14. Stratified sampling does work - but is dependent on the model used

to stratified the population (The university types and the Census Bureau folks worked out the math more than 50 years ago).

If the model is correct, half of the 787 would give a reasonable answer. If the model is incorrect, 10 times the 787 would not give a reasonable answer.

Zogby's model was the best in 00 and 02 - but that does not mean it will be the best in 04 - but that is the way to bet!

:-)

[T Roosevelt]

2. From a stat theory standpoint

that number of samples should be sufficient. The problem arises in the way the questions are phrased, asked, in what order, etc, as well as when and where the samples are taken. Unless the pollster acknowledges the bias of localities one way or the other, the poll is going to be skewed.

Of course we all know polls are BS anyway...

[Booberdawg]

3. I don't know about the sampling but ...

Here's a quote from the article:

Davis's campaign spokesman, Peter Ragone, attacked the numbers.

"It's a joke," Ragone said. "It is so far from what every other public poll and every other internal poll by both Democrats and Republicans have found."

I don't understand how these numbers could change so drastically so quickly either. It's been bugging me. There was a spokesman on MSNBC this morning, don't know if it is the same Davis campaign spokesman or not, but he was also disputing these results. Then of course there was a Republican spokesman practically creaming his jeans over it and beaming through the whole segment.

I am beginning to wonder myself if this is just a crock of shit. I'd rather see a Zogby poll.

[T Roosevelt]

6. That's the thing

If all the people polled were from Orange County (strongly Rep) you'd get these kind of results. OTOH, make the calls in San Fran, and Ahhhnold would be at the bottom. Without knowing anything about the sample population, the results are meaningless. Pretty much goes for all polls...

[im4edwards]

4. thats why there is always a plus/minus factor

but the quantity is OK. 1500 would be better but if its the right 800 its enough.

[emad]

7. My quantitative methods tutor drummed this into me:

"there are 2.4 standard deviations at a 95% confidence level" - ie no matter how sure you are of your sample database, expect small but significant non-conforming factors to play key role when making overall assessment - ie. sod's law always casts a shadow on generalisations, however scientifically measured

[rock]

5. Well, if it's a representative sample

It can represent its target if correctly chosen. But that just begs the question. I agree with above comments: polls are propoganda.

Hi CatWoman. I always read your posts cause you're still the Cat's.

[CatWoman]

11. and you're still my rock

:hi:

:loveya:

[Bozola]

8. It's not so much the numbers....

but the demographic that I find disturbing and unquestioned.


A telephone poll measures the opinions of people who are the natural prey of telemarketers, i.e., people who are gullible enough to blindly pickup the phone and meekly submit to the demands of complete strangers.

The error bars should be based on the percentage of the pollees who answer "yes" to having vinyl siding on their house and/or having their address painted on the curb.

And you wonder why polls seem bias towards freepers.....

[gristy]

13. ---

A telephone poll measures the opinions of people who are the natural prey of telemarketers, i.e., people who are gullible enough to blindly pickup the phone and meekly submit to the demands of complete strangers.

THAT is the key. Telephone polls are biased.

[mrboba1]

9. representative sampling

If you ask a representation of the general population, then the numbers work out.

As an example:
If you ever fill out a poll online somewhere where the topic isn't inflammatory and brings in bias, you will notice that once about 1000 people have responded the percentages don't change much over time - maybe 3-5% - I've noticed that like at ESPN. And that falls in the margin for error.

If you get a decent representation of the population, then the numbers should be fairly close to the actual numbers.


It has to do with the Central Limit Theorem if you really want the statistical theory behind it - but I'm not going that far.

[Gin]

12. I agree..someone somewhere decided that number would

represent the majority opinion..that statistic justifies their job and shortens the number of calls to make.

We are being spoon fed shit...I don't believe a word of it.

[TahitiNut]

15. Welcome to DU, mrboba1.

:hi:

Yup; the Central Limit Theorem it is, with some anomalies at the edges. Another point often misunderstood is the methodology of picking the "representatives" of the population. When you refer to a "topic (that) isn't inflammatory and brings in bias," the bias is in the picking/selection of the representatives which, in this case, is self-selection (or repeated 'selection' from repeat voting). Too many believe 'random sampling' is required. While (often more than) sufficient, it's not required. All that's required is that the method of selection be independent of the results of a survey (a poll, in this case) -- and that the population from which the representatives are selected is statistically congruent with the target population whose attributes (opinions, in this case) are being characterized. It is arguable that Internet users are congruent with any other general population. Thus, inferences from online polls are arguable. (I believe the various print and broadcast media use them, somewhat loosely, merely to measure "hot topics" - the 'hotter' the better?) Likewise it's arguable whether telephone polls select from a population that's congruent with those who self-select to participate in an election.

[newyawker99]

19. Hi mrboba1!!

Welcome to DU!! :toast:

[chadm]

16. If the population is random...

30 is considered the minimum you need for an accurate sample. 900 is VERY safe...so long as the population being sampled is truly random.

[Stuckinthebush]

17. The theory is fine...

787 is an adequate sample of a population if theory is used correctly.

Remember, in statistical theory, one assumption is almost always present (except in the case of some non-parametric statistics) - that the sample is derived from a random selection from the population. In theory, a random selection will give you members distributed as in the population. Since this distribution matches the distribution of the population, then the opinions of the sample will match those of the population in theory.

The +/- X% you see with all polls is the amount that is taken into account based on measurement error. This error is still determined with the original assumption of randomness in place. So, error is increased dramatically when there is no randomness of selection.

This brings us to polling, and polling professionals. All pollsters know the theory, but they also know the limits of their sampling methods. It is virtually impossible to get a true random sample in polling. I could list all of the issues surrounding sampling error, but you could probably already guess most of them. Suffice it to say that pollsters do their damnedest to ensure a random sample, but understand that they can't get it.

Now, all pollsters have favorite sampling methods that they use. These methods will either yield over or under estimation on any number of variables. The best thing to do when looking at a poll is to look at the demographic breakdown of the poll to see if it matches reality, also, take a look at a pollsters accuracy. I have been wanting to do the latter, but have not had the time. If an enterprising DUer would like to take a group of pollsters, look at their predictions of a presidential/congressional/state race in the past, and then look at the actual vote to see the margin of error - it may be insightful. You would need to take the last poll result closest to election day as a comparison, but it may lead to some insights.

My final thought about polls is this: Polls are extremely useful tools to examine public opinion. While there are firms that use polls as a public opinion manipulation tool, the vast majority of pollsters are legit. It is their individual methodologies that should be scrutinized. Remember, it does a pollster no good to make a poll look good for a particular client. That client wants to know what the real opinion is out there. If I client is misled, and loses an election because the pollster was painting a rosy scenario, then that pollster gets fired. Bad news does not make a bad pollster - bad results does.

Take all polls with a grain of salt, and look at the sampling methodology and demographic breakdown of the sample to understand how accurate the poll will be.

[javadu]

18. Turnout Models are the Real Problem Here

Based on probability theory, 787 is more than an adequate sample of a relatively homogenous population. However, you will have varying rates of turnout for different segments of the population. I think (but I am not sure) that all turnout models are based on turnouts in the past. And, this is probably the best that can be hoped for. At the same time, this election is unprecedented and there is no real way to know how the varying segments of the population are going to come to the polls.

Does anyone know how they constructed their turnout models? Are they based on historical turnouts, or did they ask people about the probability that they would go and vote? Moreover, did they oversample small (but important) segments of the population and then control for that in their analyses? If so, did they use the right segments of the population? I don't know the answer to any of these questions, but these questions are the basis for whether or not these polls will accurately predict the outcome of the race. Of course, all of this assumes that every person is given an equal opportunity to have their vote count, which may be a dubious assumption.