Basic Dynamics of Exit Polling

[RaulVB]

Again, might be interesting to review concepts:

1. Exit Polls are the ones made AT THE EXIT OF A POLLING PLACE.

2. Depending on the hour of a day, you'll see some inaccurate results.
The early exit polls are not as good as THE LAST BATCH OF EXIT POLLS.

The odds of inaccuracy for THE LAST BATCH OF EXIT POLLS IN ANY COUNTRY, in any fair election, NEVER GOES OVER THE ACCEPTABLE MOE. For more on that, check TIA's works.

3. In a fair election, THE EXIT POLLS NEVER CONTRADICT THE ACTUAL RESULTS OF THE ELECTION.

4. EXIT POLLS ARE A USEFUL TOOL EVERYWHERE. THEY PREVENT FRAUD FROM HAPPENING. THE ONLY COUNTRY WHERE THEY "FAIL" ON REGULAR BASIS IS THE US.

5. In any country, after the last batch of polls is received, THE CANDIDATE GOES IN PUBLIC RECORD PROCLAIMING HIMSELF AS A PROBABLE WINNER.

They are that good.

6. In any country, the networks posted results ON THE TV SCREEN NEVER CONTRADICT THE EXIT POLL NUMBERS.

The opposite happened from the "get go" here in the US. No explanation.

So there you have it.

If you think "*" "won", you leave in a parallel universe, I'm afraid.

[RaulVB]

1. "Casual" kick (n/t)

[Salomonity]

2. nonsense.

Point 1 is correct.

Point 2 is partially correct--the later in the day, the more people have voted, the better the poll.

"The odds of inaccuracy for THE LAST BATCH OF EXIT POLLS IN ANY COUNTRY, in any fair election, NEVER GOES OVER THE ACCEPTABLE MOE." is not merely false, but is nonsense. The margin of error specifies what's the largest deviation of the poll results from the real population, with greater than a given possibility. Odds and margins aren't comparable: it's like saying that the odds are never greater than the number of apples--you're comparing objects of different types.

Also, there are certain to be results outside the margin of error. In a normal poll, the margin of error merely means that 95% of the time the results will be within the margin of error of the correct value--but 5% of the time, the poll is just plain wrong, outside the margin. I don't have a good sense what the margins of error and confidence levels are for Mitofsky's poll, and I'm not sure those numbers are yet public.

When the margin of victory is smaller than the poll error, there certainly will be cases where the poll is wrong. Point 3 therefore is wrong.

One thing to keep in mind: US elections are very close by world standards--if US presidential elections typically had 60% majorities, it would be reasonable to say "exit polls are never wrong". There are two important differences between the US and other countries here:
We're unusual in having not merely one national election, but 50 state races. This means that we should expect at least a few states to be outside the exit poll margin of error each election.

Further, our elections are often very close, and therefore more likely to fall within the poll error margins. It's easy to call the Palestinian election based on exit polls when the poll result is 66-20. It's much harder here, where it's 51-48.

[TruthIsAll]

3. You don't know the MOE? What DO you know?

You say:
Also, there are certain to be results outside the margin of error. In a normal poll, the margin of error merely means that 95% of the time the results will be within the margin of error of the correct value--but 5% of the time, the poll is just plain wrong, outside the margin. I don't have a good sense what the margins of error and confidence levels are for Mitofsky's poll, and I'm not sure those numbers are yet public.

Here is a free education for you:

You obviously don't know the MOE for the National Exit Poll.
It is 1.0% as per Mitofsky.

Kerry wins the poll 50.78-48.22%, basewd on his proporuon of the female/male split (54/46). Yes, women, mostly young, new voters, came out to vote for Kerry in droves.

A 1.0% MOE means that 95% of the time, Kerry's actual vote would fall in the range {49.78-51.78}.

It also means that Bush's vote would be in the range {47.22-49.22%} 95% of the time.

Further, it means that the odds of Bush getting 51.23% of the vote (a 3.0% gain over his exit poll) is one out of 547 million.

Do you know what the normal distribution is?
Do you know what the standard deviation is?

Do you know what the relationship is between the standard deviation and the margin of error?

Do you know what this represents?
= NORMDIST (.4822, .5123, .0051, TRUE)

I didn't think so.
Do your homework.
Take Statistics 101.
Learn Excel.

Then come back for your next lesson.

[Salomonity]

23. not under these circumstances

Those numbers do NOT say that the chance of a 3% Bush gain over the exit polls is 1 in 547 million. They say that that's the chance of the poll being off by 3%, IF IT WAS A PERFECLTY RANDOM SAMPLE. It wasn't, so your numbers are simply nonsense.

I have a fairly good grip on statistics, thank you. If any of my professors were handed your "analysis", they would giggle--and fail the student who submitted it. It is a cardinal error in statistics to use formulas beyond their domain of applicability--for instance, to assume a sample is random when it clearly isn't. This has been pointed out to you many times.

[TruthIsAll]

34. Mitofksy said it was a random sample. What the hell more do you want?

Can you read?

Once again:
National Exit Poll
13,047 randomly selected
1.0% MOE

The odds are 1 in 547 million.
You don't like those numbers?

I have a True or False test for you.

Answer YES or NO to the following.

1) Does Mitofsky call it a random sample?
YES or NO

2) Does he say the margin of error = 1.0%?
YES or NO

3)IS THE STANDARD DEVIATION = MOE/1.96= .01/1.96 = .00051?
YES or NO

4) Does Kerry, based on the female/ male split (54%/46%), win
with 50.78%-48.22%?
YES or NO

5) Is .5078 = .54*.54 + .47*.46 = 50.78% (for Kerry)?
YES or NO

6) Is .4922 = .46*.54 + .52*.46 = 48.22% (for Bush)?
YES or NO

7) Did Bush "win" with 51.23% of the vote?
YES or NO

8) Is the probability of a 3.01% deviation in favor of Bush
given by this formula: PROB = NORMDIST(.4822,.5123,.00051, TRUE)
YES or NO

9) Are the odds = 1 out of 1/Prob = 1 out of 547 million?
YES or NO

10) Do you get it yet?
YES or NO



You just won't accept that will you?

[Salomonity]

36. it's certainly false

The sampled individuals were clustered by precinct--as everyone admits--and therefore, in a statistical sense, WERE NOT RANDOM.

I don't care if it says 'randomly sampled' at the top of the page--saying it doesn't make it so. Nobody disputes that only some precincts were polled. That fact, ALONE, means the sample is NOT random, despite any masthead to the contrary.

We're not gonna overturn an election on the basis of two words on the top of a page of unoffical results, that are undeniably and undisputedly false.

[RaulVB]

4. Not asking for your certification

Your response is ridiculous.

Regards

[Lurker321]

5. And to illustrate the wonderful "accuracy" of exit polls

here is some info on Ukrainian exit polls:

http://www.myrtlebeachonline.com/mld/myrtlebeachonline/10499250.htm

There were three exit polls. Their results were:

1.  58.1%  to   38.4%   MOE=2
2.  56.5%  to   41.3%   no MOE given
3.  56%    to   41%     MOE=2

Official results: 
    51.99% to   44.2%    

http://tinyurl.com/3ufxp


Every one of the exit polls was wrong and every one of them was way outside the margin of error. Compared to these polls, US national exit polls of 2004, though incorrect, were a lot closer to the final result.

[Der Blaue Engel]

6. What on earth are you saying?

Now you're trying to say the exit polls were wrong in Ukraine?

You've lost me.

[RaulVB]

8. That poster is doing literal interpretation

He is trying to "demonstrate" a preset (on his mind) thesis.

[Lurker321]

9. Yes, exit polls were wrong in Ukraine

it is clear. Why are you surprised? Exit polls are often wrong.

[RaulVB]

11. "Are often wrong..."

Actually the opposite is true.

You have a mindset that wants to "debunk" the validity of exit polling...good luck!

[Der Blaue Engel]

13. I thought it was the exit polls that gave the lie to the original count

So...Yanukovych actually did win then? What am I missing?

[RaulVB]

14. Nobody knows who won on the first process

That's the only known fact.

The exit poll construction was used to push "*"'s friendly candidate, Yuschenko.

It was a power game between Putin and "*".

[Lurker321]

16. It was not the exit polls that proved that the first

Ukrainian elections were fraudulent. It was evidence, video-taped, witnessed, and documented, of massive fraud.

See http://www.telegraph.co.uk/news/main.jhtml?xml=/news/2004/11/28/wukra28.xml

The exit polls that I showed above were from the second "do-over" elections. The numbers illustrate that exit polls are not and cannot be the measure of the "fairness" of the election by themselves.

See http://www.aceproject.org/main/english/lf/lfd08e.htm

"The majority of exit polls carried out in European countries over the past years have been failures."

[RaulVB]

17. False, but frankly amusing.

Your link takes people to a "study" last updated in 1999. Please, show some honesty.

Exit polls in Germany, France, Spain, Italy, Ireland, Scotland, etc., etc. are accurate because they are honest.

Those are European countries, still, right?

[Lurker321]

18. You claim a lot of things -

but you are yet to give links to the exit polls in Germany, France, Spain, Italy, Ireland and Scotland that were, in fact, accurate.

How come?

[TruthIsAll]

35. Why is it that in every election since 1988, the Democrat did

worse than the exit polls predicted?

I give you one guess.

[Lurker321]

38. Oh please let me take two guesses

1. There has been a massive fraud conspiracy that changed (up to 8% in 1988) the final election results from the correct exit polls in every election since 1988.

2. The exit polls have a systemic bias in them toward Democrats in federal elections.

Which one would you pick?

[Salomonity]

22. I'd trust a 99 study more than a 2004 study

A study from 1999 cannot be accused of having been written to support Bush against exit polls in 2004.

[pauldp]

15. So I guess we still need the raw data don't we? n/t

[Lurker321]

19. Here is a good article on this

http://www.salon.com/tech/feature/2005/01/07/exit_polls/

[pauldp]

20. You didn't answer my question. n/t

[Lurker321]

24. read the article - it will talk

about whether getting raw data will do anything.

Exit polling involves lots of massaging data, adjusting it and fitting it in. It is more of an art than a science. Treating it as if it was all cut and dried is doing exit polls a disservice.

[pauldp]

25. I read the article....

My question to you is do you think we should be putting pressure on the Media Outlets to release the raw data. You seem to be avoiding my question.

[Lurker321]

26. sure - we raw data should be released

although I don't think pressure will do that at all. The data will be available to anyone who pays for it within a month or so, I understand. One thing that will be interesting to do would be to compare the per-precinct exit poll data (if it can be broken down that deep) with the election results in the particular precincts. That would at least partially eliminate the "clustering effect".

[pauldp]

27. So why do you think they have not released it inspite of the controversy?

[Salomonity]

30. there's a process

Mitofsky et al don't want to release the data until they've finished checking for errors, have organized it in a sensible way, and are able to distribute the whole mass of it. Remember, there are results from many precincts, and to do the analysis properly, all this data must be checked and organized. Be patient.

[pauldp]

32. Mitofsky said it was up to the media to release it.

People are asking for the data not an analysis of the data. It's been over two months. How come the media can see it but the people can't?

[RaulVB]

33. Exactly!

Nobody wants Mitofsky's "take on the data."

He has said the networks have it and the networks HAVE REFUSED TO RELEASE IT.

[RaulVB]

7. Those exit polls had methodology errors

We agree on that.

A HONEST EXIT POLL IS ACCURATE. A HONEST ELECTION ALWAYS GIVES THE ACTUAL WINNER.

Not on the US.

I agree.

[Lurker321]

10. "Methodology errors"?

Maybe you can expand on that. Do you have any information on their methodology? Can you compare and contrast their methodology with US exit polls?

[RaulVB]

12. Do your own research. You're the "expert" (n/t)

[thanatonautos]

28. If the MOE quoted is one sigma: ie +-2% then you expect

that the actual results will lie inside
the range +-2% from the result of the poll
68.26% of the time, but outside of the
range 30.74% of the time, assuming normally
distributed sampling error and, of course,
no systematic error.

For two exit polls with a MOE of 2% to both
be outside the margin of error is an
event with overall probability about 9.5%.

That is, it's expected to happen about 1 time
out of 10. So that this happened in the second
Ukrainian elections, in and of itself, is not that
improbable an event.

But, if you look more closely, and take into
account the actual deviations of the two polls from
the official result, you'll see that exit poll 1 is wrong at
about the 3-sigma level, and exit poll 3 is wrong at
about the 2-sigma level. Both results deviate on the
high side.

2-sigma corresponds to a 95.46% probability, and 3-sigma
to a 99.73% probability, so that the actual probability
of this result occuring by random error alone is
estimated at 3.0645x10^-5, or about 1 in 33,000.

All this is assuming no systematic errors existed
in the exit polls.

If I trusted the pollsters here to have done a
professional job, then my conclusion would be that
the new election in Ukraine as well as the exit polls
certainly deserve to be investigated, to see what
the source of the discrepancy was.

The state-funded Ukrainian Institute of Social Research and Social Monitoring Center showed Yushchenko winning with 58.1 percent of the vote and Prime Minister Viktor Yanukovych garnering 38.4 percent. The margin of error was 2 percentage points.

The Western-funded Razumkov Center of Political Studies and Kiev International Institute of Sociology showed Yushchenko winning with 56.5 percent and Yanukovych collecting 41.3 percent of the vote, with no margin of error given.

A third exit poll, by Frank Luntz, a pollster for the U.S. Republican Party, and Douglas Schoen, of the Washington-based market research company Penn, Schoen & Berland, showed Yushchenko winning with 56 to Yanukovych's 41 percent, Schoen said. The margin of error was 2 percentage points.

I do not, personally trust Frank Luntz to do a
reliable exit poll in Ukraine unless I had a lot
more information about his methodology, so I would
reserve judgement about the accuracy of his exit
poll 3. I've ignored exit poll 2, and I have no
feeling about exit poll 1.

However, Edison and Mitofsky have a very long
experience in the business of exit polling, especially
in the US, and know what the likely problems
are: they've had a long time to work out the problems.
This tends to make me trust that they've made every
effort to eliminate systematic errors. So I do
take the apparent deviation of the published exit
polls from the election results here in the US
pretty seriously.

I would certainly not dismiss either the
results of exit polls you quote for Ukraine,
or the results for the US out of hand.

Both are highly improbable, and in fact,
do tend cast doubt on one of:

(1) The official results of the elections.
(2) The results of the exit polls.

In either case, this a serious issue, worthy
of examination.

Every one of the exit polls was wrong and every one of them was way outside the margin of error. Compared to these polls, US national exit polls of 2004, though incorrect, were a lot closer to the final result.

This last statement does not appear to be correct, if
the results of the election day polls that have been
leaked from Edison/Mitofsky are what they seem to be.

The margin of error, you see, is smaller in the
Edison/Mitofsky polls, therefore the observed
deviation is much less probable.

[Salomonity]

29. systematic error

Mitofsky himself, who as you say is one of the real exit-poll experts has said both that there was systematic error, and that deviations don't seem to correlate with voting technology.

We'll know more when the data is released.

[thanatonautos]

31. Agree, I mentioned systematic error as a possibility.

And I hope that Mitofsky will indeed release
the data as soon as is possible. I think that
the delay in doing so creates a lot of suspicion.

It really would be in the public interest
to put it all out in the open.

[Mistwell]

21. Correction

1. Exit Polls are the ones made AT THE EXIT OF A POLLING PLACE.

FALSE: Exit polls include some phone interviews with absentee voters.

[Laura PourMeADrink]

37. Raul, Do you know off-hand, the top 5 states where the divergence

from exit polls was the greatest. We are trying to come up with concensus on which states to study first on my post about patterns compared to y2k

[RaulVB]

39. From the top of my head,

Florida
OHIO
Wisconsin
Minnesota (totally off)
New Hampshire (totally off)

[TruthIsAll]

40. Look here, Laura

http://www.democraticunderground.com/discuss/duboard.php?az=show_topic&forum=203&topic_id=112108

[Laura PourMeADrink]

41. Thanks! Step back and look at these!!! Got to mean something

Top 5 B vote gain, relative to Kerry, compared to Gore in Y2K
1. FL
2. NY
3. TX
4. TN
5. NJ

Top 5 Exit Poll Divergence
1. DE
2. NH
3. VT
4. SC
5. NE

Top 5 States - Most Red Shift
1. NH
2. OH
3. PA
4. MN
5. FL