Ending "The Debate on the Exit Poll Debate"--Do it yourself. FREE SOFTWARE

[autorank]

This thread on the "Exit Poll Debate" being a "waste" is enormous.

Who want's to do some work?

This spread sheet like the Guttenberg Bible -- you get all the figures in a spread sheet, alter assumptions, and YOU DECIDE. It's free and you can do it yourself.

Just go here for the down load.

http://us.share.geocities.com/electionmodel/InteractiveElectionSimulation.xls

PRE-ELECTION STATE POLLING AND EXIT POLL ANALYSIS

National Exit Poll Demographic Analysis

Choose your spread sheet calcualtions for Exit Polls or Pre-election Polls

***Adjust the MOE -- yourself

***Adjust the Undecideds -- yourself

***Adjust any and all demographics -- yourself
It's your chance to take the real data and prove, disprove, modify etc.

This should summarize things nicely and you can decide based on your analysis.


http://us.share.geocities.com/electionmodel/InteractiveElectionSimulation.xls

[applegrove]

1. That is really scary. Will all people be given and master's in regression

analysis and stats too?

Why do we have to have our information dumbed down?

[autorank]

2. Give it a try and see how you like it. Easy on the vino, though, it

may alter your interpretive skills;) Seriously, you know this stuff. Make or break the case.

[Name removed]

3. Deleted message

Message removed by moderator. Click here to review the message board rules.

[kster]

4. So many great threads when do you sleep? .. K&R..nt

[autorank]

5. Sleep is vastly over rated!

[OnTheOtherHand]

6. who has already done some work?

I say it's misleading for TIA to present Kerry as ahead in the pre-election polls, when in several cases the people who actually conducted the polls drew the opposite conclusion. But that's old news.

I say that interminable recalculations of margins of error don't prove anything -- we've had those for almost a year now. More old news. As Febble has pointed out, there are plenty of U.S. presidential exit polls whose discrepancies were outside the margin of error. (Actually, I thought TIA pointed that out at least once himself.) What does this prove?

If folks think they can use this spreadsheet to bypass the debate, well, I guess they can, but then they shouldn't expect to contribute much to it.

[Febble]

7. What TIA's spread sheet shows, conclusively

is that the poll was outside the margin of error.

Actually, I can do better even than TIA. We now know, from the E-M report, that the error wasn't in precinct selection but at precinct level - "Within Precinct Error".

In only six states was the mean WPE positive (blue shifted)

Kentucky
Montana
Oklahoma
South Da
North Da
West Vir.

In all the rest, the WPE was red-shifted (negative).

So the average of the these mean state WPEs was negative - in fact it was -6%. To test whether it was significantly negative we can do a 1 sample t test on the mean of the means, which will tell us whether the mean of the state mean WPEs was significantly different from zero. And it was. I calculate that the probability that it was different from zero solely by chance is:

1 in 13,144,526,648

TIA only gets a

1 in 378,161,882 probability of the probability of the National Poll discrepancy being due to chance using his default assumptions.

This of course does not mean that TIA's math is wrong - or even his assumptions, though they may be - it's just that E-M have given us the actual precinct level error rates, after removing any error due to non-representative selection of precincts.

So: let us all agree, and agree that we agree:

The exit poll discrepancy was not due to chance.

The relevant question is not: how improbable was it that it was due to chance?

But: what non-chance factors caused the discrepancy?

The "debate" as to whether it was due to chance or not is the flimsiest of straw men. There IS no debate. It wasn't.

[bleever]

9. "So: let us all agree, and agree that we agree:

The exit poll discrepancy was not due to chance....what non-chance factors caused the discrepancy?"

Very succinctly put. Thank you, febble.

[Time for change]

12. You took the words right out of my mouth bleever

I've said it before myself, but I've never seen it as well put as Febble just did here.

One of the main questions I have is: Is anyone arguing whether or not chance could have accounted for the exit poll discrepancy?

[OnTheOtherHand]

23. no, as Febble says, at most the sheet "refutes" a straw man

As you will recall, E/M in January (echoing earlier statements) wrote inter alia: "Our investigation of the differences between the exit poll estimates and the actual vote count point to one primary reason: in a number of precincts a higher than average Within Precinct Error most likely due to Kerry voters participating in the exit polls at a higher rate than Bush voters."

The spreadsheet does not address that issue. As far as I can tell, it has no relevance to the actually-existing exit poll debate whatsoever (unless people find that it pulls numbers together in a handy way). If anything, it is likely further to confuse the debate.

To give some idea of how the spreadsheet promotes confusion, consider this explanation of the pre-election analysis (on the "instructions" tab of the spreadsheet) :

"The MoE defines the range in which we can expect the final results to occur (with 95% probability). For example, if a poll with a 3% MoE yields 51% for a candidate, then the probability that his/her actual result will fall between 48%-54% is 19 out of 20 (95%)."

That would be true if sampling error were the only source of error. Good luck with that. The electoral-vote.com database has seven different results from Ohio completed on October 31 or November 1 (I'm only counting the Zogby tracking poll once). Bush was ahead in six of these polls, by up to 6 points; Kerry led the seventh poll by 7 points. It would be brave to assume that these differences were due to sampling error alone.

So, if six out of seven polls showed Bush ahead in Ohio, why does the spreadsheet have Kerry ahead by exactly three points there? Heck if I know, and heck if the spreadsheet cites any sources. However, I suspect that its designer has chosen to ignore the likely voter results -- literally ignore them, as if they never existed -- and rely on registered voter results. That controversial decision certainly doesn't warrant the 100% confidence it is given in this exercise.

Similarly, the explanation of the Monte Carlo simulation (which applies to both the pre-election and the exit poll analysis) says, "This model consists of 200 trial 'elections' using state win probabilities, to determine the expected Electoral Vote and win probability." Well, unless I'm much confused, these "win probabilities" assume that there is no bias whatsoever in the exit polls -- even though every exit poll we've looked at has overstated the Democratic candidate's performance.

The spreadsheet seems to be designed to give one interpretation of the pre-election polls, and one interpretation of the exit polls, the aura of simple fact. Kerry led in the pre-election polls; the exit polls were unbiased. Just enter numbers, press buttons, and see the statistical proof!

[bleever]

24. I'll agree with Febble again: "what non-chance factors caused the

discrepancy?"

Too many Kerry responders? Too few * responders?

These are strawmen too, as has been well-established.

Disproportionate disenfranchisement of Kerry voters, via felon lists, machine shortages, and rigged machines?

These are the non-chance factors that are well-documented.

Have I missed any?

[kiwi_expat]

25. Of course, felon lists and machine shortages did not affect exit polls.

The voters disenfranchised by felon lists and machine shortages never voted and never participated in the exit polls.

[bleever]

27. Thank you, you are correct.

These factors only exacerbate the injustice pointed at by the exit poll discrepancy.

So we're left with rigged/non-random-erring machines, and manipulated/faulty-but-nonrandom tabulation.

Anything else?

[kiwi_expat]

28. Don't forget the uncounted under/over votes & rejected prov. ballots n/t

[bleever]

29. Thank you. Two more cans of worms, which if opened will not

unskew the skewage.


:)

[Febble]

33. Well, they may unskew it a bit -

a poll conducted in a precinct that had more uncounted votes for Kerry than for Bush would tend to show a red-shift.

But here's the thing: there are two main sources of non-random error that could account for the overall exit poll discrepancy:

1. bias in the count, which you rightly suggest, but also:

2. bias in the poll

Bias in the count could arise from many forms of vote corruption:

* Vote switching
* Ballot stuffing
* Greater spoilage of Kerry votes (undervotes; overvotes)
* Greater numbers of provisional ballots issued to Kerry voters and subsequently rejected

Bias in the poll could arise from two sources:

* non-response bias (refusers more likely to vote for Bush than Kerry)
* sampling bias (Kerry voters more likely to be selected)

Neither of the sources of bias in the poll have anything to do with "sampling error", which is what the "margin of error" (MoE) tells you about, as they are non-chance (aka non-random) sources of error. Both TIA's spreadsheet and my t-test tell you only about random error, aka "sampling error".

The kinds of analyses that will help distinguish between bias in the count and bias in the poll are analyses that look at patterns in the data. For example: correlational analysis - analyses that tell you what kinds of precincts were associated with the most bias. If it was the precincts with DREs, then the answer would look like bias in the count. If it was precincts with the least well-trained interviewers, then the answer might look more like bias in the poll . It is of course perfectly possible that there was both.

So, if we want to know what caused the exit poll discrepancy (and we know it was not chance) the two questions that need to be asked is:

1. What precinct characteristics are associated with the greatest bias?

2. Are these characteristics suggestive of bias in the count or bias in the poll?

[Time for change]

38. One "characteristic" that I'm very interested in is

correspondence between the pre-tabulator counts for each precinct in a given county and the final post-tabulator county count. But nobody seems to have that information.

Lacking that, I would at least be interested in comparing red shift in precincts where these pre-tabulator countes were available and documented versus those where that was not the case. But I don't imagine that that information is available either.

[Febble]

34. I'm a bit concerned

that you may think I've been misleading you, which was certainly not my intention.

I'm glad you agree with my statement that the exit poll discrepancy was not due to chance. I think we all need to be really clear about this. The discrepancy was definitely caused by non-random factors.

But I don't agree that the "Too many Kerry responders? Too few * responders?" factors have been "well-established" as strawmen, I'm afraid. Certainly the MoE calculations don't do this.

Three legitimate arguments that have been advanced to refute "Too many Kerry responders? Too few * responders?" as explanatory non-chance factors are:

1. The USCV analysis (replicated by TIA) indicating that bias was greater in high Bush precincts.

2. TIA's argument that when the exit polls were re-weighted to the vote-count, the apparent proportions of Gore v Bush voters are actually impossible.

3. The fact that overall completion rates were not lower in precincts in which most of the counted votes were for Bush than in precincts were most of the counted votes were for Kerry (demonstrated by USCV, E-M, TIA, Jim Knapp, and many others).

However, without re-opening any of those cans of worms, I'd simply say that these three refutations are not unassailable by any means. I don't think (1) is supported by the precinct-level data, and is probably an artefact of the way the analyses were done, and I think (2) is undermined by that fact that it rests on "recalled vote" data which has itself been shown in other studies to be subject to systematic bias. (3) is an excellent point; however it does not rule out bias in the selection process, as opposed to bias in the response rates, as a sources of bias in the poll.

So I can't agree that the case against bias in the poll is "well-established". But I completely agree that there was bias somewhere, which is what my little t-test exercise above tells you. It could have been bias in the count. I have myself dug out evidence that may have been at least partially due unequal patterns of undervotes in NM.

So the exit poll is simply not a slam dunk for fraud. It certainly does not rule it out. However, neither does it rule it in.

But whether fraud is ruled out or in or neither by the exit poll evidence makes absolutely no difference to the unassailable case for electoral reform, laid out so eloquently by autorank here:

http://www.democraticunderground.com/discuss/duboard.php?az=show_mesg&forum=203&topic_id=394982&mesg_id=394982

[bleever]

41. "...it does not rule out bias in the selection process, as opposed to bias

in the response rates, as a sources of bias in the poll."

This also is true, and an important point from a purely forensic point of view. (Purely forensic not meant in any pejorative way; simply meaning as a matter of strict science.)

But stepping back and looking at things from the context of known and generally accepted methods, at this point the burden of proof (for those who would say that "this proves nothing") shifts the other way, and poses the question: How did systemic and non-random bias in the selection process enter in this time, when the evidence suggests that this bias has never been present before?

We're looking at things from two perspectives, and trying to stereoscopically synthesize them. One perspective is scientific and reductionist, and the other is based on common knowledge and consensual reality. Sometimes I think these conversations have been tremendously complicated by the confusion when those two paradigms are acknowledged, because they are contradictory in one dimension, and synergistic in another.

[Febble]

43. I won't argue about where the burden of proof lies

because that is completely context dependent. I accept that given the hideous insecurity of your electoral system, it cannot be assumed that the vote-count was correct.

You ask the right question, in my view (because it is my question also!):

How did systemic and non-random bias in the selection process enter in this time

But you add:

when the evidence suggests that this bias has never been present before?

Now, I take your last point about perspectives, and I agree there is confusion between paradigms. But there is also confusion about data.

I would submit that the evidence does not suggest that the bias has never been there before. Indeed from the data given in the E-M report it is relatively easy to demonstrate, using exactly the same t tests as I used above, that the exit polls have had an apparent Democratic bias in every year for which they give the data (1988 onwards), and a statistically significant bias for all years except 2000. And in fact, if we transform the WPE means using what may or may not be a valid application of my formula for getting less warped yardstick out of the WPE, t-tests show that in every year, the bias was significantly Democratic. (If you are interested, I diaried both these statistical exercises on Daily Kos, here and here).

Evidence from the UK also suggests an inbuilt bias in polls against the conservative voter (one reference here).

So while I think I understand your point about conflicting paradigms (and I am aware that statistical analysis is inevitable bound by the concept of the "null hypothesis" and cannot, in fact, "prove" a null), I do think it is worth baselining both paradigms to the same datum!

So to rephrase your question, I would ask not why bias has entered for the first time, but why the so much greater this year than in previous years - and indeed why bias should fluctuate (as opposed to show a steady increase or decline) from year to year. There may be good answers to these questions, but I do not know them. However, it is certainly not true to argue that there is no precedent for bias, even in electoral systems where we can be sure of our count (ours, UK, for example).

[Time for change]

36. I can think of a few, particularly in Ohio

Voter intimidation, failure to issue absentee ballots, failure to issue provisional ballots in accordance with the law, probable illegal voter registration purges, giving voters incorrect information in order to prevent them from voting, and obstruction of a full and fair recount.

[Febble]

37. Hang on!

Voter intimidation,
failure to issue absentee ballots
giving voters incorrect information in order to prevent them from voting

Couldn't be non-random causes of the red shift, though they would lose Kerry votes.

probable illegal voter registration purges
failure to issue provisional ballots in accordance with the law

Might cause a red shift, if they meant more subsequently uncounted provisional ballots were issued to Kerry voters, who would nonetheless respond to the pollsters.

obstruction of a full and fair recount

Wouldn't cause a red shift, though it might indicate a the presense of a fraudulent factor.

All these things probably happened, and cost Kerry votes. But if we are evaluating non-random factors that may have contributed to the exit poll discrepancy, then we need to distinguish between factors that would have produced a red-shift, and factors that would not have done.

But I think you know this! I'm just trying to clarify the point.

[Time for change]

39. Well yes, of course

I was just thinking of mechanisms for fraud, not necessarily those types that would cause a red shift. I guess I got carried away.

[Febble]

40. Well, I'm just trying to keep the case coherent!

And I agree (vis a vis your other comment) that the precinct v tabulator data would be extremely interesting.

[bleever]

42. Yes! And in addition to failure to issue provisional ballots in

accordance with the law," I would add differential standards of counting provisional ballots.

[autorank]

16. What are the chances Bush wouldn't steal it!

Mon Dieu! Has the exit poll debate ended! On my watch, how ironic!

Seriously, we could have an interesting debate on causality but it's clear even to me that there is no plausible scenario one can construct where Bush wins. I manipulated the "undecideds" and found that you have to get into the 30's for Kerry in order for Bush to win. I won't dredge up all the reasons that's simply not possible.

It's not the probabilities so much as the fact that there is NO plausible or realistic VARIABLE I can see that puts Bush over the tope, not one. The National Exit Poll Demographics scenario makes that perfectly clear.

Let's get a jury of 12 randomly selected British bank tellers (who also count votes, as you educated me) and let them decide...just randomly selected, or that group in the picture.

Based on this evidence, did Bush steal the election. Yes NO
Since he stole the election, should he be removed. Yes (No is not an option)

Thank you British bank tellers, for saving democracy. (In Great Britain they recruit bank tellers to count paper ballots.)
;)

[Febble]

18. Well, I hope the debate about probabilities has ended

We seem to have agreed at least on the shape of the table.

And whatever I think about the non-random variables that produced the exit poll discrepancy, I do like your list of goals.

(But bear in mind that I'm a Brit - dunno what I think about ID cards, we just have voter rolls, and I don't know whether hand counted paper ballots are a realistic goal in the US. I do know they work though, and they actually get our results through quicker.)

[kiwi_expat]

26. 11.5% Gore voters and 94% Bush voters voting for the "War President".

"...there is NO plausible or realistic VARIABLE I can see that puts Bush over the top, not one. The National Exit Poll Demographics scenario makes that perfectly clear." -auto

What is implausible about 11.5% Gore voters and 94% Bush voters voting for the "War President"?

Kerry lost with this scenario. Please see:

http://www.democraticunderground.com/discuss/duboard.php?az=show_topic&forum=203&topic_id=352989#353010


Cheers!

[autorank]

30. Plausible does not mean popular tie and electoral loss; popular war pres?

      VOTED IN 2000 
             Mix  Votes Kerry  Bush   Nader Total
      No   18.31% 22.39 57.00% 41.00% 2.00% 100%
      Gore 39.75% 48.60 88.50% 11.50% 0.00% 100%
      Bush 39.32% 48.08  5.00% 94.00% 1.00% 100%
      Other 2.62%  3.20 71.00% 21.00% 8.00% 100%

      Total 100.00% 49.44% 49.59% 0.97%
      Vote  122.27  60.45  60.63  1.18

This is a stretch on your part, but a nice try.  It's still
implausible to come up with a Bush win.

POPULAR WAR TIME PRESIDENT (ASSUMPTION):  Not so.  Less than
50% support, early October and just days before the election
does not indicate a "popular" war time president.

This Marist College Institute for Public Opinion poll reports:

           11/1/05  10/7/04
Approve        49%    49

Disapprove     49     48

Unsure          2      3

[kiwi_expat]

31. That might well have been the case:

Bush could have "really" had a slim popular vote win, and Kerry could have "really" won the Electoral College.


I guess I took you too literally when you said,
"It's not the probabilities so much as the fact that there is NO plausible or realistic VARIABLE I can see that puts Bush over the top, not one. The National Exit Poll Demographics scenario makes that perfectly clear."



My variables "put Bush over the top". And they take into account the National Exit Poll Demographics.

[autorank]

32. Show me your variables (LOL)

Kerry almost won the electoral college with a loss (sic) by several million votes and your difference is in the thousands. With that small a margin, Kerry win's Ohio and wins the electoral college given the razor slim margin. Just to get to that, you have to presume that 11.5% of Gore voters 1/9th would have voted for Bush. If Bush had been popular on the war and Kerry McGovern-like, I might accept that. But as the Marist poll showed, Bush didn't even top 50% approval for the war and the 'disapprove' was equal to that, hardly a "popular" war time leader. Plausible IS the name of the game and 1 in 9 Gore voters, GORE (voted for him), going for Bush with (a) the memory of a stolen election and (b) the enthusiasm for Kerry's campaign (it was quite something) is NOT plausible.

The effort is commendable but you are not presenting a plausible outcome.

Why don't you just manipulate undecideds on the Exit Poll module?
See how far down you have to go for Bush to win?
By my reconing the number becomes pretty rediculous but I'd like to hear what you come up with.

We're expecting you back in the country by 2006 to vote in person and be a poll watcher.

[OnTheOtherHand]

35. welcome!

Look, it's fine to believe that Kerry won the election every which way. To say that anything else is "not plausible"... well, you can say whatever you want. If it means "I just can't believe," cool. If it means "no reasonable person can believe," then we have a problem.

(1) While I don't think anyone here has described Bush as a "popular" war president, historically speaking his late approval ratings were probably good enough to win. In fact, the official returns give Bush a lower percentage than one might expect by just drawing a best-fit line through final Gallup approval ratings and vote totals.



One problem with that scatterplot is that the "final" approval ratings come at different times -- but I've fiddled with alternative specifications, sets of years, etc. and haven't yet found one that makes Bush a loser based on his approval ratings alone.

(2) You are probably right about the electoral college, although it's close. Bush won the official returns nationally by about 2.5 points and in Ohio by about 2.1 points -- so depending on assumptions about new voters (and how closely Ohio parallels the national results), a narrow Bush popular-vote victory could accompany a narrow electoral-vote victory.

(3) In the 2000-2004 National Election Study panel, 314 people in 2000 reported having voted for Gore. Among them, 302 reported voting again in 2004, and 39 of those reported voting for Bush. That's 12.9%. (The nominal margin of error for this proportion is about 3.8%, so -- just based on this sample -- the true figure might be closer to 9%, or 17%.)

People who participate in NES panels tend to be somewhat more politically aware than people who don't, both because of inevitable selection bias and because of learning effects (participating in long interviews about American politics seems to provoke some people to pay more attention). So, I think the NES may tend to underestimate the proportion of Gore 2000 voters who voted for Bush in 2004. But I don't know.

If someone can give me an intellectually defensible reason to ignore that 12.9%, and agree with you that even 11.5% is just not plausible, I would be interested to hear it.

You apparently are assuming that almost everyone who voted for Gore in 2000 is convinced that the election was stolen. Do you have evidence?

(4) It's only meaningful to manipulate undecideds on the Preelection projections. But then, if you rely on the spreadsheet, you are buying into the assumption that the polls showed Kerry ahead, even though the people who conducted the polls thought they mostly showed Bush ahead.

http://pollingreport.com/2004.htm
http://people-press.org/commentary/display.php3?AnalysisID=102

The spreadsheet is only as good as the assumptions built into it.

[Bill Bored]

22. Even Mitofshy agrees it wasn't due to chance. nt

[Helga Scow Stern]

8. Good challenge, autorank. However...

I don't need statistics to show me what happened. I saw it happen on national TV.

How stupid are we pretending to be????

[autorank]

14. Ojai...for those who have eyes to see, see!

You got it, it was stolen in every way. This is just a nice version to emposer the 'math' community and let them come up with methods of monitoring our damn elections. The Carter Center would do well to just stay home in 2006 and get volunteers in every precinct to work our elections. They would, however, need a "free" National Vote Monitor ID with their picture, a DNA sample, and their entire medical history embedded in a micro chip!

[Mr_Jefferson_24]

10. The fact that we...

...had an election/voting system shoved down our throats that does not provide for complete transparency (during vote collection and tabulation) and conclusive accuracy verification through some kind of redundancy check, tells me everything I need to know. Who but a thief would even contemplate such a system, much less propose and implement it? Who but a thief would fight so hard to prevent effective means of transparency and verification from being put in place as the GOP has done? If we were to sit down and play poker with my new poker system where only I (the dealer) get to look at the hands dealt, determine the bets and cards taken, and then report the results without anyone but me ever allowed to look at any cards, might you become suspicious after I win all your money? Would you need to do statistical analysis to decide whether or not you'd been defrauded?

[byronius]

11. Now that, sir, is a coherent analogy.

[Febble]

13. See here (link)

http://www.democraticunderground.com/discuss/duboard.php?az=show_mesg&forum=203&topic_id=395044&mesg_id=395173

(edit of subject for clarity!)

[autorank]

17. haha...I thought you were saying "here, here" Nice link...

...oops,"hear hear" as my wife's distand relative would say, maybe before "The Sitting Speech"

[autorank]

15. No "need" but it sure helps. And great line!!!

"Who but a thief would even contemplate such a system," damn, that is so good.

It's all politics and it's global and local. The Republicans were glad to trumpet the exit polls in the Ukraine, Lugar in particular, and force a new election (or have their pals there do it)...and good for them in that case. Now the logic is applied here...just another variation of the argument against them.

Your own methodology, Mr. President, supported and implemented in the Ukraine, calls for you to call a new election. Do it now!

[Mr_Jefferson_24]

19. Thanks. And I didn't...

...mean to suggest that statistical and other means of examining what took place in November 2004 is unimportant. I think it's very important for it to be done. I just feel that the unconscionably horrible election/voting system foisted upon us is, in and of itself, more than sufficient grounds to cry foul.

[autorank]

20. WE agree! Multiple grouds, independent and interdependent...

...if you can steal an election, the question is do you have the motive and lack the restraint...that's about all we need to know.

Prosecutor Earle has begun a process today that would make your namesake proud. DeLay is the biggest bully on the block. You take him down then scare the others. If I were a Cheney staffer, I'd be very worried what Prosecutor Fitzgerald wsa up to...and we know he's next.

THEN, then then...the rationale for the legitimacy debate rockets to the forefront and we're ready on all fronts.

[Mr_Jefferson_24]

21. "Ready on all fronts"...

...How I yearn for it!

[kster]

44. kick.nt