Now this again: exit polling is impertinent and irrelevant!

[ngant17]

AP just floated a news story for 03 Oct 2008:
<http://news.yahoo.com/s/ap/20081003/ap_en_ot/tv_exit_polls>

The whole slant sounds rather sinister or foreboding if you ask me.

Nowhere in the story does it imply or suggest that exit polling was designed as a type of statistical quality control in the vote-counting process. This is how I understand exit polling was developed originally. And this is how I understand it is used across the world today.

Exit polling has been supported by professional statisticians so it's not psuedo-science as the article implies.

The story trys to dwells on subjective or personalized factors which it is claimed will bias the results of the entire population of surveyed voters. Here's a typical quote:

"In general, Republicans tend to be less enthusiastic than Democrats about taking exit polls."

IOW this is not just a silly way of dismissing exit polling, this statement actually seems to be saying that Republicans care less than the Democrats about the overall integrity of the democratic process and the integrity of the election system itself. Which could be true, I suppose.

According to what the article is suggesing, it is scientifically impossible to take independent, random samples from a population set.

So now that the AP press has come up with some ridiculous reasons to dismiss exit polling, how do they propose to have a clean election for 2008?

Really, whether you are manufacturing widgets in an industrial factory, or counting votes in an election office, you have to use some kind of statistical process control (SPC) to insure integrity in a given process. Exit polling is the best tool that's been invented to keep out defective hardware, intentional deception or fraudulent vote-counting. How can anyone assume that a properly designed exit poll will always introduce bias?

To dismiss this type of SPC when it's actually needed more than ever, this is not an example of bad reporting but I believe it goes deeper than that, it's a deliberate attempt to destroy our democracy itself.

[WCGreen]

1. When I was a young man, way back in the 70's, I did polling work for

one of the local TV stations.

It was exit polling and it was done in order to show who was winning throughout the day.

Well, there was an uproar about this and people didn't want to know, so the story went, how the election was going before the votes were cast. So the station stopped the poll.

It was never intended, from my personal understanding, as a quality control on how the election was being conducted from the nuts and bolts side of the equation. It was developed as a marketing tool by media outlets.

[ngant17]

3. that's a totally different application

I don't believe for one minute that marketing analysis is the motivation why exit polls are used on election day. No one is trying to sell something, or get someone to buy something.

It's clear that educated, trained statisticians are able to view exit polling in the manner that I have stated, as a variation of a process control, applied to the voting and elections.
See <http://uscountvotes.org/>
<http://uscountvotes.org/ucvAnalysis/US/paper-audits/VoteCountAudits-PPMEB.pdf>

Now I suppose all that means the whole field of statistics must be denounced as a leftist conspiracy and, for that matter, US university faculties are biased to the left. By the same people who will suggest that global warming is a political issue and has no scientific relevance.

[WCGreen]

5. Exit polling was developed by the media and the polling organizations

to get a feel for the vote on election day.

It's different now.

But in the 60's and 70's, that's pretty much what exit polling was all about.

I did it for two years, stood at the poll, entered the stuff in the computers at the TV station, sat with the professor and watched how he read the data and then hand it over to the TV station that had hired us to do the survey.

It's marketing analysis, it was a news tool like Dopler Radar.

I was there. I saw it all. From developing the polls, to deciding what voters were targeted and when. And guess what, it was pretty acurate.

[OnTheOtherHand]

9. you've been very much misinformed

You're welcome to interpret exit polling as a "variation of a process control" if you want, but that isn't what it was invented for in the United States, as the other poster has explained.

This isn't to say that exit polling is a "pseudo-science." Blind faith in exit polling is pseudoscience. We can try to interpret the data as process control data, but we can't simply assume its accuracy. (That would be true, of course, even if the data were collected for process control.)

I have no idea why you linked to the US Count Votes paper you did: as far as I can tell, it has nothing to do with exit polls. As for US Count Votes generally, the analysts never really got around to assessing the evidence. Most of them peeled off in mid-2005.

As I've mentioned many times here, the exit poll Best Geo projections indicate that Kerry won Pennsylvania by about 14 points, and New Hampshire by 15 points, and New York by over 30 points (and so on). I don't think any knowledgeable political observer believes any of these estimates.

Of course, the exit polls could be "wrong" and still contain useful information about vote miscount. Analyses such as this one indicate that, at best, the 2004 exit polls contained very little such information. The exit polls actually do not support the inference of massive, widespread miscount favoring Bush.

If you are interested in discussing these issues, I'm happy to do that.

[ngant17]

17. you can't divorce exit polling from statistical science

because it is based on independent, random sampling of a population set.

There are dozens of PhD statisticians in the US who have supported the scientific work behind exit polling.

As one example, Professor Steven Freeman, Ph.D., a statistician at the University of Pennsylvania, has conclusively shown that there is only 1 chance in 250 MILLION that the 3 main exit polls (OH, FL, PA) conducted during the 2004 general elections could all be wrong. This is strong evidence that fraud was the cause and the election was stolen in the previous presidential election.

Elsewhere statisticians have noted that precincts with hand-counted paper ballots showed no statistical discrepancy between the exit polls and the official results, but for other voting technologies, i.e. black box voting, the overall discrepancy was far larger than the polls' margin of error.

OK, assuming you are right, then if exit polling is not a variation of SPC, we do need to start implementing SPC into the election and voting process while we still have some vestiges of a democracy left. And you want to call that something else besides "exit polling", it doesn't matter, it's "6 one way, half-a-dozen another", but there has to be a redundant process, based on science and logic, which can insure integrity in the elections. I won't argue with you about the semantics of the word.

[OnTheOtherHand]

18. still some problems here

While an exit poll is intended to be a random sample, as a practical matter, the interviewers often cannot choose at random from all voters at a polling place. Even if they could, the voters retain discretion to participate or not to participate. I would never propose to "divorce exit polling from statistical science," but I do insist that random sampling error is only one source of error.

Steven Freeman is not a statistician. Or, if you prefer, he is no more a statistician than I am. His Ph.D. is in organization studies.

I doubt Freeman ever claimed that there is "only 1 chance in 250 MILLION that the 3 main exit polls (OH, FL, PA)... could all be wrong." Although I disagree with Freeman about many things, I'm pretty sure you are misstating his view. What Freeman said was that there was a very small chance that the discrepancies "could have been due to chance or random error." (See here at page 13 -- this is a revised edition with somewhat different odds.) The probability that the polls "could all be wrong" can't be computed without heroic additional assumptions -- for instance, that non-response bias is impossible.

As I mentioned, the exit poll data indicated that Kerry won Pennsylvania by about 14 points. The final pre-election polls ranged from Kerry +4 (Zogby) to a tie (Quinnipiac). It seems rather selectively credulous to assume that the exit poll was correct.

The mean WPE reported for the 40 hand-count precincts in the sample was -2.2. It's true that that value is not statistically significantly different from 0, there being only 40 precincts to analyze. However, the mean for hand-count precincts isn't significantly different from that for precincts using other voting technologies in either urban areas or rural/small town areas. (Freeman complains that this is an unfair comparison, but the truth is, there just aren't enough data to draw a solid inference.)

I'm certainly not against SPC. I'm not even against trying to use exit poll data to support that effort, as long as we are accurate and honest about what we can infer from it. I advocate stringent post-election audits of hand-marked paper ballots, or failing that, voter-verifiable paper records.

(edited subject for clarity)

[ngant17]

21. vote fraud 2008

Thanks for pointing out that Steve Freeman is not a statistician by training.

BTW there is a difference between random sampling and random sampling error. Statistical errors are factored in, after the sampling of the population set.

But more importantly, 12 out of the 13 swing states in this election will be vunerable to undetected vote miscounts, with no audits possible in almost half of these states!

[OnTheOtherHand]

22. ?

BTW there is a difference between random sampling and random sampling error. Statistical errors are factored in, after the sampling of the population set.

Certainly there is a difference between sampling and error; I think we all know that. I don't see what bearing that has on any of the other claims you've made in this thread.

But more importantly, 12 out of the 13 swing states in this election will be vunerable to undetected vote miscounts, with no audits possible in almost half of these states!

I'd be interested to see your list; that "almost half" seems high to me. Maybe it depends on what we mean by "possible." Regardless, post-election audits this year will not be what they should be.

Again, this point doesn't seem connected to any of the other points you've made.

[ngant17]

26. Still the song remains the same

Audits, exit polling, and election fraud, that's what I have been trying to discuss. The subjects can be related at times. And if I have to repeat and emphasis the facts until it is clear, I don't mind doing so if it will help anyone to understand.

The list for unaudited records in swing states:
http://uscountvotes.org/

<http://electionarchive.org/ucvAnalysis/US/2008Election/SwingStatesSusceptible2008.pdf>

I don't have a problem with associating SPC and exit polling. These are parametrical statistics.

More robust ways have been developed, of measuring the accuracy of votes, but part of the problem is lack of openness from the government in supplying the public records.

"In democracies with hand-counted paper ballots such as Germany and the United Kingdom, exit polls predict the outcome of national elections with extreme accuracy. Around the world, exit polls have been used to verify the integrity of elections, and discrepancies between exit polls and the official vote count have been used to successfully overturn election results in Peru, Serbia, Georgia, and Ukraine. At the same time that Republican National Committee Chairman Ed Gillespie was saying, "I would encourage the media to abandon exit surveys on Election Day,” the administration was paying for exit polls in the Ukraine to help ensure that any fraud committed would be exposed." Steve Freeman
<http://blogcritics.org/archives/2006/09/30/1008004.php>

"If you are going to steal an election you go after votes most vigorously where they are most needed. In this case, the 11 swing states where the election was close. As it turns out, even though there is no reason why exit polls should be more or less accurate in key states, the discrepancy is significantly higher in the swing states than other states and significantly higher yet in the three critical battleground states of Ohio, Florida and Pennsylvania."
<http://blogcritics.org/archives/2006/09/30/1008005.php>

In the 2000 election in Yugoslavia, Milosevic claimed that he had defeated Vojislav Kostunica. However, exit polling cast great doubt as to the accuracy of the reported count, suggesting that the vote count had been corrupted. Public and media pressure eventually forced him to concede, turning a stolen election into an honest election result.

I repeat for clarity --- exit polls were not consistent with the official recorded vote. A significant difference between exit polling data and the official recorded vote tally was PRIMA FASCIA evidence of election fraud and was sufficient evidence to INVALIDATE THE ENTIRE ELECTION and cause the entire country to recast their ballots.

Sure, verifying votes thru exit polling in this election is just one part of the solution. We also must not lose track of the other ways the Repugs will attempt to disenfranchise the American people from voting.

"In what may be the single most astounding fact from the election, one in every four Ohio citizens who registered to vote in 2004 showed up at the polls only to discover that they were not listed on the rolls, thanks to GOP efforts to stem the unprecedented flood of Democrats eager to cast ballots."

Robert F. Kennedy, Jr., Rolling Stone article
"Was the 2004 Election Stolen?"

[OnTheOtherHand]

27. for what it's worth

In the 2004 U.S. election, the largest modeled exit poll discrepancies were in Vermont, Delaware, New York, New Hampshire, and Mississippi. Pennsylvania tied for sixth with New Jersey. Freeman's claim isn't wrong, but I would construe it as misleading. Were the forces of darkness frantically stealing votes in Vermont, Delaware, and New York? (Or even New Hampshire, for that matter -- considering that the final result was close to that indicated by pre-election polls?) Freeman is remarkably selective in his discussion of this subject.

I've seen no evidence that exit poll evidence has been used to overturn election results anywhere, and in Ukraine, I'm pretty sure that it wasn't. It did help to rally demonstrators, but it wasn't the evidence cited by international observers or relied upon by the Supreme Court. Likewise for Yugoslavia in 2000, asserting that exit polls were "sufficient evidence to INVALIDATE THE ENTIRE ELECTION" doesn't make it true. As for the rest of that sentence, the country didn't recast its ballots. Maybe you were thinking of Ukraine in that respect? (It's hard to tell whether exit polls offered prima facie evidence of fraud in any of these elections, given that there was other evidence of fraud in all of them.)

RFK Jr. was a bit off the wall there, too: the figure is derived from a study of Cuyahoga County. The study estimated that 26.5% of new registrants in the county had to vote provisionally. It also found that a roughly equal proportion of Bush and Kerry voters in Cuyahoga County had to vote on provisional ballots. So, it's at best speculative that "GOP efforts" contributed to the high provisional rate among new registrants in the county. Nevertheless, I agree with your broader point. We've seen all kinds of schemes to keep people from voting.

[Jackpine Radical]

2. The story is intended not only to attack the credibility of exit polls, but to

explain in advance why the exit polls aren't going to reflect a large Democratic majority. The story seems to be that Republican voters are too ashamed of their votes to admit them to pollsters.

Wow. Blatant or anything?

[99th_Monkey]

4. Memo to AP: Exit polls have been the gold standard for elections for decades ...

especially in OTHER countries, universally used by International election monitoring organizations
to determine if the election was being manipulated fraudulently, or was won fair & square.

How they can be spouting this bullshit with a straight face? Esp. when everyone with an ounce of
knowledge about elections knows otherwise.

Sadly, I expect to hear more of this from M$M however, as the election nears; to try to provide a
fig leaf for the next stolen election.

[emlev]

6. I just wrote about this over on this video thread

Cynthia McKinney Takes 'Standing For Voters' Election Integrity Super Pledge. Come and 'kick it' over there?

[99th_Monkey]

7. Done. Thanks for the heads up. ~nt~

[Wilms]

8. And sometimes those foreign exit polls are domestically sponsored.

He said the exit poll would "enhance the credibility" of official vote results.

http://www.democraticunderground.com/discuss/duboard.php?az=view_all&address=203x488796

[OnTheOtherHand]

10. O RLY?

You have evidence that for decades, exit polls have been "universally used by International election monitoring organizations to determine if the election was being manipulated fraudulently"?

Avoid exit polls, which are unreliable in a climate of suspicion and which will create a negative atmosphere if the voters feel they are being watched.

Carter Center, Elections in Mexico: Third Report, August 1994

I could keep going (for instance, did you know that the OSCE report on Ukraine 2004 doesn't mention the exit polls at all?).

[99th_Monkey]

11. Exit Polls in Ukraine

Published on Sunday, December 26, 2004 by Reuters
Exit Polls: Liberal Yushchenko Wins Ukraine Election


KIEV, Ukraine - Exit polls in the re-run of Ukraine's presidential election Sunday said liberal challenger Viktor Yushchenko had beaten Prime Minister Viktor Yanukovich by a wide margin.

Yushchenko, who called crowds of supporters into the streets to denounce cheating in the last poll, scored 56.5 percent to 41.3 percent for Yanukovich, according to a poll by the Kiev International Institute for Sociology and the Razumkov Center.

A second poll, by the Center for Social Monitoring, gave him an even wider lead -- putting his share of the vote at 58.1 percent and Yanukovich's at 38.4 percent.

Yanukovich had initially been declared the winner in last month's run-off vote, but his victory was overturned by the Supreme Court which agreed with opposition charges that the election was rigged in his favor.

The figures in the first exit poll represented 80 percent of a sample of 30,000 voters who were polled across the former Soviet republic. The second exit poll surveyed 13,000 voters.

http://www.commondreams.org/headlines04/1226-06.htm

[Wilms]

12. Exit Polls in Ukraine

The Hand of the CIA?

Blowback in Ukraine

November 29, 2004

By DAVE LINDORFF

An interesting and disturbing aspect of the post-election uprising in Ukraine is that it is not entirely a spontaneous event.

Although the election outcome was clearly manipulated by the government of outgoing president Leonid Kuchma, a corrupt and dictatorial Soviet-era autocrat, whose supporters in and out of the government's secret service apparatus have not shied away even from killing opponents, and although much of the movement that took to the streets in Kiev to protest and overturn the results, which had given victory to Kuchma's handpicked successor, Viktor Yanukovich, has been indigenous and heartfelt, there is also clear evidence that the U.S.--the CIA and various American "pro-democracy" front groups--is playing a crucial hand in destabilizing the pro-Russian regime.

Several excellent pieces in the British paper the Guardian have highlighted the role of U.S. agencies and NGO's, in helping, for example, to finance the very exit polls that have raised doubts about the outcome of the election, and in helping to pay for Yushchenko's campaign itself.

As the Guardian observes, it is more common than not that elections in the former Soviet republics have been manipulated by government authorities, who control most of the media, and especially television, as in Ukraine, and have often been stolen. The interesting question is why this time, in Ukraine, the U.S. government has taken such a strong position on behalf of the opposition.

http://www.counterpunch.org/lindorff11292004.html


Now. How about a link supporting your claim of exit polls as "gold standards"?

[99th_Monkey]

14. Exit Polls as gold standard for fair elections 101

"This is the book I have been waiting for. Freeman and Bleifuss are reasonable, balanced, and insistent--addressing the questions about the 2004 elections that won't go away. They specifically target the exit polls and explore how these exit polls (the gold standard of polling) could have been so far off from the official count in so many states. Readers will have the opportunity to step outside a comfortable conventional wisdom, not into a world of conspiracies but into the territory of careful searching, combining the best features of science and true investigative journalism."
http://www.electionintegrity.org/book.shtml
—Karen Parker Lears, Raritan Review (Rutgers University)

"Election day voters’ polls are commonly called “exit polls,” since the data are collected from voters as they “exit” their places of voting. The person best known for developing exit polls was a statistician named Warren Mitofsky, who died last September 1. According to Richard Morin, then polling director for The Washington Post, Mitofsky “introduced exit polling to the world, including to countries new to democracy and free elections. His reputation for accuracy and independence—a reputation he fiercely guarded—made his exit polls the gold standard with which election results were compared and confirmed.” And yet exit polls have been surrounded by controversy in recent years, with some questioning the accuracy—and hence the value—of the data they yield."
http://www.publicopinionpros.com/features/2007/jan/shapiro.asp
--Robert Y. Shapiro and Lisa Ferraro Parmelee (Robert Y. Shapiro is a professor of political science at Columbia University. Lisa Ferraro Parmelee is the editor of Public Opinion Pros.)

[OnTheOtherHand]

15. not there yet

Karen Parker Lears appears to be a bit out of her field here, but setting that aside, to say that exit polls are "the gold standard of polling" is not to say that they're the "gold standard for fair elections."

Your quotation of Shapiro and Parmalee quoting Morin is at least about using exit polls to confirm election results, but gets nowhere near your claim that "international election monitoring organizations" rely on them. Actually, if you clicked through to Beltran's article "Warren in Mexico," you know that Mitofsky's polls there were commissioned by Mexican media. If Mitofsky ever worked for an international election monitoring organization, that would be interesting to know.

(It might be worth mentioning that Shapiro and Parmalee appear convinced that the 2004 U.S. exit poll was erroneous, but maybe you agree with that.)

[99th_Monkey]

19. My point was/is, NOT that exit polls are ALWAYS 100% accurate, but that ...

... for "decades" exit polls have had a "reputation for phenomenal accuracy" ... hence the "gold standard" reference.

"Over time, the exit polls acquired a reputation for phenomenal accuracy. From 1967 to 1988, CBS made only five mistakes in approximately fifteen hundred calls. During the 1990s, VNS had only one error in seven hundred projections. This excellent record derived more from careful interpretation of exit poll data combined with election returns than from any uncanny accuracy in the data themselves, a distinction lost on many casual observers. For instance, the 1988 CBS exit poll interviewed more Dukakis voters than Bush voters, although Bush won the election by about eight percentage points. Here, as elsewhere, the unweighted results were misleading. Much (but not all) of the discrepancy can be attributed to an overrepresentation of black voters, which CBS presumably could detect and compensate for as the results came in—although, seventeen years later, we are unable to reconstruct the election night scene. CBS did incorrectly call one state (Illinois) for Dukakis, although not primarily based on exit poll data."

See middle of page 2 at:
http://www.publicopinionpros.com/from_field/2006/jan/lindeman_i2.asp

[OnTheOtherHand]

20. that may have been the point you intended to make

but it isn't the point you actually made. I could quote it to you again and see whether you see it this time!

This time, you're actually quoting an article I coauthored, and I'm flattered -- but I certainly didn't argue that the exit poll data were being used, or should be used, to check the accuracy of the election results. The money quote might be: "This excellent record derived more from careful interpretation of exit poll data combined with election returns than from any uncanny accuracy in the data themselves..." (emphasis added). Some people talk as if the pollsters "call" every race the moment the polls close, but people who actually watch the coverage realize that this isn't true. The standard of "accuracy" here is whether the pollsters eventually make a call they have to retract, not whether the data or models initially point to the eventual winner. (Note that this standard assumes that the official results are correct.)

[99th_Monkey]

23. OK, I stand corrected, but find it more than a little depressing

that exit polls may not be as reliable as I imagined, because what other way is there to
detect fraud with these damn e-voting machines?

Anyway, thanks for the exchange. :hi:

[Wilms]

24. Risk-Based Audits is a way to check election results

http://www.democraticunderground.com/discuss/duboard.php?az=view_all&address=203x504485

[OnTheOtherHand]

25. yeah, I take your point

For the paperless DREs, we're really up a tree for now. Some attacks or actual glitches can be detected and corrected, but let's not kid ourselves.

On the other hand, there's decent reason to think that they, or most of them, weren't hacked in 2004. Of course, past performance is no guarantee of future results, but it's some basis for hope.

Also, it's worth noting that we have lots of statistical tools we can use to (possibly) detect fraud. Even the exit polls can be somewhat useful in that respect, even though they can't be assumed to be accurate. Now, detecting fraud may not be enough to overturn an election result, but it's better than not.

In the long run, I don't think such bad machines will stay around.

It's been good 'talking' with you.

[Wilms]

16. Thanks for the reference.

In the second example the authors, too, wonder about the value of such polls.

I don't hold anything against exit polling, per se. I just think they're as reliable as the voting system is in general in this country at this time...which is not very much.

Do I think 2004 was stolen? Yes. Do I think the exit polls prove that? No. Do I think using exit polls as an argument to question the results of the election will do anything of import? The record, nearly four years later, is clear.

Have you pondered the Ukraine story I posted? Or the one further up regarding the International Republican Institute?

[OnTheOtherHand]

13. ?

Are you asserting that the Kiev International Institute for Sociology, Razumkov Center, and/or Center for Social Monitoring are "International election monitoring organizations"?

So far you haven't even gotten to one, which is damn far from "universally." (It's not my impression that the outside groups that supported these exit polls count as international monitoring organizations, either.)

It's helpful, however, that you cite two Ukraine exit polls from the second run-off, one of which gave Yushchenko a 15.2-point margin, the other of which gave him a 19.7-point margin. The difference between those two exit polls is almost as large as the exit poll discrepancy in the United States. I believe that in the first run-off, the difference between the two exit polls was actually larger than the U.S. exit poll discrepancy. So when people cite Ukraine as evidence of exit poll accuracy, I have to wonder. (I'm not saying that the polls are "impertinent and irrelevant," just that obviously they aren't always accurate.)

Luckily, there was lots of evidence of election fraud in the first Ukraine run-off, and the election was overturned based on that evidence. Also, the difference between a 15-point margin and a 20-point margin isn't all that consequential.