Why Zogby polls are crap

[ringmastery]

http://www.washingtonpost.com/wp-dyn/articles/A41186-2004Jan23.html?nav=hptoc_p

Why do so many pollsters go squinty-eyed when Zogby's name is mentioned? Here are some of their biggest complaints about the ubiquitous pollster from Utica:

Zogby draws his samples only from computer files of people with listed telephone numbers, thereby missing 30 percent of the population with unlisted telephone number.

The biggest advantage to what Zogby does is that most of those telephone numbers are good. The disadvantage is that you miss everyone with an unlisted number, and people with unlisted numbers tend to be different than those who are in the telephone book. Most other public pollsters rely on Random Digit Dialing, a technique that captures both listed and unlisted numbers. (Zogby says he plans to release a study later this year that shows no demographic or ideological differences between people with listed numbers and those who are unlisted.)

Zogby also calls people during the day as well as in the evening. About 30 percent of his interviews are collected before 5:30 p.m. Daytime interviews are great if you want to talk to lots of retirees and housewives. But his critics charge that they're not so good if you want to interview working men and women. Zogby counters that daytime interviewing actually produces a more representative sample because he can talk to people who work at night.

Zogby also adjusts his sample based on historic trends and his judgment of "what is happening on the ground" in a particular race, and it is this imposition of his own judgment that disturbs many pollsters.

He will, for example, reduce the proportion of 18- to 24-year-olds in his sample of self-described likely voters if he suspects on the basis of past voting history and the "lay of the land" that a sample contains too many younger people. He also, on occasion, adjusts the religious composition of his sample if he suspects he has over or under-represented one faith.

Most pollsters cringe at such extra-curricular adjustments. "I know I do some things different that others," he said. "I know the so-called 'Poll-ice' would deny it, but there's art as well as science involved in this."

[Namvet04]

1. He was way off with his Iowa picks

I think we need on here a list kept by state of the final poll numbers by each group and see if we can find one we can trust.

[JCMach1]

2. He had the order right though... if not the numbers.

[robcon]

6. Zogby had Dean at 19% - he got 18%

Zogby had the order right, and was closer than others on the leaders.

[JI7]

3. as long as we know what he takes into account

i don't trust just about any polls. but i do want them to release the methods they use to get the results.

[TruthIsAll]

4. Zogby was the ONLY pollster to get Gore/Bush exactly right..

CNN/Gallup was gyrating all over the map from week to week.

Zogby was right in the 2002 senate races, but Diebold patched 22,000 machines in GA on the final weekend. Exit Polling was canned at the last minute. So the media spread the garbage that Zogby polling was bad.

The standard method is random, weighted statistical sampling via phone, using (3-5) day moving average daily tracking to smooth out the trendline.

[JCMach1]

5. Caucuses are really impossible to pick numberwise

because of the 15% threshold thing.

For example DK may very well have had the support of 14%... However, we will never know because of the rules.

[Awsi Dooger]

7. You keep believing in Zogby and I'll thrill to bet against him

True, his 2000 national margin was near perfect. But his state polls are laughably inept, in 2000 and 2002 and earlier.

In 2000 his final poll had Rick Lazio just slightly behind Hillary in New York. Consequently, a major offshore betting outfit plunged the odds on Hillary down to 5/8. That was essentially equivalent to wagering 160 to win 100 that Tiger Woods would break 100.

Also in 2000, Zogby's final California poll had Bush within a couple of points of Gore. The offshore outfit overreacted again and I was able to lay 2/5 on Gore in California, instead of off the board entirely.

Zogby's 2002 senate numbers were every bit as hysterical. You conveniently leave out Colorado, minus any Diebold conspiracy crap, where he had Tom Strickland leading by double digits, nearly 20 points askew from actual. He was wrong in Minnesota, way off in Texas and virtually everywhere else.

Georgia state polls are ridiculously generous to Democrats, and have been for more than a decade. Zell Miller barely escaped in '94 gov, Max Cleland the same in '96 senate after being expected to cruise, and Bush dumped Gore by roughly 8 points more than was projected. All that came before wimpy Ralph Reed and his 2002 tactics, which were as effective as they were disgraceful and led to the upsets of Cleland and Barnes.

[robbedvoter]

8. His 2000 margins came after an October exposee in the Voice

that he goosed the numbers for the GOP. Zogby barely bothered to deny it - then did the next poll straight - hence your awe of him/
Here's Moderate Independent's opinion:

NEW HAMPSHIRE UPDATE – A WORD ABOUT POLLS

by Betsy R. Vasquez
http://moderateindependent.com/
JANUARY 24, 2004 –  The race remains Kerry leading, Clark in second, Dean down in third, Edwards fourth.

Why are our numbers so different from the rest of the media?

Because they use the untrustworthy, right-wing puppet polls, like the Zogby polls.  As the Washington Post reported yesterday, "ABC's polling department has reviewed (Zogby's) methodology and rated his polls "not airworthy."  This admission came directly from John Zogby himself.More...

[Bombtrack]

9. If Zogby's crap, they're all crap, because who's better?

media polls(newspapers, tv stations) are always infinitly inferior to the big wigs like Zogby. They basically do it to have a story.

[quaker bill]

10. Zogby was the only one to call Fla a tie in 2000

The rest all had Bush leading by 5+ percent.

[incapsulated]

11. All polls are crap

I'm sick to death of them. Yes, they give you a very broad idea of trends, but the overemphasis on them here and in the media gives them far more credibilty than they deserve.

[Name removed]

12. Deleted message

Message removed by moderator. Click here to review the message board rules.

[Lydia Leftcoast]

13. I was getting Zogby Internet polls for a while in 2001-2002

and whenever I looked at the cumulative results, I couldn't help noticing that his sample was heavily slanted towards Republicans. Sometimes more than half the respondents self-identified as Republicans, so OF COURSE they were going to give Bushboy high ratings.

If the polls had been truly representative, the proportions of Ds, Rs, and Indies would have been more like their distribution in the general population.

On Edit: He got a lot of credit for being the only one to predict that Gore would win the popular vote in 2000, but I'm reminded of that psychic (Jean Dixon, I believe her name was) who predicted in 1962 that Kennedy would be assassinated. She turned that guess (almost her only prediction that panned out) into quite a lucrative career as a TV personality and writer.