An anecdote about anecdotal evidence

[DanTex]

As we all know, anecdotes do not constitute scientific evidence. Nevertheless...

Some 10-15 years ago, Atkins and other low-carb diets were going through a fad phase. I saw people go on Atkins and lose tons of weight. But the official story was that it was dangerous, your cholesterol would spike from all the fat you were eating, and any weight you lost would be water and it would come right back when you went off.

I went on Atkins. I lost some 30 pounds rather quickly (about 3 months). My blood tests got better. After I went off, I did not just gain the weight right back.

As a scientific person with scientific friends, I would often encounter people who insisted that you only lose water weight on Atkins and that it was dangerous. And I was like, I understand that there are not many studies backing me, but there's no way in hell that I lost 30 pounds of water. I remember thinking, I wish people would just try cutting out carbs for three weeks, and then come back and tell me about how it doesn't work. I couldn't tell you the biological mechanism, but it was clear that, for me at least, low carb resulted in remarkable weight loss with relatively little effort or hunger.

Fast forward.

In the 2000s, studies starting coming out confirming what I "already knew". For example,
http://www.annals.org/content/140/10/778.abstract
http://www.annals.org/content/140/10/769.abstract


So, does this prove anything? I guess not really.

But it has affected the way I treat personal experience and the dreaded "anecdotal evidence". After all, here we have a situation where my personal experience ran in direct contradiction to what was widely believed at the time. Rationally, my own anecdotal evidence was no match for the medical establishment. And yet, ten years later, it turns out that "my own lying eyes" were right the whole time.

So now, It's not that I take anecdotes to be scientific evidence, just that I don't dismiss them offhand, especially if the person involved is me or someone I know closely and trust. I try to keep an open mind: maybe there's something going on here that hasn't been fully explored yet in clinical studies. In the case of low-carb diets, it turns out there just weren't many/any big studies done comparing low-carb to conventional low-fat diets done until the 2000s. The anti-low-carb arguments were based not on direct evidence, but on the implications of generally accepted theories of nutrition and metabolism.

[CBGLuthier]

1. My anecdote about Atkins

Doesn't matter what you eat if you don't wear good shoes and watch your step.

[elleng]

2. Its just logical, to me: Anecdotes are the beginning of EVIDENCE.

And of course it works! Years ago I cut out bread, and voila!

[cleanhippie]

3. You make a great point

and hit it home (whether you realize it or not) at the end with this: "The anti-low-carb arguments were based not on direct evidence, but on the implications of generally accepted theories of nutrition and metabolism."

Now, think about that for a sec. There is a difference between using anecdotal evidence in the face of hard clinical data (such as the effectiveness of homeopathy) where people who have an experience that they feel contradicts the clinical data, yet ignore the fact that the data actually tells them WHY they feel they had a positive experience: PLACEBO.

Atkins opponents did NOT have any actual data to support their claims, but "on the implications of generally accepted theories of nutrition and metabolism." Actual studies proved Atkins users correct.

[salvorhardin]

4. The plural of anecdote *is* data

That's actually the original quotation, attributed to political science professor Raymond Wolfinger, circa 1984. From a scientific perspective, it's true. Anecdotal evidence is still evidence. It's just that, aside from a single, unconfirmed observance (e.g. one person sees something strange in the sky), it's the weakest form of evidence we can have. There's no way of testing anecdotes, or repeating them, because we don't know what all the initial conditions were, and there's no way of controlling for bias.

Yet if 10,000 people tell you something happened, then it's a pretty good bet that something did happen. Whether that something corresponds in any way to what those 10,000 people said happened is a whole other question. A good example of that is the so-called "Miracle of the Sun"^* at Fatima, Portugal in 1917 where tens of thousands of people were reported to have witnessed the sun doing acrobatics in the sky. That's a lot of people who say they saw something quite remarkable, but which is the more likely hypothesis -- that the sun actually turned cartwheels in the sky or there was a mass hallucination brought about by religious fervor? Either way, the anecdotes are evidence, but what they're evidence of is the question.

So when someone here says "the plural of anecdote is not data", they're technically wrong. However, the larger picture they paint -- that anecdotes aren't very good data -- is still correct.

^*The Miracle of the Sun is pretty interesting, and much more complicated than my capsule synopsis. If you want to know more, Brian Dunning did a good episode of Skeptoid on the topic. It's only a few minutes long and you can either read the transcript or listen to the mp3. Brian also lists some references if you want to dig deeper. http://skeptoid.com/episodes/4110

[ThatPoetGuy]

5. It's interesting.

When Atkins was gaining popularity in the 90s, I was a cyclist, and I spent time with a healthy crowd. My observations at that time were that when people went on an Atkins diet, they lost energy and grew tired and sick -- but that was an observation of athletic people in their late 20s and 30s, who had put on a few pounds over Christmas or from childbirth. I developed a hostility toward the Atkins diet.

Both those studies were of obese people, a completely different group. And maybe Atkins works for them (though I'd still like to see studies of their immune systems and energy levels before I'd conclude it's a good thing). It'd be awesome if it does.

My observations are another form of anecdotal evidence, and those two studies, at least, don't contradict what I observed; it's possible that Atkins can help the obese but harm the could-afford-to-lose-five-pounds crowd. It's also possible that the five-pounds-overweight-and-on-an-Atkins-diet crowd had gone on trendy but unnecessary diets because of private emotional stress, and maybe the lethargy and illness I observed were a result of the stress and not the diet.

[lizerdbits]

6. Athletic people probably need more carbs

than people who may take an evening walk if they exercise at all. I wouldn't be surprised if someone very active became tired on an Atkins type diet.