The Role of Anecdotes in Science-Based Medicine

[HuckleB]

http://www.sciencebasedmedicine.org/?p=33

"...

An anecdote is a story – in the context of medicine it often relates to an individual’s experience with their disease or symptoms and their efforts to treat it. People generally find anecdotes highly compelling, while scientists are deeply suspicious of anecdotes. We are fond of saying that the plural of anecdote is anecdotes, not data. Why is this?

Humans are social storytelling animals – we instinctively learn by the experience of others. My friend ate that plant with the bright red berries and then became very ill – lesson: don’t eat from that plant. This is a type of heuristic, a mental shortcut that humans evolved in order to make quick and mostly accurate judgments about their environment. From an evolutionary point of view it is probably statistically advantageous just to avoid the plant with the red berries rather than conduct blinded experiments to see if it really was the plant that made your friend sick.

Further, the most compelling stories are our own. When we believe we have experienced something directly, it is difficult to impossible to convince us otherwise. It’s just the way humans are hardwired.

Understanding the world through stories was a good strategy in the environment of our evolutionary history but is far too flawed to deal with the complex world we live in today. In fact, the discipline of science developed as a tool to go beyond the efficient but flawed techniques we evolved. Perhaps, for example, your friend became ill because of the raw eggs he consumed earlier in the day, and the plant had nothing to do with it. Evolutionary pressures favored a more simplistic approach to nature, one that tended to assume that apparent patterns were real.

..."


----------------------------------------------------------


This seems like a good time to post a piece that covers this topic well. And, yes, this really needs to be read as a whole.

:hi:

:beer:

[MineralMan]

1. Good article. Well worth reading.

Thanks.

[HuckleB]

8. You're welcome.

:hi:

[cleanhippie]

2. Great post, but I doubt it will have any effect on those that "feel" differently.

[HuckleB]

3. True.

Still, I think it's worth providing an explanation...

[Scuba]

4. The plural of "anecdote" is not "data". n/t

[salvorhardin]

7. Actually...

This is a repost of what I said in the other anecdote thread.

The plural of anecdote is data

That's actually the original quotation, attributed to political science professor Raymond Wolfinger, circa 1970. From a scientific perspective, it's true. Anecdotal evidence is still evidence. It's just that, aside from a single, unconfirmed observance (e.g. one person sees something strange in the sky), it's the weakest form of evidence we can have. There's no way of testing anecdotes, or repeating them, because we don't know what all the initial conditions were, and there's no way of controlling for bias.

Yet if 10,000 people tell you something happened, then it's a pretty good bet that something did happen. Whether that something corresponds in any way to what those 10,000 people said happened is a whole other question. A good example of that is the so-called "Miracle of the Sun"^* at Fatima, Portugal in 1917 where tens of thousands of people were reported to have witnessed the sun doing acrobatics in the sky. That's a lot of people who say they saw something quite remarkable, but which is the more likely hypothesis -- that the sun actually turned cartwheels in the sky or there was a mass hallucination brought about by religious fervor? Either way, the anecdotes are evidence, but what they're evidence of is the question.

So when someone here says "the plural of anecdote is not data", they're technically wrong. However, the larger picture they paint -- that anecdotes aren't very good data -- is still correct.

^*The Miracle of the Sun is pretty interesting, and much more complicated than my capsule synopsis. If you want to know more, Brian Dunning did a good episode of Skeptoid on the topic. It's only a few minutes long and you can either read the transcript or listen to the mp3. Brian also lists some references if you want to dig deeper. http://skeptoid.com/episodes/4110

[HuckleB]

9. Alas, for far too many, no plural is needed.

:crazy:

[patrice]

5. Anomalous data can be, but isn't necessarily, the key to a more or less significant

scientific revolution, so the question is whether those who support its significance have the discipline to identify whether anecdotes are reliable and valid and, thus, scientifically meaningful.

Sometimes people just want to know whether their hypotheses are given just/due consideration.

Science CAN do that; it doesn't actually exclude anything that is framed within its own standards/language, even the anecdotal. Perhaps some people also need to consider what validity is. If they are saying that their anecdotes are valid manifestations of certain phenomena, we need to know relative to what population. If they are talking about something that manifests exclusively in an extremely limited population, that information is of limited use to the rest of us, unless they also identify the difference between the group they are describing compared to others.

[HuckleB]

6. I see DanTex and some others are unreccing science again!

:rofl: