And so our ongoing project to learn about evidence through nonsense enters its sixth improbable year. This week the assembled celebrity community and vitamin pill industry will walk us through the pitfalls of reading through a systematic review and meta-analysis from the Cochrane Collaboration, an international not-for-profit organisation set up 20 years ago to create transparent, systematic, unbiased reviews of the medical literature on everything from drugs, through surgery, to community interventions.
Last week Cochrane produced a gold-standard review, looking at 67 trials describing the experiences of 230,000 people, which showed that antioxidant vitamin pills do not reduce deaths, and in fact may increase your chance of dying.
In the Health Food Manufacturers' Association press release Gloria Hunniford and Sir Cliff Richard issued their definitive refutations. Carole Caplin said: "It must be obvious to everyone who hasn't got a vested interest in supplements that this review is absolute rubbish, it contains fundamental flaws." In a press release issued on behalf of the food supplement industry. Criticising an academic collaboration which does not accept any corporate funding.
So what were these flaws? The pill community was worried by the way that trials were selected for inclusion in the group analysis.
The ex-head of the Harley Street Hale Clinic, Dr Rajendra Sharma - a man who advertises his use of a "bioresonance" machine called the Quantum Xrroid Consciousness Interface to diagnose his patients - explained science to the nation on More4 News: "The writers of this study started with 16,000 studies, and we're asking the question, why did it go down to 68 . Clearly there's a bias that we're not yet quite sure about." Let the mystery be revealed. The answer to his question can be found in figure 1 of the Cochrane report (which of course he read). Of the 16,111 studies which the Cochrane authors found - by using search terms in databases - 12,703 were duplicates, 983 were in children and so not applicable to this review's predescribed remit, and so on.
<snip>
http://www.guardian.co.uk/commentisfree/2008/apr/26/medicalresearch.health