Why Most Published Researching Findings are False

[MountainLaurel]

Interesting article from the Public Library of Science Medicine open access publication.

There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research.

[wtmusic]

1. "Simulations show....it is more likely

for a research claim to be false than true..."

What does research about simulations about research claims show?

:silly:

With specious creds like the author's, this looks pretty worthless.

[enlightenment]

2. Just out of curiosity (not trying to flame)

what do you find specious about the author's credentials? The school he teaches at or something I'm not seeing? I'm assuming you're using the word 'specious' in the sense of 'false' -- do you believe that the credentials are fake?

[wtmusic]

4. No title, no degree, sounds like a load of BS

"John P. A. Ioannidis is in the Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece, and Institute for Clinical Research and Health Policy Studies, Department of Medicine, Tufts-New England Medical Center, Tufts University School of Medicine, Boston, Massachusetts, United States of America."

Is he "in" those places kinda like Michael Moore was "in" General Motors?

[MountainLaurel]

5. He's an adjunct

Which probably means that he's a practicing physician who also lectures within the med school. (Med school classes usually include several different lecturers for a particular course.) He's got more than 300 article citations in PubMed (the main database of peer-reviewed medical literature in the U.S.). What more does he need in terms of credentials?

[wtmusic]

6. Usually if he's a physician he'll list a degree abbreviation

as will his title as in "Adjunct Professor", "Guest Lecturer", and so on.

Extremely general statements like "Most Published Researching Findings are False" don't have a lot of credibility either. What does "published" mean? Does it include the internet? Are we talking about only medical research? How can he "prove" this hypothesis without actually reviewing all research -- an impossible task?

Statements like this sound ridiculous...

"As has been shown previously, the probability that a research finding is indeed true depends on the prior probability of it being true (before doing the study)..."

The probability of a research finding being true is exactly the same before a study as it is afterward.

After this he dives off into deep mathematics (based on his own estimations and virtually no data) which is often a smokescreen for lack of rigor. I may be off-base with this, but I would love for someone who is well-versed in statistics to show me where I'm wrong.

[varkam]

7. Trying not to flame as well...

but does that mean that those of us without MDs are completely retarded and our opinions aren't worth a glance?

on edit: John P.A. Ioannidis' CV

[trotsky]

9. Yeah, something doesn't seem right here.

That's a bold claim to make - and one that would be on much more solid ground had he actually taken real studies over a given timeframe and analyzed their "truthiness." ;-)

As it is, with such a generalized conclusion, this will simply be used by the science-haters to dismiss valid studies that they simply don't agree with. (Not that they didn't do so before, but now they have an actual SIMULATION to back it up!)

[wtmusic]

3. self delete

[bemildred]

8. Seems right to me.

Adding some mathematical rigor to an already obvious situation.

[BrklynLiberal]

10. The first question I always ask is "Who financed the study?"

You know, the same old rule that always applies: FOLLOW THE MONEY.
that usually will give one a good idea whether or not the study was credible to being with, regardless of anything else.
If it starts out looking for predetermined results, the rest is garbage.

[Odin2005]

11. What a bunch of anti-intellectual BS garbage

This is the same kind of crazy reasoning creationists and global warming denialists use. It's basically a bunch of post-modernist sophistry masqeurading as a logical argument. I wonder if he's trying to push some kind of altenative medicine snake oil.

[oscar111]

12. drug firms refuse to let professors see results of prof's own studies

the drug firms financed the studies, and not liking the results, they blocked the investigators from seeing their own results.

paired with financing lots of studies and only releasing those which favored their drug,

...

I find the OP to be likely correct, if possibly for the wrong reason.

I cant evaluate the OP without more time, but looks to be right to me. Shocking.

OP may overstate the percent wrong, but the two factors i cite here are not yet ended and are shocking.

Medicine, clean up all corners of your "house". You are a crime center in our land.

25 nations outlive us , even tho we pay the most for your services.

[azurnoir]

13. Not to side with anyone here

but I have an example of how stats can be made. When my son was a baby he got a middle ear infection (those who are parents are most likely familiar with this) routine trip to the doctor, he was going to a pediatric clinic at a large teaching hospital at that time. The appointment was routine and appropriate for the problem, until the resident was writing his prescription-he wrote it and started to hand it to me then stopped and asked "is there a smoker in the home?" yes, I replayed there was. He then balled up the prescription, threw it in trash, and preceded to write another. He gave me that and left the room, I had to wait for the nurse to bring in a return appointment slip. Being curious I pulled the first script out of the trash it was for a 14 day course of Amoxicillian 5cc three times a day. The script he gave was for the same drug same dosage but only for 8 days. I asked the nurse about this she said he "probably made a mistake and if I wanted to talk to him she'd page him ,but he had gone off duty. So I had the script filled son took anti-biotic and everything was fine for about 12 days then wham ear infection back and process repeat. This was 10 months or so before the study came out that children in homes with smokers have more frequent ear infections and more that need to be treated multiple times. I called the hospital and after being redirected several times got someone who would answer questions. Yes they sell stats, yes they participate in studies, yes they had participated in that study, and no they don't tell patients unless they're taking an experimental drug or treatment.
I'm not saying the smoking-ear infection link is false, but it does seem in this case it helped along a bit. I wondered just how many patients were subject to this and yes I did immediately find another doctor.