August 9, 2011

"Mistakes in Scientific Studies Surge."

The Wall Street Journal reports:
Why the backpedaling on more and more scientific research? Some scientific journals argue that the increase could indicate the journals have become better at detecting errors. They point to how software has made it easier to uncover plagiarism.

Others claim to find the cause in a more competitive landscape, both for the growing numbers of working scientific researchers who want to publish to advance their careers, and for research journals themselves.

25 comments:

Sal said...

All hail the consensusists, er, I mean, scientists!

I'm Full of Soup said...

Perhaps, it's due to global waning [in ethics and honesty].

Sprezzatura said...

Come on!

These scientists can't even figure out how they should teach folks to perform CPR. It seems like they come up w/ a new preferred procedure every decade!

Stop screwing up!!

glenn said...

First you have to have honest people.

Anonymous said...

If the only avenue to money is publishing results, soon every experiment looks groundbreaking.

I've personally talked to graduate students in chemistry that participated in one single success in an experiment, with a half a dozen failures to reproduce the results, and their professor figuratively put the results in a 'jogger' to get the edges square and published it anyway.

SteveR said...

Publish (bullshit) and get tenure. Sounds like a plan

Wince said...

Al Gore explains: "BULLSHIT!"

Mark D. said...

To get tenure and grants, scientists who work at universities have to publish. Since often tenure and grants are predicted on the quantity of research published, rather than the quality, there is an obvious incentive for people to publish as much as possible. In such a system, accuracy will fall prey to production. Want a different outcome? Get different incentives.

sorepaw said...

From one of the graphics accompanying the WSJ piece, it looks as though there have been more retractions in medical journals than elsewhere.

Carol_Herman said...

The price of "publish or perish."

While ya know what? All the theories that get published in scientic journals are supposed to bring about a response from others.

Theories are shots in the dark.

Theories get countered all the time.

And, sometimes the person who comes out guessing it right ... was the loner.

traditionalguy said...

The research lab under Jonas Salk at U. of Pittsburgh was doing amazing complex work on polio vaccine with a killed virus, but one that had been killed without destroying it.

That left a small chance that sloppy manufacture could leave an impotent amount of killed virus in a batch.

His bitter rival named Albert Sabin slandered and connived to deny Salk funding and tests for a killed virus approach for 10 years so his live virus approach would win.

Scientists often sabotage a rival scientist's work.They do it by false reports and dogmatic theory that they know is untrue.

And then along came Al Gore.

Fred4Pres said...

Science makes false and dead end theories all the time. Scientific consensus takes time and lots and lots of experimentation, with major shifts being resisted vehimently until they are accepted.

As for climate change, it was resisted, then adopted and now the shift is happening again.

Anonymous said...

I see Fred4pres has read his Thomas Khun.

David said...

I think there are more idiots with advanced degrees.

chuck b. said...

Science is increasingly reliant on statistical methods and analysis and statistical literacy among scientists is quite low. I speak from experience as a former working scientist, and a current statistics graduate student.

But, imo, that is not the only reason, or even the main reason, and it isn't an either/or situation.

I'm also not convinced the situation is quite what the WSJ describes, but they make a good argument.

caradoc said...

Most likely it because they're finally getting exposed when caught instead of having it conveniently dropped down the memory hole.

rhhardin said...

There are too many statistical experiments, with the result that even outliers are a huge population and get published as likely to be true.

The threshold ought to take in to account the size of the publishing world.

viator said...

Lysenkoism of western science. The left is using science as a political weapon against capitalism. In the end it will be bad for the left, bad for science and bad for all of us.

They make something up, for instance global warming, which is an attack on hydrocarbon energy the life blood of modern society. It is a brilliant scheme which only now is beginning to fall apart as the fiercely hidden lies and bad science at the heart of the project begin to see daylight.

Triangle Man said...

Until someone goes through the historical scientific literature and applies the same standard of review one can't really say that errors are increasing, just that the reporting and retraction of errors is greater than it used to be.

Zach said...

The article is gated for me, so I can't tell if they discuss this, but there has been a major shift in the geographical composition of submitted papers in the last decade or so. Asia (definitely China, I'm not sure about India being included in this category) went from having very few submissions to having huge numbers. Naively, I would expect that this would be the largest factor driving any change in statistics of published papers.

The surge in submissions has put a lot of stress on the peer review process, which relies on busy people doing work for free.

CyndiF said...

I am not surprised that medical papers would be more heavily retracted. Those people publish 1 sigma results that would be laughed out of my field and they have no understanding of systematic uncertainties.

Anthony said...

The surge in submissions has put a lot of stress on the peer review process, which relies on busy people doing work for free.

This is a big part of it. And universities make a LOT of their money off of overhead costs on research grants and those who publish most get more grants, etc., etc., etc. Reviewers by and large don't have time to "check the math" on the globs of papers they get to review.

Also, I wonder if electronic submissions have anything to do with it. It seems far easier to do sloppy work when you just have to press a button to submit a paper (I exaggerate slightly) rather than prepare 5 pristine copies that will have be sent physically back to you if something's wrong.

Michael K said...

Lancet has become notorious for political papers, such as the ridiculous study of civilian deaths in Iraq. The people who wrote the paper interviewed Sunni families in Baghdad and extrapolated. The numbers exceeded the capacity of all the morgues but Lancet published it and hasn't retracted it. Then, of course, there is Wakefield and his autism paper that will kill more children for years to come. It took them 12 years to retract that. I'm sure it fit some left wing theme that an editor favored about immunizations.

lonetown said...

I'm thinking the ethics class is graded on a curve!

n.n said...

Ego-driven "mistakes". It is a mystery why anyone would expect scientists to be immune from the corruption which is a perpetual plague on humanity, even journalists.

The causes may differ, but the outcome is the same: progressive corruption; driven by dreams of instant gratification, funded through involuntary exploitation.

This is a predictable outcome of sabotaged character development. The consideration for ethical and moral behavior is a progressively distant priority, which, incidentally, is justification for the introduction of progressive totalitarian policies. This is the path to restricted liberty. If we cannot self-moderate our behavior, then someone else (e.g., government) will.