February 24, 2024

"Scientific papers are like someone’s dating profile on an app."

"They’re picking what pictures to show you and what stories to tell you. You don’t get to see the whole library of photos on their phone. Researchers are only presenting a sliver of what they’ve actually done. And just like a dating app on your phone, everything is inaccurate."

From "We’re Not Curing Cancer Here, Guys/Are leading scientists just making stuff up? Vinay Prasad breaks down the cancer research scandal" (The Free Press).


RideSpaceMountain said...

It's called peer review and 'replicatability'. A paper can have one or the other, but only 1% have both. In that way they are a bit like dating profiles and the hot/crazy matrix...women and men that exist at the top of logarithmic curve are effing unicorns.

RJ said...

The academic success metric is tilted toward quantity of published papers, not quality. There is a phrase: “the least publishable increment” which is the the true goal - you need to spread out your research to generate the longest list of publications possible. And that applies equally to valid or fake research.

Krumhorn said...

If you have ever taken the time to read a climate “science” paper…and I’ve read a great many of them….you would have a prime example of academic mendacity and rent-seeking, particularly if you know anything about multivariate statistical analysis. I just love it when they refuse access to their cherry-picked data. And they (collectively) get vast sums of grant money for that manufactured horseshit.

- Krumhorn

Temujin said...

From what I've read over the past few years, there is an entire industry of faux or lightly 'reviewed' research that has been published and from there...becomes 'expert opinion', or even 'fact'. It's probably not a new thing having charlatans or people just wanting a faster route to stardom, but it can come at a cost to society at large.

Let's consider the Michael Mann's climate hockey stick.

gilbar said...

Anyone that thinks that "scientists" are performing actual science is Not paying attention

Sebastian said...

"everything is inaccurate"

That's also inaccurate. But we should take Prasad not literally but seriously.

But the inaccuracy is in a way beside the prog point. The point is to Follow the Science.

gilbar said...

serious question:
Does Anyone STILL think that that covid "vaccines" were a good thing?

Even More serious question"
IF there ARE people who think that.. How would they feel about them if Trump was still President?

Covid "vaccines"
gender affirmation
psycho meds
cancer "cures"

What do ALL these snake oils have in common? That they make BIG MONEY for someone

JAORE said...

In my former (working) life I was the research coordinator for a federal agency. The amount of politicking even in my largely non-political field was astounding.

Money really does make the world go round.

FWIW there were no reports, no matter the actual outcome, where the summary did not include, "Further research is warranted".

JAORE said...

Lots of peer-pressure reviews out there too.

Original Mike said...

""They’re picking what pictures to show you and what stories to tell you."

Of course they are. It's called analysis, and is a crucial part of the dissemination of new knowledge. Of course, you're dependent on the integrity of the authors, and the scientific community as a whole. Unfortunately, as we've seen recently, that trust is often unwarranted in a distressing number of cases. I weep for what these low-lifes have done to science.

Oligonicella said...

Nothing new here. It's been that way for a long time. The red dye panics?

Wince said...

Here's a short video showing how it's done with "climate."

Judith Curry: How Climate “Science” Got Hijacked by Alarmists

Scientist [Judith Curry] published research that fueled climate change alarmism.

“I was...treated like a rock star."

Then she realized some of her research was wrong.

When she admitted it, the corrupt climate change industry attacked her.

Here she exposes how alarmism is REWARDED.

n.n said...

You can't change your sex, but you can simulate/emulate gender. You can't cure cancer, but you can mitigate its progress. Science is a near-domain philosophy - logical domain - of observation, replication, and deduction.

Big Mike said...

Keep in mind that Michael Mann’s papers were peer-reviewed, and not a single prediction he has made has come true.

tim in vermont said...

The thing about scientific papers is that there is no "conclusion police" who will call out faulty logic.

Like here are some snippets from a paper about experiments being done on coronaviruses back in 2016.

"Simon Wain-Hobson, a virologist at the Pasteur Institute in Paris, points out that the researchers have created a novel virus that “grows remarkably well” in human cells. “If the virus escaped, nobody could predict the trajectory,” he says.

"“The only impact of this work is the creation, in a lab, of a new, non-natural risk,” agrees Richard Ebright, a molecular biologist and biodefence expert at Rutgers University in Piscataway, New Jersey. Both Ebright and Wain-Hobson are long-standing critics of gain-of-function research."

It has this beauty: "The latest study was already under way before the US moratorium began, and the US National Institutes of Health (NIH) [Anthony Fauci]allowed it to proceed while it was under review by the agency, says Ralph Baric, an infectious-disease researcher at the University of North Carolina at Chapel Hill, a co-author of the study. The NIH [Anthony Fauci] eventually concluded that the work was not so risky as to fall under the moratorium, he says."

But then at the end of the paper is this absurd statement put there after the pandemic began:

Editors’ note, March 2020 We are aware that this story is being used as the basis for unverified theories that the novel coronavirus causing COVID-19 was engineered. There is no evidence that this is true; scientists believe that an animal is the most likely source of the coronavirus.

I think that this is a pretty strange version of "no evidence" They simply say absurd things and expect us to believe them if they are presented authoritatively enough. Their "plausibly deniability" of an animal source that they think gets them off the hook is so improbable, the idea that of all places on the globe, it happened right in the same city where the experiments were done, that the "plausible" part fails completely.

William50 said...

"Are leading scientists just making things up?"


John henry said...

The poster boy for peer review has to be joh schon https://en.m.wikipedia.org/wiki/Sch%C3%B6n_scandal

At one point he was publishing several highly technical physics papers every week in the tip-toppiest scientific journals. All peer reviewed. All made up bullshit. All retracted.

Also retracted were a number of prestigious awards as well his PhD.

Peer review is often bullshit. To paraphrase something the head of p&g supposedly said about advertising in the last century "50% of peer review is worthless but there is no way to tell which 50%".

John Henry

tim in vermont said...

By the way, the lack of comprehensive safeguards made it cheaper to do the experiments in Wuhan.

John henry said...

I've published 2-300 articles,, columns, white papers, videos and other stuff over the past 30 years (depending on how you define "publish")

Including 3 peer reviewed articles in respected scholarly journals.

Not much difference between peer reviewed and the rest. Main differences is the peer review does not pay. One does it for the prestige. Also much more limited readership.

John Henry

tim in vermont said...

"Keep in mind that Michael Mann’s papers were peer-reviewed,"

Not just that one, but the new Hockey Stick, which is also full of holes, was peer-reviewed as well.


typingtalker said...

"Are leading scientists just making stuff up?"

Next week, Global Warming.

JK Brown said...

Is it "bad science" or bad scientists. The latter, being mostly academics and government lanyard-wearers, are on the whole petty and mercenary. They need grant money more than then need integrity in their eyes.

Mikey NTH said...

Ants will find sugar.
Reasearchers will find grant money.

Big Mike said...

Editors’ note, March 2020 We are aware that this story is being used as the basis for unverified theories that the novel coronavirus causing COVID-19 was engineered. There is no evidence that this is true

If you knew what to look for, the evidence was obvious and irrefutable.

Lem the artificially intelligent said...

Listen to the science… on instagram.

Aggie said...

The discussion dances all around the problem without addressing it directly, thus perpetuating it.

This is a Quality issue. Quality is something that is handled by management systems that have been agreed and ratified, and enforcement is ruthless. These are typically held by independent classification societies - Die Norske Veritas and so on. And corrupting the Quality Management procedures is grounds for immediate firing and professional disgrace. This is the philosophy that keeps people alive on the operating table, and safe in flying things with many moving parts, helicopters and jets. This is how merchant marine ships, drilling rigs, fire-fighting equipment, and countless other safety-critical systems are kept in top working order so that people can depend on them in emergencies. You agree to abide by the quality management system, it's in virtually every single chartering contract, and it's a well-understood concept.

Now: Science. If science were governed by similar principles, not a single paper would be published until the review by peers has been completed and signed off. The peers would be certified, and their certification would have to be periodically re-verified. And the loss of certification would mean, directly, an immediate loss of income and prestige. If the paper was on research, then the research would have to be replicated under peer witness.

You see how this works? Right now, research science is operating in Wild West mode. It's not that hard to bring very good beneficial changes, but there has to be some determination and will to see it happen. It would not be the creation of a whole new concept. It would be quite straight forward, and a lot of bullsh*t would come to an end.

Rabel said...

Biden already cured cancer. Twice.


Mea Sententia said...

Media, politics, education, business, government, religion, and science. All the institutions are corrupt, to one degree or another. Science was just the last domino to fall, the last institution we trusted.

walter said...

Further research is warranted....

Anthony said...

Editors’ note, March 2020 We are aware that this story is being used as the basis for unverified theories that the novel coronavirus causing COVID-19 was engineered. There is no evidence that this is true

Reminds me of the mask literature. Prior to 2020 some group did a thorough review of the literature for masking, found it to be of very poor quality, and concluded that there was no evidence that masking worked to prevent the spread of virii.

But then sometime in 2020 they added a blurb to it saying that they still "stand by their conclusions" and don't want it removed (!!!!!) from whatever web site it was on, and oh, by the way, they still support mask mandates.

But yeah, academic success is built around numbers of pubs, citations by others ("impact"), and also the quality of the journal. There's a built-in bias to get things published so other people can cite them, and be cited in return.

Plus, there's a new formula, which has a name but can't recall it right now: You publish a largely polemical paper, it gets cited multiple times until it becomes "established" with a whole research program developed around it, and then a few years later someone actually looks at the original paper critically and it's all based on hooey.

ccscientist said...

It is even worse that that. Journals are not interested in articles that find even major flaws in published works and will simply refuse to review them. A common problem is p-hacking, where a very small positive effect--possibly spurious--that gets a significant p value is considered valid. Other studies of the same thing that found no effect don't get published because no journal will publish "no effect". For example, at one time bran muffins were supposed to help your heart.....then you never heard about it again.

Tom Grey said...

Science is powerful, and believed by non-scientists because it replicates.
Rather than peer review, we need far more result replication experiments, with the results published.

It’s good that Althouse is reading and linking to more than the often dishonest NYT. TFP and substack are becoming more reliable, with replicatable analysis & disagreement, than is the legacy Dem media.