Quoted in "The Harvard Professor and the Bloggers/When Francesca Gino, a rising academic star, was accused of falsifying data — about how to stop dishonesty — it didn’t just torch her career. It inflamed a crisis in behavioral science" (NYT).
How screwy was this field?
The bloggers are the Data Colada bloggers, who have been checking the evidence and finding problems.
Why was Kahneman so credulous in the first place? He seems to be admitting that he wanted to believe cool stuff, so he just did.
How screwy was this field?
It’s often difficult to identify the moment when an intellectual movement jumps the shark and becomes an intellectual fad — or, worse, self-parody.
But in behavioral science, many scholars point to an article published in a mainstream psychology journal in 2011 claiming evidence of precognition — that is, the ability to sense the future. In one experiment, the paper’s author, an emeritus professor at Cornell, found that more than half the time participants correctly guessed where an erotic picture would show up on a computer screen before it appeared. He referred to the approach as “time-reversing” certain psychological effects.
Posts
61 comments:
Behavioral economics is psychology dolled up with numbers.
Let that sink in.
Behavioral "science."
"He seems to be admitting that he wanted to believe cool stuff, so he just did. "
Isn't that how most of society works?
Kahneman saidd...
"When I see a surprising finding, my default is not to believe it. Twelve years ago, my default was to believe anything that was surprising."
Althouse said...
"Why was Kahneman so credulous in the first place? He seems to be admitting that he wanted to believe cool stuff, so he just did."
Book Synopsis, Thinking, Fast and Slow...
In the highly anticipated Thinking, Fast and Slow, Kahneman takes us on a groundbreaking tour of the mind and explains the two systems that drive the way we think. System 1 is fast, intuitive, and emotional; System 2 is slower, more deliberative, and more logical. Kahneman exposes the extraordinary capabilities—and also the faults and biases—of fast thinking, and reveals the pervasive influence of intuitive impressions on our thoughts and behavior.
https://www.goodreads.com/en/book/show/11468377
Kahneman was "fast," now he's "slow"?
Think fast, think slow, but please think. Be better Kahneman.
I think the "jumping the shark" study is this one by Daryl Bem at Cornell
https://news.cornell.edu/stories/2010/12/study-looks-brains-ability-see-future
Is there evidence that he falsified data, or was the study poorly designed so that the results are unreliable? Granted that pre-cognition seems unlikely, but is research into things that are unlikely now considered "jumping the shark"?
The Data Colada researchers are apparently being sued for big bucks for pointing out the falsity of the findings
John Henry
Critical Race Theory, Diversity (Equity, Inclusion), human rites, political congruence, etc.
Diversity of individuals, minority of one.
Can you capture it in a math statement?
It’s just women wailing.
Like The Crack Emcee, I find this all unsurprising given the people I have known. Children are like this by their very nature- willing to believe wonderful things like Santa Claus, the Tooth Fairy, and the Easter Bunny while unwilling to believe the more mundane aspects of life like, for example, writing a check for something doesn't mean it you can have it without having to pay for it (an idea of which my mother had to work to disabuse me when I was 6 or 7 years old). I don't know what the adult world was like when I was a child or before my birth, but I know what it is like today- adults in the US are increasingly child-like in their behaviors and beliefs. Skepticism seems to be hard work for far too many people today.
Dennis Prager says "there are two kinds of psychological studies, those that confirm common sense and those that are wrong."
I would add that there are two kinds of psychological studies, those that confirm common sense and those that get you tenure and funding.
Goodness. I enjoyed his book, but what kind of researcher gets an unexpected result and says, "Cool! My work here is done!"?
"He seems to be admitting that he wanted to believe cool stuff, so he just did. "
Isn't that how most of society works?
It does now, it seems. The valorization of the New. (Yes, including the New Age!)
I was listening to a Sam Harris interview the other day and was struck, not for the first time, by his incredibly arrogant attitude about (what else?) religion. He seemed to take it as read that "of course" we - by which he meant he - could create, out of whole cloth and with recourse only to his giant brain, a moral system that could not only supplant religion but improve on it. And then he based the whole rickety edifice on his perception of "minimizing suffering" and "promoting human thriving"... with himself, or in a more charitable reading every individual human being, as the highest authority in determining what actions would bring about these ends.
He's such a tool.
I can get along with an atheist. But arrogant atheists get under my skin. (To be fair, arrogant theists do too, but the difference is that within the theistic worldview everyone but the arrogant person knows that the arrogant person isn't the highest authority.)
In one experiment, the paper’s author, an emeritus professor at Cornell, found that more than half the time participants correctly guessed where an erotic picture would show up on a computer screen before it appeared.
A good guess without looking up the study details — the algorithm that selected where to place the picture on the computer screen was not as random as the study authors thought it was. So there was an underlying nonrandom pattern and some people got good at spotting it. Rewarding horny college-age males with erotic pictures would be all the stimulus-reward mechanism you’d need.
As a side remark, humans will at times look for a pattern where there really is no pattern. It can cause societal problems.
Behavioral Economics is very much the investor's parallel to the Diversity (DEI) crowd's Unconscious Bias training (e.g., see the Harvard IAT)...wishful thinking...
Cognitive/psychological processes are real, but only testable in narrowly controlled lab conditions and subject to multiple real-world interpretations. As such, most applications of behavioral investing and implicit/unconscious biases tend to fail because the theorists routinely forget to consider 150 of the 200 relevant factors.
Sexy. Exciting. Not testable. Not actionable.
Putting aside the debate on behavioral science: what really caught my attention is that an Ethics professor is under suspicion of fraud.
At the same time we don't want to fall back into a microeconomic rational actor model that has often led to significant losses. Behavioral economics should be applied more thoroughly — with adequate oversight to test its application.
The problem is that behavioral studies are hard. So instead of admitting that, and dealing with uncertainty, many behavioral scientists retreat into doing "easy" — but largely useless — minor experiments or, even worse, hypothesizing and then presenting the hypotheses as true.
Years ago, I had a discussion on this topic with the husband of a colleague, who taught Business at Stanford. We agreed that many of the interesting questions being asked about behavioral science would be extremely difficult and expensive to determine via studies.
He said "That's why we have to try smaller studies or modeling , and interpret from those". I laughed and said "No, it means you just have to say "We don't know and can't test that".
This conversation has stuck with me because I think it highlights a key difference between the physical sciences (including biology) and the social sciences. In our fields, it's perfectly acceptable — even routine — to say "Eh. That can't realistically be tested yet, so we don't know". In social sciences, though, that encompasses so much of the field, that I think they are afraid to say it.
Somehow my tax dollars are subsidizing this nonsense.
"explains the two systems that drive the way we think. System 1 is fast, intuitive, and emotional; System 2 is slower, more deliberative, and more logical"
I read a chunk of the book and found this claim a bit puzzling, since in several instances it seemed to be little different from saying that our minds do some things fairly fast and others fairly slow.
Are there ANY "findings" in social science that were surprising when first published AND that have held up over time in light of further testing?
“ It’s often difficult to identify the moment when an intellectual movement jumps the shark and becomes an intellectual fad — or, worse, self-parody.”
I could see see from the outset of the CAGW scam and the transgenderism scam.
Science follows the money.
Data Colada are true heroes.
That’s why they must be sued out of existence.
Notice how studiously the academics ignore the obvious data manipulation in the 2020 election in the United States.
Our elections are FAKE.
System 1 is fast, intuitive, and emotional; System 2 is slower, more deliberative, and more logical.
This is the best summary of the difference in thinking between Liberal and Conservative minds that I have ever heard.
Most of what we call the soft sciences (which includes basically anything touching on human behaviour) isn’t rigorously scientific, but tries to make itself look all sciencey by expressing its ideas as mathematical formulas. But the math is bunk and their certainties are at best probabilities.
Economics, once you get past a very small number of bedrock principles (like supply and demand), is a soft science. It’s as much voodoo as anything.
Maybe the key is to be 'the wrong kind of' behavioral scientist, which would criticize the default assumption. The default assumption being that the greater the surprise, the bigger the impact/relevance.
Darkhorse podcast delved into this a number of times with regards to health authorities covid response.
This is why I have been reading Althouse for almost 20 years: I like seeing bullshit exposed.
There is so much bullshit that everyone seems to believe despite its obviousness, prima facie stench. It's like everyone first checks to see if other people believe it instead of using their own nose.
He seems to be admitting that he wanted to believe cool stuff, so he just did.
How screwy was this field?
There is NO PLACE to question SCIENCE.. Science MUST BE Believed... TRUST THE SCIENCE
The fundamental tenet of SCIENCE, is mindless blind faith.. The alternative is HELL!!!
DEATH TO UNBELIEVERS TRUST THE SCIENCE!!!
Sebastian said...
Are there ANY "findings" in social science that were surprising when first published AND that have held up over time in light of further testing?
oh, GOSH YES!! Let me just name, a few
(all other things being equal, there is a general tendency for..)
* growing up with 2 active parents somehow leads to a better life (how ever you define better)
* people with higher IQs tend to do better in schools
* leaders tend to be people with higher IQs
* a person's criminal history tends to predict their likelihood of committing future crimes
* people that drink and do drugs to excess tend not to advance as far in society
* people whose mothers were drunk and drugged during pregnancy tend to not advance as far in society
The are LOTS more surprising shockers, but the thing that they ALL share is: NO ONE KNOWS WHY
"It’s often difficult to identify the moment when an intellectual movement jumps the shark and becomes an intellectual fad."
It is not the intellectual movement that changes. It is your perception of what was always an intellectual fad. If an author says that his results are "statistically significant", that means he needed a bunch of pseudo-mathematical mumbo-jumbo to make his results sound even remotely plausible. And yes, that includes the "medical sciences", which are right in the thick of the "reproducibility crisis".
Tim Maguire said
Economics, once you get past a very small number of bedrock principles (like supply and demand), is a soft science.
Demand is not fully predictable. Think tulips. ESG funds and bugs as protein.
behavioral economics (noun) the neo-Marxist hypothesis that economic choices are fundamentally non-rational, as opposed to the rational self-interest model upheld by classical macroeconomics since Smith's The Wealth of Nations, derived from the infamous Skinnerian weight-loss protocol briefly summarized by the maxim, if you hit someone hard enough every time he reaches for the doughnuts, he will lose weight.
Rich said...
Putting aside the debate on behavioral science: what really caught my attention is that an Ethics professor is under suspicion of fraud.
Also amazing is that Sam Bankman-Fried’s mother, Barbara Fried, taught advanced seminars at Stanford Law School seminars in moral/political theory. She and Sam’s father, also a Stanford Law professor are now being sued by FTX— suit says that SBF’s parents “exploited their access and influence within the FTX enterprise to enrich themselves” by millions of dollars and knowingly bilked FTX and its customers as they did so.
Moral theory and moral behavior apparently exist in separate spheres.
Gilbar: "surprising shockers"
Hey, hey, no fair.
What's Daniel doing here?
The answer doesn't jump out at me by clicking the tag.
Daniel was a wise guy who was cancelled but came back after a tour in...
There's hope for Trump?
I just need a glimmer.
#TrumpChallenge.
The basis of Behavioral Economics, not much different than this: Ten Principles of Economics
@JeanE - I remember the controversy over Bem's paper mainly because it brought to my mind an unsettling experience I had in the mid-80s. Basically, I had a 'vision' of a motorcycle wreck. Of course, I was aware that observing the rider's erratic behavior and speed for a few miles beforehand led to this imagining. I'm sure I'm not the only person who has accurately predicted "They Are Going To Crash!" It was unsettling because the actual wreck was as I had imagined it - that position on that curve as well as the position of the bike and rider.
I was spooked and briefly considered that I'd had a "premonition" but discarded that idea because I was so familiar with the road, had observed the behavior and speed of the motorcyclist, and concluded that my brain came up with a highly probable image of an accident.
"How screwy was this field?"
Not as screwy* as this one:
"At present, Leif, Joe, and Uri do not have pro bono representation. The lawyers they’ve spoken to currently estimate that their defense could cost anywhere between $50,000 and $600,000 (depending on how far the lawsuit progresses)."
*For some definitions of "screw."
It’s taken me fifty years to realize that many people have terrible judgment about what might be plausibly true or at all likely. This explains many of the mistakes that journalists seem to make; they believe enough in the first instance that they don’t ask the right counter-questions.
If one has bad judgment in this area, one is just bumbling along believing the standard story and whatever deviates from that is intellectually shiny and new. Academia is not helpful here because there are so few penalties for being wrong about things that are mostly theories. Being “interesting” is a good way to get ahead in academia.
Not even a nibble on my dissing of Sam Harris, hmm? Fine...
In the new Clown World, evidence-based policy making has been replaced by policy-based evidence making. Much of modern science is now indistinguishable from Lysenkoism.
Trust no government scientist or government-funded scientist, they are on the side of the slave masters of the enemy occupation government.
Maybe the key is to be 'the wrong kind of' behavioral scientist, which would criticize the default assumption. The default assumption being that the greater the surprise, the bigger the impact/relevance.
The default assumption is that results which tend to justify higher taxes, aggrandizement of government, loss of individual liberty, denigration of disfavored subgroups (whites/men/Christians/straights), and glorification of favored groups are those which are presumed correct and result in tenure and funding.
It's simple. Too many researchers, too few true things to find.
"He seems to be admitting that he wanted to believe cool stuff, so he just did."
It's sociology. That's the core trait.
JeanE said...
"falsified data, ... poorly designed so that the results are unreliable?"
Again, a basic trait of sociology.
Two-eyed Jack said...
"those that get you tenure and funding."
Yes. Hence my first two posts.
Yancey,
Conversation I had once with my 5-year-old daughter. We are sitting in church, the offering plate passes by, and I drop a check in it.
She [whispering]: You can give a check?
Me: Yep.
She: A writed-on check??
Me: Uh-huh.
She: Then how can the poor people use it???!?!?
"The Crack Emcee said...
"He seems to be admitting that he wanted to believe cool stuff, so he just did. "
Isn't that how most of society works?"
Michael Polanyi said, "Yes."
(in Personal Knowledge, Towards a Post-Critical Philosophy)
"System 1 is fast, intuitive, and emotional; System 2 is slower, more deliberative, and more logical."
Myers Briggs.
..."How screwy was this field?
Was? Was>>? That's an innerestin' choice of tense, there. I'll tell you, the really disturbing part of this story is, how easy it was to perpetrate the self-delusion and expand upon it to commit research fraud that was uncritically incorporated into policy.
Now let's imagine the same thing happening in more traditional science. Climatology, for instance, or maybe communicable diseases. Yeah, baby.
People (or at least a lot of them) seem to think that doing science -- or, rather, doing good science -- is easy. It's not. It's hard. Exceedingly hard. Bias can (and does) creep in at every single level of the research endeavor, from problem formulation to funding to publication bias. And the farther you get away from physics the more difficult it gets.
The educational system is an abject failure in this regard. Teachers and media promoters of Science(tm) have sold it as an Easy! and Exciting! way to Know Things (see the utterly idiotic "I F***ing Love Science" crowd). "Wow! Look how excited I am about Science!"
Was just perusing one of those silly list links that pops up on teh Facebooks and such ("Here's 25 weird things celebrities have done!") (which I admit I'm kind of addicted to) and the first item in one was how some person in the 1950s showed that "Climate Change Is real". Hellooooo numb-nuts, we've known that the climate changes for a couple of hundred years now, a basic fact that one would learn in middle school if the history of science were actually being taught.
The Cornell precognition study was absurd. Scientists sometimes say a result is statistically significant if there's less than a five-percent chance that the result was one you'd get just by chance. So if the chance that you'd get a particular result just by chance is, say, 4.5%, that's "statistically significant" but it doesn't mean it's not just chance: things that have a small chance of happening do happen. If you shuffle a deck of cards and deal them out, the odds against getting whatever result you got are astronomical. That doesn't mean that it wasn't just chance that led to that result. There have been countless attempts to discover precognition, almost all of them have failed, and even the Cornell study found that all but one of its tests failed to get even a statistically significant positive result.
Kahneman, unlike the Cornell guy, is a serious scientist who won a Nobel Prize in economics for his work on behavioral economics. "Thinking Fast and Slow" is a superb book. I don't think he's right about everything, but it's one of my favorite books.
I just hope that people never begin to think that the "science" of behavior genetics is cool. This is a "science" that began inside the American Eugenics Society. Members founded the Behavior Genetics Association (BGA) and members led it. They made a substantial contribution to the "research" used in The Bell Curve and many signed Mainstream Science on Intelligence which asserted that some human groups have a genetically-based IQ deficit. At the BGA 1995 meeting the BGA President Glayde Whitney gave a speech asserting a link between crime and the genetics of certain human groups. More recently behavioral geneticists have faded into the background but they have not faded away. Their leading initiative is to introduce the idea of polygenic risk scores as a policy tool. Behavioral geneticists allege that they are are or soon will be able to able to determine from a polygenic composite a score which predicts at birth to what extent an individual will become a social risk and in what way. Will they smoke, will they be obese, will they be criminals, will they be Republicans and so on.
My prediction is that the score will be shown to be another scam like the IQ scam and for this reason. Considering the variety in human behavior in any one place and considering our presence in every part of the world, we don't have very many genes by which to control all the different behaviors we know about. As a result, the genes network to achieve their results (so far as behavior has a genetic basis.) But the polygenic risks scores don't take networking into account. In other words, if I have genes associated by the polygenic risk scoring system with being a Democrat I might also have an upstream gene that negates the cluster. (And I might have a soul, anyway, but that's another argument.)
So, materialists, be cool to the idea of behavior prediction when it is thrust forward as a solution for the chaos in the Dem cities. Bet on the horses, not the polygenic risk score.
It could just as easily be that the Slow Thinking System comes up with reasons why we should assume the Fast Thinking System's answer was correct.
We are pattern-matching rationalizing animals.
Democratic gerrymandering. Part of the multitrillion dollar social industrial complex (SIC) to influence, manipulate, coordinate a consensus.
The article refers repeatedly to "the bloggers," without ever revealing that they are all three professors in their own right, all at prestigious institutions: University of Spain/Barcelona; UC/Berkeley; and the University of Pennsylvania. "The bloggers" sounds, as it is meant to, like . . . well, like a bunch of guys in their pyjamas in Mom's basement.
OK, none of the the three comes from HARVARD!!! But that oughtn't to matter in scientific work, yes?
As a cognitive-behavioral psychologist, I found the research of Danny Kahneman and Amos Tversky to be way beyond reproach. The work was clearly stated, non-political, and easily replicable.
Unfortunately, there is a ton of psychological research that is taken for fact that has never been replicated.
Much of it was conventional "wisdom" that supported philosophical and political positions.
wildswan, please elaborate! If I understand correctly, you're not dismissing the possibility of differences in the "average genome" (for lack of a better term?) of populations separated for millennia (which seems, evolution-wise, to make sense, as they would have had different pressures on them). You're just saying that - sort of like the heatsink capacity of the oceans? - the genes can act in concert or in negation of one another, to minimize these differences. Is that more or less it?
What's the "IQ scam"? Is it just that it measures something valuable to only one sub-population of H. sap.? Or something else?
This is a very interesting topic. I have been thinking about it for awhile but I recognize my own ignorance.
JeanE said...
Is there evidence that he falsified data, or was the study poorly designed so that the results are unreliable? Granted that pre-cognition seems unlikely, but is research into things that are unlikely now considered "jumping the shark"?
By the "standards" of their field, the study was NOT "poorly designed".
The point is that his "study" produces such ludicrous results that it forced people to start thinking about the reality that "this field's standards are utter crap."
And that's why it was the "jumping the shark" moment
Darkisland said...
The Data Colada researchers are apparently being sued for big bucks for pointing out the falsity of the findings
Yes, they are. See here:
http://datacolada.org/113
and here
http://datacolada.org/114
But the reality of the situation is that Francesca Gino, the utterly fraudulent "dishonesty researcher", is a moron for bringing the lawsuit.
The second link is to an article that links to her lawsuit filings. Which includes the letters Harvard Business School sent to Journals retracting the articles.
And what those letters clearly state is that they (HBS) pulled up the data that was collected for the papers, and conclusively proved that the data IN the papers did not match the data collected.
Which means the Data Colada people are going to get to do the following in Discovery for this lawsuit (for "Request" below read: "demand, and have the court back them up"):
1: Request all that initial data from HBS
2: Request an interview with Francesca Gino, where they ask her about how the data changed. And then pound her into little pieces, because the only way the data could have changed is that she deliberately changed it while engaging in fraud
Because Francesca Gino was the sole source of the data for the published papers. Everything went to her, she personally took charge of it, and then personally distributed it to her co-workers.
HBS tried to engage in CYA, and so refused to say "this clearly proves that our tenured professor was a total fraud". Which is her grounds for suing HBS.
The Data Colada people, OTOH, couldn't care less about HBS's rep.
Either her lawyers are idiots, or she just refused to listen to them. In any event, the Data Colada people are going to slow roast her over an open fire
Jamie!
I totally nodded and smirked in agreement bringing in Sam Harris into this.
thumbs up.
JeanE said...
Is there evidence that he falsified data, or was the study poorly designed so that the results are unreliable? Granted that pre-cognition seems unlikely, but is research into things that are unlikely now considered "jumping the shark"?
This article discusses that near the top:
https://www.experimental-history.com/p/psychology-might-be-a-big-stinkin
The joke is that anything in this field is considered "science".
Post a Comment