Writes David Mamet, in "Everywhere an Oink Oink: An Embittered, Dyspeptic, and Accurate Report of Forty Years in Hollywood" (commission-earned link).
Wyner is a professor at the Wharton School who specializes in statistics, data science, and probability models. He was called by the defense in this defamation case, which aims to cast doubt on the famous "hockey stick" graph.
I want to quote some of Wyner's testimony that deals with the problem of human observation and the formation of opinion about causation:
"[A] complicated statistical task... is like taking a walk... Every time you get to a fork, you have to make a decision. You go right, you go left. Then you make the decision and you head down and come to another fork. You go right or you go left. And the idea is that these small decisions, they seem irrelevant. But they can make all the difference. And that in particular, if you kind of see where you want to go with the answer, you can lead yourself into a conclusion that would be very different if someone made a different set of decisions and walked down a different set of paths....
"And it requires a certain skill and an awareness and attention to affect all the details to get a solution [that] is reliable. [Otherwise, you get] what you might call manipulation, the idea I will manipulate the set of paths I'm going through to get the result at the end of the day that is ultimately not reliable. In some cases, wrong or misleading."
Posts
31 comments:
Only Ann Althouse could link David Mamet on drama and an Ivy League professor of data science’s testimony in a defamation trial.
Well done!
Live freely in writing, indeed!
Way back in 1975 I had an engineering prof comment on the infancy of computer models for design. What he said was that he did not fear garbage in garbage out. He feared garbage in GOSPEL out.
His point was that engineers have to know what the models do to data in order to understand whether the outputs are rational.
Today's models (especially ones like climate change) are so complex there is no way to really understand the process in a rational way (all those left-right decisions).
A very small example. Many years go I was in a group of engineers sent to Michigan State to study pavement models available. There were myriad ones even in early days. The professor encouraged us to explore the boundaries of the models to see where they failed. noted a concrete pavement program that (of course) included a variable of the stiffness of the layers below the concrete. Less firm, more thickness to the concrete. On a whim reduced the lower layer to zero. Kind of like paving over water. Gee, all had to do was make the concrete another inch thick. Sadly my plan, and my fortune, to use a paving machine to replace bridge building failed the realty test.
The Wyner excerpts reenacted on the blog show Wyner as an effective explainer of the subjectivity of Principal Components Analysis (PCA)-- the ways in which a predetermined outcome can be engineered with a patina of scientific certainty.
The comparison of PCA to the subjectively manipulated outcomes of psychiatry and dramaturgy is apt.
I am glad to see you following the Mann V Simberg case. At least as consequential as Sarah Palin's suit against NY Time over the "targeting" column. Stipulating the science, a big issue is whether or not free people -- citizens and otherwise -- can criticize the majority view. Can one publish an essay saying the climate is not warming? That COVID-19 is not uniquely dangerous? That no combination of pharmacist and surgeons can transition a child from one sex to another? That illegal immigrants are a net cost to federal budgets regardless of their social security "contributions"? That Asians on average are smarter than Europeans? That UFOs are being covered up by the US government? Are there limits? Do people have the right to be wrong?
Is he riffing on our inability to discern origin and expression?
Climate, weather, and forcings?
Grooming and dysfunction?
Fetus and baby?
Science practiced outside of a limited frame of reference?
There is order in chaos.
Years ago, I wrote a computer model to produce an x-ray spectrum emitted by an x-ray tube and compared it to spectra I was measuring. My interest was in determining the configuration of the target from the measured spectrum. My initial model results didn't quite match my measurements, but it didn't take much tweaking to the target configuration I fed the model to get a perfect match. Now, did that mean I had found the actual target configuration or had I just manipulated the model to give me what I wanted? I didn't know, and it was clear that answering that question was a whole lot more work than was worthwhile, so I dropped it. But if my grant funding had been dependent on that result, I had a nice graph with a believable story, so…
Point being, model outputs have to be viewed with great skepticism. What the climate crowd have done in making their results sacrosanct (denier this, denier that) is not conducive to good science.
I bought this latest book by Mamet through the Althouse AMZN portal. I liked his two prior books much better.
Mamet is the closest we have to Tom Wolfe so I’ll read anything he writes.
The dramatist also chooses forks in the path. Sometimes the character becomes so narrow by those choices that the writer can't find a satisfying end to the story. It's just fiction, though. Rewrite and rebuild. It's not someone's mental health or world climate policy.
This is what the statistician is talking about:
http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf
And that in particular, if you kind of see where you want to go with the answer
Too generous. The incorrect assumption is Mann was interested in scientific method and study but he’s sociopath that began with a conclusion. Such is with the sociopathic liberals attacking their political opponents. The forks scatter about but all roads lead to the same conclusion…
They aren’t ignorant of the process, they are sinister shits what feel justified in their actions…
The hockey stick stuff used to be taught as error in eighth grade science. The belief, a correct one, was that between controlling the narrative and intimidation of those not supportive of the policy solutions nobody would bother with an examination of the junk science…
Only Ann Althouse could link David Mamet on drama and an Ivy League professor of data science’s testimony in a defamation trial.
Of more interest to me is that she linked data analysis (a mostly objective science) to psychoanalysis (a most subjective pseudoscience).
And that in particular, if you kind of see where you want to go with the answer, you can lead yourself into a conclusion that would be very different if someone made a different set of decisions and walked down a different set of paths...
Indeed, the problem with building models is you sometimes get high from the glue that's holding it together.
And it requires a certain skill and an awareness and attention to affect all the details to get a solution [that] is reliable. [Otherwise, you get] what you might call manipulation.
"The exactness, the attention to every conceivable detail."
JAORE:
His point was that engineers have to know what the models do to data in order to understand whether the outputs are rational.
Hence the refusals to share data and code for models. Or (paraphrasing Mann himself) "Why would I give them my data? All they want to do is disprove my results."
When the zip file containing masses of leaked/stolen data, emails and code was put online, I downloaded it.
Did you know that in the data section (in this case literals they used) there was a literal whose comment read "fudge factor"? Twern't no baking recipe. Not one for a home kitchen anyway.
Steyn is Canadian.
He knows hockey sticks.
Judgement to Steyn.
‘Question: "Dr. Wyner, do you have an opinion as to whether the techniques used in Dr. Mann's Hockey Stick research are manipulative?"
Answer: "Yes."
Question: "What is your opinion?"
Answer: "It's my opinion that the techniques used by Dr. Mann in his earliest work (98/99), and to some degrees in his later works, are manipulative."’
Ah, “what you might call manipulation” … “to some degrees”.
Can anyone tell me which episode of the podcasts contains Stephen McIntyre's testimony?
"The grand perception of psychoanalysis, for the dramatist, is that all actions are performed FOR A REASON..."
Anyone who takes psychoanalysis seriously should not be taken seriously.
Dave Begley @9:51: what you said. Judging by the quoted testimony, this Wynan fellow has a gift for explaining the deep stuff; and I do think the “buzzing blooming confusion” we inhabit is a matrix laden with possible causal threads, and we tend to seize in the biggest, brightest ones sticking out quite handily. To approach it with (ahem) cruel neutrality and with a mind truly empty of prejudice and preconception (and ambition), would require enormous discipline and self-knowledge. As so often, I go back to paraphrasing Feynman’s dictum: “the easiest person to fool is yourself.”
The first principle is that you must not fool yourself and you are the easiest person to fool. - Richard Feynman
What Begley said at 9:51. Only Althouse could put these things together.
I suggest everyone join me in buying the book through the Althouse link. Its a good read.
A big part of my job is modeling nonlinear dynamical systems. The gold standard for a math model of a nonlinear dynamical system is like this: you do your math modeling; you run the model; you compare it to your experimental data; you tweak the free parameters in the model (and they have to be truly free parameters; no fair changing the value of pi) until the model matches the data to the needed accuracy. You then run the model against non-trivial observations (no good verifying the model can reproduce, say, that multiplying by zero gets you zero) that were never used in tuning the model. If the model can replicate substantial non-trivial observations that were NOT used in tuning the model, then it can provisionally be called adequate, but even then only adequate in the realm in which the model was developed and where the match with real data can be demonstrated.
I teach this kind of stuff to other engineers, and after deriving a set of equations, I always go back and point out every assumption I made - things like a flat earth (lots of complicated engineering problems don’t really care whether the earth is round or flat), constant mass (if the model involves burning fuel for power, then if the dynamics I’m interested in are much faster than the dynamics induced by a very slow loss of mass, I can ignore the fuel burn), etc. Knowing the assumptions, I can define the region of validity of the model, and know what I can and can’t use it for.
Have the climate people done this? My impression is not. I find it plausible that human activity, in particular burning of fossil fuels, has affected the Earth’s climate, but I have a hard time thinking we should upend our civilization based on models that have no significant predictive power. And it seems to me that most of the loudest “climate alarmists” are at heart the same people who have been railing against technological civilization for decades, and really want everyone to go back to an 18th century agrarian life, with a commensurate total population, with them in charge, of course. If they were truly serious, they’d all just kill themselves immediately and cease being a burden to Gaia, but obviously they’re not serious at all, except about their conviction they are uniquely chosen to rule over others.
” You go right or you go left. And the idea is that these small decisions, they seem irrelevant. But they can make all the difference.”
So Robert Frost was right.
I used to think the actions of people around me were always done for a reason. I think the same now, but my definition of "reason" has expanded to include "because they are batshit crazy."
Brava, Ann, for seeing the linkage between Mamet’s discussion of character manipulation and the professor’s insight into statistical manipulation.
PFC Wintergreen @ 2:53: Nice explanation of how modeling should be done. In contrast, the climate oracles produce models that “can’t even hindcast.” They use parameters for some of the most powerful drivers of climate —like clouds— because they simply cannot model clouds,,especially at the necessary scale. They create the trends fhey want for temp over time by amending without explanation the old numbers (making the past seem colder). They treat temp readings off airport tarmacs -established to guide pilots needing conservative (high) readings to choose their takeoff and landing points— as if the readings were representative of “the” temperature for thousands of hectares of forest, cropland, swamp, hilltop. I could go on.
Climate models model climate the way GTA4 models New York City, only less so.
Shorter article: Don’t get high on your own supply.
Ex-PFC Wintergreen:
"Have the climate people done this? My impression is not."
Indeed not. The inability of the current models to hindcast our current conditions is a huge scandal that the alarmists refuse to talk about.
I can guarantee you that if the models could, we would be inundated with examples of them doing so, as "proof" of the truth of AGCC
Ex-PFC:
Your final paragraph pretty much sums up my take on the matter too, except I think they are aiming at something a few centuries earlier.
15th century lifestyle with a 15th century energy usage necessitating a 15th century sized population, which nets out to about a 95% die off. That is indeed what they are aiming for.
Post a Comment