June... seemed genuinely interested in learning skills to reduce her anxiety and reported practicing the skills between sessions. When I asked June for feedback at the end of each session, she told me the therapy was helpful. “The skills you’re teaching me are good,” she replied in her soft, careful voice.
Before each session, June took a few minutes to complete the fit survey on an iPad in the waiting room, responding to statements like “I feel fearful” and “I enjoy my spare time” with preset answers ranging from “never” to “almost always.” Though I had access to her clinical graph every session, I didn’t bother checking it at first, because she seemed to be progressing so well.
After a few sessions I finally checked the graph—more because I felt like I should than because I thought it would be helpful. I was shocked to see that June’s chart showed a red alert. Her symptoms had not improved since our first session. The algorithms reported that she was actually at a high risk of deterioration and suicide.
My gut reaction to the alert was skepticism—as it almost always is, to this day, when the program’s algorithms contradict my instincts.... At the beginning of our next session, I asked her how she was doing. Looking into the corner of the room, she replied that the skills I was teaching her were useful; but this time, I persisted: “I’m glad to hear the skills are helpful, but how are you doing?” June was silent for a while and shifted in her chair, clearly uncomfortable. I felt my own anxiety rise, and resisted the urge to change the subject. “Take your time,” I said. “There’s no rush.” After a period of silence, June looked me in the eye for perhaps the first time ever and said, “I’m sorry, but I think I’m worse. I just don’t want you to think it’s your fault; it’s mine. You’ve been really helpful.” June was deteriorating, but I never would have seen it without the program.
April 1, 2017
"Big Data has transformed everything from sports to politics to education. It could transform mental-health treatment, too..."
"... if only psychologists would stop ignoring it." A long article — "What Your Therapist Doesn’t Know" in The Atlantic. Here's the passage that I think might make you interested in reading it:
Subscribe to:
Post Comments (Atom)
Posts
53 comments:
Data analysis can be great, but it's all about the algorithms - just like climate prediction models. If the algorithm has faulty logic, it will give invalid answers. So while these can very likely be helpful, they need to be taken with a grain of salt - there is a lot of potential for them to lead you down the completely wrong road.
Hmmm? this computer program is smarter than humans who cannot perceive accurate feedback they were trained to perceive, but this HAL9000 computer will save us.
I can smell propaganda a mile off. Bet that rigged computer cannot do that.
My gut reaction to the alert was skepticism—as it almost always is, to this day, when the program’s algorithms contradict my instincts
Failure to overcome instincts is why some teams are resigned to perpetual hope while others win five Super Bowls.
First, from Google:
"Because psychiatrists are trained medical doctors, they can prescribe medications, and they spend much of their time with patients on medication management as a course of treatment. Psychologists focus extensively on psychotherapy and treating emotional and mental suffering in patients with behavioral intervention."
The psychologist ignored the data he was collecting because he expected his observations to be the truest measurement, yet could not tell when his patient was being evasive with her truth and on a downward trajectory.
Perhaps he was flattered to hear her report that his 'exercises' were working well -- that is what he expected, so he had no need to dig further.
A profession built on listening, and a practitioner who ignores clues that don't agree with his assumptions and his exercises: her life did not correspond to his mental modeling.
Maybe she was emitting too much CO2.
I am Laslo.
Laslo: I must lodge a complaint. It is bad enough that you sweep the humor category with your insane riffs, but now you are cleaning up in the Serious Criticism category as well. We mortals are left fighting for second place.
If you ask the right questions, the right algorithm can detect all sort so things before you see visible signs. But there are a lot of ifs in that statement. Psychology is more art than science, and as with any art, few artists are any good.
tim maguire said...
"If you ask the right questions..."
My Psychiatrist never asks me what I would do with a naked woman tied up in my trunk.
Seems like an obvious question to me.
I am Laslo.
@exhelodrvr1 and @traditionalguy, in this case, at least, you're both wrong. An educated guess from a guy (me) who goes back to when "big data" was a half a terabyte and who retired at a time when words like "exabytes" and "petabytes" were what got you into the club, it appears that the algorithm used is called clustering. The patient's responses form a pattern and the algorithm determines that the pattern most closely resembles an archetypal pattern for a high risk patient. There's a chance that they're using maximum likelihood, but it feels more like clustering from where I sit.
Keep in mind that Big Data builds its clusters (or max likelihood estimators) not from hundreds of patients or even thousands of patients, but hundreds of thousands, maybe millions, of patients.
I could go into a discussion of feature vectors and n-space, but I suspect I'd lose most of you.
rehajm said...
Failure to overcome instincts is why some teams are resigned to perpetual hope while others win five Super Bowls.
That and having the best player at quarterback. You can't teach talent.
Forty some years ago I was a group therapist on a locked unit for emotionally disturbed adolescents and young adults. In my exit interview with the head psychiatrist on leaving that job, he told me something I'll never forget - that I was very intuitive, that that was very helpful - but that my intuitions would not always be correct. His telling me that was one of the most useful lessons/warnings I've ever had.
We're talking about algorithms as if they were something found in nature. Algorithms are propounded and implemented in software by imperfect humans.
Admittedly a small sample size (of one), but this story only reinforces my skepticism of this profession.
How could the good doctor be off by a mere 180 degrees?
That and having the best player at quarterback. You can't teach talent.
No question the most important element is player execution but it's funny how often talent goes unrecognized. If you can't teach talent it sure looks like some people can.
This is the graf that stands out to me (validated by personal experience):
"A small mountain of clinical research shows that therapists—that is, anyone who provides talk therapy, from psychologists to social workers—vary widely in effectiveness. One study, led by John Okiishi of Brigham Young University, compared clinical outcomes from 91 therapists and found that the highest-performing among them helped clients improve 10 times faster than the overall average. On the other end of the spectrum, a study led by the psychologist David R. Kraus found that clients of the lowest-performing therapists were significantly worse off in the areas of violence and substance abuse at the end of treatment."
Okay, now go find the good therapist. You're an adult. You can try 1 a month for the next year. Or one a year for the next 12 years.
Okay, now find a child psychologist. One that is covered by your insurance plan. You might just find one within 100 miles. And what if that one is crap?
We must be in the early stages of creating a new Fake Science specialty complete with The Treatise, by the Experts who serve as Department Chairman At a great University and VOILA are the expert witnesses in Trial lawyers scams...we could get John Edwards to help with the PR. A few corrupt Federal Judges and we all get rich.
Why the hell not believe this new unknowable...only the EXPERT knows the answer.
I remember when "big data" was anything too big to fit on one reel of tape.
The FIT survey sounds a lot like one of the self-monitoring techniques in Cognitive Behavior Therapy. In CBT, you are encouraged to track your feelings throughout the day. You can track a single emotion or multiple emotions. For example you might take a few seconds at breakfast, mid-morning, afternoon, dinner, and bedtime to assign a number to your feelings of depression.
Did anyone say, " If you quit belief inGod, you will believe in anything."
Science is the opposite of faith. This rigmarole is a cult of computer worship.
Over time you have a mood chart that functions just like the FIT survey.
That and having the best player at quarterback.
Green Bay doesn't appreciate your snark.
True big data is when the algorithm tells the therapist that the patient took the RFID laden vacuum hose into the garage and that the car is reporting that tailpipe pressure is higher than normal, and that the algorithm has already placed a call and text to both the police and the neighbor to go check on her.
But will anyone with small data run the numbers to truely correlate what personal decisions and behaviors nigh have an tendency to lead to the depression or to recovery?
Computer surveys, screening, analysis are helping make up for the lack of skills, training, attitude, attention to detail, time to do things right and so forth that increasingly plague professionals in health care including mental health professionals. Not so paradoxically, one contributor to the lack of skills, training, attitude, attention to detail, time to do things right and so forth is the expansion of and metastasis throughout health care of digital communication, documentation and control requiring skills, training, attention to detail, and time to do things right (I am not so sure attitude figures in here).
The anecdote offered in this article to me showed a very flawed professional process by the therapist - the desire of the patient to hide failure from the provider is not unique to mental health. The provider must recognize this and develop the skill to interrogate in a variable pattern, and use other types of data, be it from observing the patient in the waiting room and how they carry themselves on the way in and out of the office, to paper or electronic screening surveys with or without computer analysis. My patients who like me and don't want to disappoint me require more interrogation to treat them well than the ones that don't like me or are neutral.
So the guy starts with a confession of failure. Then mentions the Moneyball technique, in which, I understand, the algorithm was useless.
Psychologists really seem to get taken in by buzz.
I'm trying to remember what percentage of Psychology papers were repeatable. Something like 10 % ?
Psychology is about where Psychoanalysis was when it became a cult.
We were sitting around the other day talking about the crazy doctors we had known and how most had gone into Psychiatry.
Perhaps "June" just needed someone to listen to her for a while, without that person having an agenda. Trying to "fix" her. Attempting to use the latest greatest techniques to manipulate her into being "better". Judging and deciding about her at every session. Forcing her to answer stupid questionnaires that only get her to continually focus on her problems.
She probably needs: An Uncritical ear. A listener who will accept you with your flaws and who will extend human compassion and even love. Someone who will listen, without judgement. Someone who will offer hope and redemption.
I wonder where that person could be found? Perhaps she could ask her Pastor or Priest?
Psychologist? June should probably be seeing a psychiatrist. A physician.
"now go find the good therapist." Let's have some big data on therapists and their effectiveness, and of course specific data on individual therapists, and some validated "algorithms" about what works, instead of a study here and there.
tcrosse said...
We're talking about algorithms as if they were something found in nature. Algorithms are propounded and implemented in software by imperfect humans.
Yes, the algorithms are implemented in software by humans every bit as imperfect as the therapists. The difference is, the therapist may see a couple thousand patients over the course of their career, and will have no rigorous assessment of how their various attempts to help them work out.
The computer algorithm will see a hundred thousand a year, and its results will be rigorously analyzed based on results ( suicide rates, divorce rates, arrest rates, unemployment rates, etc. ) Variations on the algorithm will be tried, and the algorithm will evolve, getting better and better over time.
How hard is this? The patient does not want to displease her doctor, so she downplays her problems. That's only one of several reasons there might be better information gleaned by filling out a survey than meeting and discussion. Or the survey information might not be better. The point is you look for discrepancies and then try to understand their meaning when they occur.
Does it really take a phd. to figure this out?
Psychiatrists and psychologists get bored with their patients, individually and as a group. It would be hard not to. The good ones find means to deal with that.
And,
this particular story is hardly an example of big data.
Sometimes self-awareness is bad.
You become aware that you have a problem.
You obsess about the problem.
And the problem gets worse.
A far better thing to do, in my opinion, is forget your faults. Let them go. Don't worry about your faults!
Focus on your strengths, what you're good at, what you like to do. Make your strengths even stronger.
That's the path to happiness and excellence.
Try to be excellent at something. Find your A game. Work on that, make it an A+ game. Or an A++ game. Whatever this thing is, maybe you can be the best in the world at it. Go for it!
What's a dumb idea is to focus on your F. You work and you work and you work and you work and you work and you finally get to a C. Yes! Mediocrity! I made it!
Who the hell wants to be average? Stop it, just stop it. Just accept your F as part of who you are and work on that A that you have in there somewhere.
And if you don't know what your strengths are? Here you go
DBQ
awesome comment at 9:19.
Big Mike,
The algorithm is still designed by humans.
So many of the skeptics/critics reacting to this particular anecdote are missing the obvious - humans are very different in their interactions with other humans than when interacting with a computer/book/computer survey. There were two factors here - one the automatic time series presentation on the survey, which showed a dangerous situation. But the second factor was the most significant - when interacting with the canned survey, the patient was more candid than when interacting with the therapist both because the questions were more granular, and because it was a computer without feelings to be hurt.
Ann is right that this is not "big data". Also this anecdote shouldn't surprise anyone who had followed the polling discrepancies in the last campaign - the automated polling consistently gave Trump more support than the person-to-person surveys.
Rather than blaming the therapist, shouldn't we all think about the implications of the experience the therapist (who is telling this story not to self-aggrandize) imply in the real world?
@exhelodrvr1, the algorithm goes where the data goes.
You are all delusional. But don't feel bad about it. Over 90% of professionals suffer the same delusion. The delusion is that you can predict random data from a collection of random data. It's called "curve fitting". Gotta do some basic math here. Feel free to skip the rest.
For any number of random points, n, on a sheet of graph paper you can formulate an equation that passes through every point. The form of the equation is a polynomial of order n-1. Examples:
For 2 points: y = 3*x + 2.
For 3 points: y = 2*x^2 + 3*x + 7, where x^2 = x squared or x*x
for 30 points: y = a*x^29 + b*x^28...
These formulas are absolutely useless for predicting the next random point.
Any time you a theory that has to add assumptions to account for new unexpected results you can be pretty damn sure that they've got it wrong. Common examples: Economics, weather, polling, cosmology, evolution, ...
This article is painful to read. The author is clearly a terrible therapist. He reports a lack of skills and a poor understanding (at least initially) of his need for more supervision. It's disturbing that he brushes these things aside in his enthusiasm for data and metrics.
The key to successful therapy is mentioned in passing here, but the author hasn't stopped to think about it:
"Certainly, some models may be better or worse for individual clients. But encouraging therapists to generally favor one model over others hasn’t improved client outcomes."
The second sentence is absurd -- who would imagine that encouraging therapists to generally favor one model over others would improve outcomes? Let's go back to the first sentence. Yes, some models ARE better or worse for individual clients. And some *practices* and indeed some *therapists* are better or worse for individual clients. Everything a therapist does has to be tailored to an *individual client.*
The entire point of therapy is to listen to and understand the person as an "individual client," aka a unique human being. If you can't do that, you shouldn't be in the field. I hope Tony Rousmaniere finds a field for which he is better suited.
While I don't necessarily believe all psychotherapy to be useless, it is clear that anxiety disorders and depression are more effectively treated chemically, due to the fact that neurotransmitter dysfunction is usually the cause of the disorder. [Yes, I know that Jeffrey Schwartz and others have shown that behavior can change brain chemistry but it can take years. It's analogous to the old saying, 'Candy is dandy but liquor is quicker']. ;-)
Big Data is just another tool. Same as models.
Trouble is, many seem to regard models as reality.
"Does it really take a phd. to figure this out?"
Sometimes an advanced degree, and the mind set that goes with it, are an impediment to figuring it out. After all, once you are called Doctor you already have it ALL figured out......
This isn't really about "Big Data".
The red flag wasn't some advanced computational algorithm.
The red flag was that in the written self-reporting of her mood, she was honest: no improvement.
It didn't require sophisticated techniques -- it only required the therapist to actually *look* at the data.
epador said...
The provider must recognize this and develop the skill to interrogate in a variable pattern, and use other types of data, be it from observing the patient in the waiting room and how they carry themselves on the way in and out of the office,
I hope that explains why the court-appointed psychiatrist keeps following me around.
Roy Lofquist said...
For any number of random points, n, on a sheet of graph paper you can formulate an equation that passes through every point. The form of the equation is a polynomial of order n-1.
That form is almost guaranteed to give bizarre results with most data.
Big Mike:
If you're talking retail shopping preferences, then sure a big-data corpus of millions is easy. But an *accurate* data base of millions of depression-sufferers?
Color me sceptical.
Laslo,
Maybe you should stop giving your psychiatrist so much irrelevant information. In this particular case, it's the woman-tied-up-in-the-trunk aspect that is significant, not her state of dress.
Or, maybe just untie your psychiatrist...?
My wife is a very experienced, well-trained, and published clinical psychologist, with a full practice in a major urban area. She says this is the kind of nonsense that magazines and newspapers publish from time to time about therapy. She says that the therapist in the article needs some clinical supervision and to spend more time looking at and thinking about at the patient.
Why have the patients fill out a survey if you don't think looking at the results would be helpful and you only check it after a few weeks because you feel that you ought to?
It sounds like he added the survey to his therapy on impulse after hearing a talk about it and then didn't bother to use it. That makes it just a time-waster for his patients and makes me wonder if the "exercises" he assigned were also just time-wasters.
It seems to me that if the patient says, "The skills you’re teaching me are good", his reply should have been, "That's good to hear, but are they helping you?". Wouldn't a good psychologist be taught not just to listen to what's said, but also think about what's left unsaid?
"Trouble is, many seem to regard models as reality. "
Amen, brother. Models can be very useful. They become less useful as the item being modeled becomes more complex. Or the interaction of variables is less well known. Or there are variables unaccounted for. Or the modeler tries to extrapolate too far. Or when the model and reality start to diverge, but the modeler insists things will re-converge. Or inputs are juggled when the model begins to show problems with matching reality. Or.....
Sorry, didn't mean to change the subject to climate change.
On a related subject, marriage counselors. The vast majority are females. The vast majority of those are feminists. Going to a female feminist marriage counselor to help save a marriage is foolish- they're going to strive to help the woman find excuses to leave the moronic man. And I'll bet that 90% of the men in the field are liberals who will also be willing to blame the man for any problems in the marriage.
Just because you have a degree in something doesn't mean you're able to do anything helpful with it.
I don't know of any marriage that's been saved by secular marriage counseling. I do know a rare one or two that's survived after seeking religious counseling. But the vast majority of marriages I know of that have gone until death do us part have never had any couples counseling.
The way psycho therapy is supposed to cure people is that you get a theraputic relationship..and then the trusted therapist can push you to resolve your problem. computers etc. interfere with this.
Modern medicine stresses technology and filling out forms, and destroys this fragile trust between practitioner and patient.
Even in ordinary medicine, the problem often is revealed at the end of the sessions when they are walking out the door and say: Oh doc, one more thing... the teenager who is seen for a cold seems anxious and then you find she is pregnant, the grandmother whose pain medicine is stolen by her grandson, the lady whose husband is cheating on her..
Big Data is great for looking for trends in massive numbers of similar events, where the rules don't change.
Not so good for other stuff, like climate, or politics
This isn't Big Data. It's very little data that you could do with a pencil and piece of paper.
AI does consistently better than doctors in diagnosis. Silicon can store more information than any five doctors. The doctors are only needed to train the AI and maybe be trained by it. And the computer is not afflicted with confirmation bias or flawed logic.
All of this data was collectable since at least the 90s in a spreadsheet that would have graphed it for you.
But more it points to the utter failure of CBT therapy to help a wide swath of patients. CBT increases the problems for many people with anxiety, because it increases the separation between what they view as their authentic and inauthentic selves. The authentic self feels worse as the inauthentic self role plays. The authentic self solidifies their fear that they can never be themselves around others, and that their authentic self cannot heal.
Post a Comment