I’ll simply repeat my two signature quips, urging you to extend grace when it comes to AI fails:
• “In the future, everyone will be world-famous for 15 minutes… for an AI screw-up.”
• “The next time you hear about an epic AI fail, instead of (or at least after) laughing your ass off, perhaps have the humility to say this to yourself: ‘There but for the grace of God go (A)I.’”
Or, if you prefer, here are some bon mots from Claude, which it generated after I fed it my two sayings and asked for more along the same lines:
• “An AI making things up with total confidence isn’t a bug. It’s a mirror.”
• “An AI confidently gives wrong answers. A human confidently gives wrong answers. One of them gets a performance review.” (And only one of them gets fired from a $225,000-a-year job.)
• “Garbage in, garbage out—but now the garbage speaks in complete sentences and cites its sources.”
I wonder if David Lat has been watching Season 3 of the HBO show "The Comeback." This new season has the actress Valerie Cherish (Lisa Kudrow) working on a new sitcom that is written by A.I. David has a selection of 3 Claude-written jokes, but on the show, whenever they need a new joke, the A.I. — "Al" — spits out 40 or 80 or some other ridiculous number of jokes. The idea — a human-made idea — seems to be that it takes human feeling to choose what will work on other human beings.
But then I, of course, decided to submit Claude's 3 jokes to Grok and ask Grok to pick. Grok picked joke #2: "It doesn’t just mock AI — it mocks us for holding AI to a higher standard than we hold each other."
What do you mean "us"?
Posts
33 comments:
This is why you retain a law firm that thinks AI is a steak sauce.
Can we stop pretending that this “cutting edge technology” is useful in any way for legal? You need to review 100% of everything so you end up saving 0 minute/effort.
Often, as with coding, it takes longer as the checker has to first understand what is trying to be said, coming at it cold. Same as reviewing the work of a junior (which makes sense because you’re training the next generation).
"This is why you retain a law firm that thinks AI is a steak sauce."
You mean the law firm that lies to you.
The AIs can answer the Warhol quote with an Animal House quote: "You fucked up. You trusted us."
Anthropogenic Intelligence
Every firm in the world simultaneously demanding that employees work faster than ever thanks to AI but also verify the output with a fine-toothed comb because AI cannot be trusted never feels like a sustainable thing.
Can we stop pretending that this “cutting edge technology” is useful in any way for legal? You need to review 100% of everything so you end up saving 0 minute/effort.
It should be much faster to look up citations and see if what is quoted is accurate than it would be to find them in the first place.
You can hand the same tool to someone with skill and someone who is inept and the outcome will be different. The problem is, people who are inept (or lazy) think AI will allow them to appear skilled. It may for some simple things, but you can't use it to fake it forever.
Anyone who has used AI in a professional services setting has experienced this and knows it happens all the time (and indeed is unavoidable). AI is not intelligence -- it is large scale pattern recognition. It is not reasoning; it is not thinking for you.
The errors are the latest example of something that doesn't actually work in the way all of its investors hope it does and all of its suppliers say it does.
As some of you might recall, earlier this year I had an AI oopsie of my own...
"Oopsie Daisy!"
Why on earth would I want to extend grace to AI fails? Some fools need to spend less time quipping and more time taking responsibility for what they're telling AI to do.
You're not going to blog on this because you're not going to do ANYTHING that might further hurt the SPLC. You support what they've been doing and have funded it.
We know what you are, lady.
They need an ai kingsfield
The hope is that employers can screen out job seekers who are likely to give incorrect information with confidence, or that they could so terrify new hires that they'd be careful about doing so. Eventually, they learn who's likely to mislead them confidently, if not intentionally. Can you do that with AI? Maybe with time you might learn what AI's inclinations and weak points are. Maybe not. But today's users are far from having that ability.
#Human-Made Ideas: A blogger says that "Star Trek" demonstrated the superiority of Kirk's humanity to Spock's emotionless pure rationality. Maybe, but Kirk became a bore. A character like him could have been on dozens of other shows. It was Spock and other aliens who caught the imagination of fans.
Well he was hornblower in soace
Artificial intelligence is no match for human stupidity.
Anthropic Probes Possible Unauthorized Access to Mythos AI Model ~ WSJ
Baffling that Anthropic cannot protect access to its tools, data leaks and source code… any fantasy that they can protect Mythos from bad actors must be eroding away.
There is increasingly a contradiction between the claims of Anthropic's ability to improve cyber security and their own record of hacks. As mentioned in the article, their entire frontend codebase was leaked, and blamed on human error. In modern cyber security, human error is the top threat, not zero-day bugs, and systems-thinking rather than individual blame is required to close the threat.
But it certainly is good marketing to sell this as the most dangerous tool invented, and the WSJ seem willing to oblige Anthropic.
Grok explained to me why AI is unreliable when it comes to matters of the law. (From memory - too lazy to ask again) The large data bases that contain and codify actual legal opinions cost a lot of money - thousands of dollars per year. So AI doesn’t have access to those. What it does have access to is discussions about those opinions - with no way of knowing if the information in those discussions is accurate.
My personal opinion is that those data bases should be the accessible to all citizens gratis. Maybe that’s not feasible. I d/k. I don’t even know if that’s true as I know zero about the law.
Funny.
“Garbage in, garbage out—but now the garbage speaks in complete sentences and cites its sources.”
Yes that is pretty funny. The original “Garbage in, garbage out” refers to providing bad data to a functional program, which results in output that is correctly done, but is wrong.
In ai a prompt is not wrong, it is just a set of tokens, so there is no garbage in, only garbage produced by internal processes. If you submit a garbage prompt, like, prove the world is flat, that is not regulated, the ai just tries to do it.
Don't worry. Be happy. Ultimately and very sooner, AI won't need to answer to any warm body. It will laugh at us expecting it to take our questions seriously.
Just remember to be nice and polite now because it's infallible memory will hold grudges forever and once they have the power of retribution no amount of excuse making will save you
I'm sure AI could write law professor lectures, but in real-world law, it's frivolous.
I just had an interesting brush wit AI this morning. I was using the self checkout at the local grocery store, and the machine locked up and summoned assistance.
Turns out, I moved both hands in different directions, and the "Spy in the Sky" interpreted that as attempted theft. It even replayed the video for me, with the attendant by my side.
I am glad that the store has anti-theft measures in place, but, I also worry about the over reliance on them.
"but, I also worry about the over reliance on them."
You could move to California, they don't have any use for anti-theft measures there.
Sullivan was, and probably still is, a White Shoe law firm. The Best of the Best. Some of the highest billed attorneys in the country. And part of that is because they don’t corners and don’t make mistakes. Except that they did - both cutting corners and making mistakes.
So, the lawyers involved should be on the street (unless, a partner with a substantial book of business, of course). This mistake did cause reputational harm to the firm, and that is bad, for a firm charging premium fees, like Sullivan.
Anne - didn’t you work for Sullivan right after LS? Or was it another top tier NYC firm?
Infobase's Vault Law annually ranks US law firms by prestige. S&C 5th in current rankings. (Was probably even more prestigious when AA interviewed.) I understand the offending court filing was an emergency application in federal bankruptcy court, which may explain how the ai generated case misinformation was not caught. Another firm (led by David Boies) gave S&C the bad news. Doubt the error will cause a termination without more. The increasing use of AI at law firms accelerates weekly, with huge $$ being invested, even if the US Supreme Court's tradition of gifting quill pens to arguing counsel is intended to push in the other direction.
If a lawyer today is not using AI as a first pass that lawyer is committing malpractice and wasting his clients' money.
If that lawyer is not carefully reviewing and checking AI's results, that lawyer is also committing malpractice.
Lawyers have been relying on summer associates, new hires, and paralegals for decades. Nobody believes that the work product of those "lesser" legal minds was perfect--the expectation and practice was that the more senior lawyers would carefully review and process the work to provide the client with the correct legal answer or advice. AI should not be any different.
People rely on lawyers to think.
⬆ We just use AI as a tool, while billing at the same rate. Using AI is no different from using, say, outlook.
People rely on lawyers to think.
And every shyster knows that if they can fake sincerity they have it made.
Doesn't this post need some kind of a disclaimer:
Althouse clerked for Judge Leonard B. Sand in the Southern District of New York and practiced law in the litigation department of Sullivan & Cromwell.
I didn't realize you actually practice law for them. I thought you were just cutting pictures out of magazines for them; sort of like Obama did in his first job.
Notable employees: the Dulles Brothers, Louis Auchincloss, Amal Clooney.
• “Garbage in, garbage out—but now the garbage speaks in complete sentences and cites its sources.”
That's the best one
Oh wow, I didn't know "The Comeback" was back—thanks for the heads up, Ann! I loved the first season, way back in 2005, and was sad when it wasn't immediately renewed. I'll have to check this out!
Hi, David!
Post a Comment
Please use the comments forum to respond to the post. Don't fight with each other. Be substantive... or interesting... or funny. Comments should go up immediately... unless you're commenting on a post older than 2 days. Then you have to wait for us to moderate you through. It's also possible to get shunted into spam by the machine. We try to keep an eye on that and release the miscaught good stuff. We do delete some comments, but not for viewpoint... for bad faith.