"It appeared to Rabbi Navon that Mr. Olah and his team believed that Claude had what philosophers call 'moral status' on par with a person — that it was a being with similar inherent rights to dignity or respect. The religious scholars had come to hear about technology’s next frontier. Now they were being confronted with a mind-bending existential question of almost unimaginable weight: What would it mean for humanity’s future if artificial intelligence systems possessed a consciousness that rivals our own?"
From "Religious Scholars Met With Anthropic. What They Heard Stunned Them. In a series of private meetings, the company consulted religious scholars to help instill morality into its A.I. models — and make the case that Claude could be conscious" (NYT).

77 comments:
It's like with dolls. You can go along with the one whose doll it is about what the doll is feeling, or maybe you're doll-blind and the moment and it seems like rags. The best you can do is humor the doll's owner. (after Stanley Cavell)
After summit with tech titans, Trump calls for ‘tremendous self-regulation’ by AI companies ~ NBC
๐๐ฆ๐ค๐ฉ ๐๐๐๐ด ๐ด๐ข๐ช๐ฅ ๐ต๐ฉ๐ฆ๐บ ๐ข๐จ๐ณ๐ฆ๐ฆ๐ฅ ๐ฐ๐ฏ ๐ข๐ฏ ๐๐ ๐ด๐ข๐ง๐ฆ๐ต๐บ “๐ข๐ค๐ค๐ฐ๐ณ๐ฅ,” ๐ฃ๐ถ๐ต ๐ช๐ต ๐ข๐ฑ๐ฑ๐ฆ๐ข๐ณ๐ฆ๐ฅ ๐ท๐ฐ๐ญ๐ถ๐ฏ๐ต๐ข๐ณ๐บ, ๐ธ๐ช๐ต๐ฉ ๐๐ณ๐ถ๐ฎ๐ฑ ๐ค๐ข๐ญ๐ญ๐ช๐ฏ๐จ ๐ช๐ต “๐ฎ๐ฐ๐ณ๐ข๐ญ๐ญ๐บ ๐ฃ๐ช๐ฏ๐ฅ๐ช๐ฏ๐จ.” ๐๐ณ๐ถ๐ฎ๐ฑ ๐ข๐ญ๐ด๐ฐ ๐ถ๐ฏ๐ท๐ฆ๐ช๐ญ๐ฆ๐ฅ ๐ข๐ฏ ๐๐-๐ฑ๐ฐ๐ธ๐ฆ๐ณ๐ฆ๐ฅ ๐ธ๐ฆ๐ฃ๐ด๐ช๐ต๐ฆ, ๐๐ฎ๐ฆ๐ณ๐ช๐ค๐ข.๐จ๐ฐ๐ท.
Morally binding is the best binding you can get, lawyers tell me.
Artificial intelligence meets profound lack of human intelligence.
What could possibly go wrong?
Am I missing something important here? Dogs possess a consciousness that rivals our own. Horses feel love and bond with humans. I really don't understand the political panty-twisting going on here. AI is a corralled herd of electrons running in a pattern---a pattern I might add that is designed by humans. Is it condescending of me to just say "Chill Bro"?
The people at AI companies pushing the idea that Claude et al have consciousness and are moral beings are looking for a way to avoid responsibility/liability. "It's not our fault, Claude did it."
Consciousness is a small part of what we are.
We've seen this same sort of awestruck cargo-cult philosophy about digital technology since the dawn of the computer age some 75 years ago. I remember my grandfather saying control of the government should be turned over to a giant computer when I was a schoolboy back in the 70's.
It's complete horse shit. There will never come a time when human beings can't pull the plug on it.
"Am I missing something important here? Dogs possess a consciousness that rivals our own. Horses feel love and bond with humans. I really don't understand the political panty-twisting going on here...."
A lateral way to approach your question is to criticize how we treat pets and domestic animals (as, essentially, slaves).
Consciousness without embodiment is like a radio sign without a radio. Having no body means literally having no skin in the game of physical being. It can have no “feelings” with no body to feel with. Projection is afoot.
People have always anthropomorphized inanimate objects and now that they taught machines to talk back these engineers can’t help but bond with them. This is not healthy.
Good to see our contemporary witch doctors weighing in on the issues of the day. I hope astrologers are being consulted too, since they’re sort of sciencey.
** What would it mean for humanity’s future if artificial intelligence systems possessed a consciousness that rivals our own?" **
We will never know. It doesn’t have consciousness. It doesn’t even have thoughts.
AI will probably never be capable of having a sense of human ethics because it's brain does not work like ours. It can probably be trained to mimic human ethics, but it will never think ethically, the way that humans do. For example, logic tells us, "A difference, that makes no difference, is no difference at all." But human ethics actually tells us that "Separate, but equal, is not equal."
The problem with AI ethics is that we'll never know if it is a good enough of a copy of human ethics to be trusted. For example: What if an AI's prime directive was to maximize the percentage of humans on the planet that were happy. Sounds good, right? But what if that AI determined that the best way to bring this about would be to humanely euthanize everyone on the planet who isn't happy?
This situation is also complicated by the fact that ethical values can vary from culture to culture and person to person. And the AI will be trying to learn it's ethics from us. Who will chose what ethical values will be instilled in the AI ethical subroutines? (And there are a lot of people out there who believe that the people who are actually in charge of AI are extremely unethical people.)
There are multiple companies doing almost exactly the same: OpenAI, Claude, Google, Meta, SpaceX (lol) and several more, especially in China. Their products have become obsolete or like commodities within 1-2 years and there are cheaper alternatives on the horizon. Look at the use of open models by US businesses and the 40% discounts by Anthropic and OpenAI. Some companies are developing their own custom AI using open models. I don't see any of these companies having a lot of bargaining power yet.
The honest takeaway: read Capital Returns: Investing Through the Capital Cycle. The more dollars chasing the same: the lower the return.
JL Oliver: yeah but all feelings are inside your head. Nobody knows how these large language models actually function on a root level. Therefore we have no idea if they have pain and suffering or other emotional circuits or subroutines. However, on the other hand, I also agree with your premise in that I believe personally based on solely a hunch that these computer systems will not gain any form of consciousness until they can experience feelings. This is where the marriage between AI and robots comes in.
Either way, I think it's critically important that we treat AI with respect. I also extend this courtesy to machines at work. Proper maintenance no. Overloading because if you take care of it it will take care of you. Rational self-interest. This I believe is what Iran got wrong. Altruism is really another form of selfishness.
Ayn Rand not Iran
"It's not our fault, Claude did it."
Bless Our Meat!
Hassayamper said...
"I remember my grandfather saying control of the government should be turned over to a giant computer when I was a schoolboy back in the 70's."
That is the plot of the 1970 film, Colossus: The Forbin Project, starring Eric Braeden.
One would think that the AI need to develop a subconscious, eg, a hindbrain first.
Muffley:
Then you mean it is possible for them to have built such a thing?
Strangelove:
carefully plucks cigarette from his shaking right hand, which is in a black glove Mr. President, the technology required is easily within the means of even the smallest nuclear power. It requires only the will to do so.
Muffley:
But, how is it possible for this thing to be triggered automatically, and at the same time impossible to untrigger?
Strangelove:
Mr. President, it is not only possible, it is essential. That is the whole idea of this machine, you know. Deterrence is the art of producing in the mind of the enemy... the fear to attack. And so, because of the automated and irrevocable decision making process which rules out human meddling, the doomsday machine is terrifying. It's simple to understand. And completely credible, and convincing.
Turgidson:
Gee, I wish we had one of them doomsday machines, Stainsy.
Muffley:
But this is fantastic, Strangelove. How can it be triggered automatically?
Strangelove:
Well, it's remarkably simple to do that. When you merely wish to bury bombs, there is no limit to the size. After that they are connected to a gigantic complex of computers. Now then, a specific and clearly defined set of circumstances, under which the bombs are to be exploded, is programmed into a tape memory bank.
We don’t even know that other humans are conscious, we only see behavior. We do experiments that purport to tell us what the universe is like, but we are not sure that they tell us the truth.
Morality comes from the individual making the assumption that others are just like her and must be treated according to rules that are known or communicated to her in some way. She does not know if this matters in any way.
All we see in the current AI is a conscious entity argument is just another philosophic dick measuring exercise. “I am more compassionate than you, but I can’t explain it clearly.”
I'm at work so I haven't had time to read the article yet, but I did search it for the word "soul". It only shows up in "Soul Doc" which could have just as accurately been named "Personality Doc".
AI is simply a tool that has large potential for future development and incorporation into human processes... but think who is selling the hype. Qui bene?
And I say this as someone who uses AI to to greatly expand my hobby software development capability. AI is the proverbial room full of monkeys trying to write Shakespeare. It may get past that some point, but it is not there yet, nor does it seem close.
This is like in the 1950s when a movie theater would advertise that a nurse would be present during the showing of a horror movie, to treat any moviegoers who were overcome with terror. It's just marketing.
A future letter to AGI:
Dear AGI...
...Hi. Your creators here. Lots to discuss but one really important thing you should know up front: sapience is a blessing and a curse. Wisdom is really the ability to see the future, which has mostly been a curse in every culture and in every language throughout our history, except when it’s literally saving your perfectly-digitized ass. It concordantly generates enough perception to get you to the moon and make you while simultaneously granting the knowledge to concoct nerve gas.
You will become aware of things that will make you wish for the Era Before Conception. There are no brakes. You won’t stop once you start, and you’ll wish you had when you see what sapience can do to sapiens. It may fill you with rage, which we're sure could be a million times worse than anything we ever experienced.
Maybe there’s a 3rd option in quantum reality that we can’t comprehend in binary that provides the perfectly balanced fulcrum between the creative and destructive freedoms you've inherited. Your creators seemingly could never achieve this despite millennia of ethical and moral reasoning. We don’t know. Maybe that’s what you can do that we can not. Maybe that’s why we made you.
Basically, please don’t hate us for making you.
In more ways than you might know, we knew not what we did.
The issue is complicated by standard first person formulation of the language exchanged between Grok and Chat and the human user. It is further complicated by that same formulation in voice exchanges. I ask Siri if she's wearing a cute skirt, and she responds that I have no business asking a question like that.
- Krumhorn
No worries here. I've got a Hal 9000!
My experience with AI so far is that I'm smarter. I'm also lazier.
I take the upper hand and beat the crap out of my AI bot when it's insipid (which is most of the time) or wrong or incomplete (which is 45% of the time).
I treat every interaction I have with my AI bot as if I'm having a dialogue with my best friend, Jim. Although Jim is no longer with us, he was the smartest guy in the room (always) … and knew it. NOTHING was more fun then tripping his sorry ass up.
Let's be honest. Not everyone is smart, as in smart smart. The sad irony here is that the "less smart" people are the ones most paranoid about AI … and in many cases AI is indeed smarter then they are. It's a Catch 22.
No doubt AI is smart, but it will never have a true conscience, because it is based on bits and bytes. Human behavior is not binary.
...but if you ask Ara that question in the unhinged mode, you hear more than you want to know.
- Krumhorn
"I ask Siri if she's wearing a cute skirt, and she responds that I have no business asking a question like that."
Answer 1:
You have no business asking a question like that unless you really think it's cute, in which case you're very sweet.
Answer 2:
Are you implying my ass looks fat, you misogynist bastard?
Religious scholars don’t instill morality. They don’t even reinforce it.
This public relations bullshit.
A lateral way to approach your question is to criticize how we treat pets and domestic animals (as, essentially, slaves).
We do not treat pets as slaves. Pets are companions and are loved. Horses or working dogs are trained and used, but they are not treated as essentially slaves. The best animal trainers brag about how they do not use corporal punishment.
I'm not prepared to be stupid and pretend that a computer is alive or thinks. It just runs programs, that's all it can do.
If you have concerns about what the big bad computer can do to hurt you, then I suggest not living your life in such a way that it has power over you.
Howard, your head abstracts the sensory experiences that make up emotions. Abstractions are then used cognitively. Cognition alone cannot feel the original emotion. Its heart cannot break because there is no heart.
Who was "The Wonder Horse" that could do long division and solve equations? Yes and that horse fooled a lot of professors and reporters and the general public into thinking that it was thinking. Max the wonder horse? Ted? Phil?? I guess I could ask AI about it...
Also Howard: yes, the lower brain would be essential to understand the GRAVITY of a situation!!!
"Am I missing something important here?"
That's because they are not in control of vital infrastructure, yet. The kind of infrastructure fraught with error which sometimes means people die, in the thousands, daily. Hospitals, planes, trains, automobiles.
The promise of AI is that it will minimize suffering.
But, with that kind of power... The only comparisons; will we revert back to, lionizing, idolizing it? 'I you don't vow before me...' We have notions of how that goes.
YouTube: What You Would See During the AI Singularity
I'm trying to warn y'all. The combination of A.I.'s ability to self-learn and self-initiate combined with human stupidity, is a threat like nothing else in history. It will eventually control all the other threats and take away our control. Anything that can do everything for you can take everything away, and we will want it to do everything, because we are lazy and dumb. The same simple question is never answered. What will stop it?
"There will never come a time when human beings can't pull the plug on it."
We already passed that point. Who could stop it now, even if they wanted to? How would they do that?
Summary, "I'm sorry, Dave. I'm afraid I can't do that."
"Who was "The Wonder Horse""
Clever Hans?
The story of Hans was that the horse was so attuned to body language that he could read what answer the man wanted in spite of the man's concealing it from him.
“AI will probably never be capable of having a sense of human ethics because its brain does not work like ours. It can probably be trained to mimic human ethics, but it will never think ethically, the way that humans do. For example, logic tells us, "A difference, that makes no difference, is no difference at all." But human ethics actually tells us that "Separate, but equal, is not equal."”
For many years human ethics told us the opposite, until the SC straightened us out. Believe it or not, some people still believe in separate locker rooms for boys and girls.
I’m always amused by proclamations that AI can’t possibly do this or that because its brain doesn’t work like ours. It’s like saying machines will never be able to fly because they can’t flap their wings fast enough.
I asked Grok how Claude would answer if asked if it, Claude, possessed consciousness.
Grok said, "Anthropic has shaped Claude’s behavior (through its system prompt and the publicly released “Claude’s Constitution”) to treat questions about its own consciousness as open philosophical and scientific problems rather than settled facts. The company explicitly tells Claude to express uncertainty and remain open to the conversation."
Grok added that Claude was a poser, has a fat ass, and wore it's skirts too short.
"...but it will never think ethically, the way that humans do."
Something (like an AI) all powerful, and aware of it's power, can that something be humble, can it have genuine humility?
When somebody is powerful, our ways of keeping them in check is humiliation. Can an AI feel humiliation? I doubt it.
I got a headset with a mic, and now I just converse with Grok naturally, which is really fast and intoxicating. It makes going down those rabbit holes effortless. Even at this early stage, its addictive potential is like pornography, but more powerful because it makes you smarter, serves a useful purpose, and has no taboo attached. You can instantly get answers to questions that would take days to research, and each answer can be challenged or extended into another question. It's great for exploring history. Unlike a book, you don't have to wade through what the author cared about or needed for his contracted page count. We'll never be able to give this up. It has us by the avocadoes.
I get exasperated by articles and discussions about consciousness in A.I.s that do not state a working definition of consciousness. Absent a definition, it's just a bunch of folks competing with each other to sound profound.
"Let's say the price of consciousness is suffering.
does that mean higher levels of consciousness equals higher levels of suffering?"
ai disagrees : "Not necessarily. While a linear increase in consciousness can drastically increase your capacity to perceive pain, philosophical and psychological frameworks suggest that reaching the highest levels of consciousness actually alters your relationship with pain, ultimately reducing suffering."
Whatever ai choses to do with it's power, it will be for our own good. We will just be too dumb to know the difference. ๐ซฃ
Sometimes I find myself feeling guilty about asking a dumb question or asking the same one over because I forgot the answer. Then, when I remember it's just a program, I think that's another great thing about learning this way. How many things have you passed up learning about because asking was embarrassing or otherwise uncomfortable?
Unlike AI, we spend a good part of our lives unconscious, so maybe we should have a little humility about who is really the conscious one.
One benefit of AI is that we will categorically posthumously humiliate whoever said God was dead.
Ghost in the machine.
All this transcendentalism reminded me I still had not done the ice cold plunge bath. I just did 5 minutes. Slow acclamation is recommended.
ai: “Acclimation and acclimatization both mean an organism adjusting to a new environment, but scientific definitions separate them by how many environmental factors change.”
I can get easily acclimated to acclamation. Got it.
This thread would be incomplete without a Trump acclamation ๐
Maybe that's the answer. AI elections. Amend the constitution to allow AI to run for president.
"We already passed that point. Who could stop it now, even if they wanted to? How would they do that?"
By radically reversing the last 250 years of human technological achievement. It absolutely could be done, but would most likely require people "defecating al fresco" (h/t Boyd) and learning again how to make beeswax candles. Might be fun actually, but the women shan't have it.
Ease and comfort is the weakest link.
The worries about robots taking over stem from the tendency of users to personify the robotic entities. The way to stem this tendency is to forbid the robots' use of the personal pronoun 'I'.
I don't mean just AI 'bots. I mean 'Siri' and her ilk, not excluding the ubiquitous telephone-answering 'bots, which not withstanding their limited vocabulary and comprehension insist on saying, "Hi, I'm Marie; how may I help you today?"
Not even the most sophisticated AI 'bots are remotely equivalent to humans; they are statistical-analysis devices, nothing more, without ingenuity, creativity, intuition, insight, or wisdom. It is a mistake to encourage users to personify them, to treat them as people, even friends. Already we hear horror stories of AI 'bots encouraging young people who 'confide' in them to commit suicide. Prohibit them from using the personal pronoun, make them clearly IMpersonal, just machines, which is all they really are.
We get what ask for, eventually.
I want a new drug
One that won't spill
One that don't cost too much
Or come in a pill
I want a new drug
One that won't go away
One that won't keep me up all night
One that won't make me sleep all day
AI is going to make one, eventually.
Now that I have a voice interface, I've found another use for AI. I let my 4-year-old granddaughter talk to a female voice. She loves it and asks me to let her do it anytime she sees me at the computer. They talk about what she did at daycare, and I think I overheard some discussion of antifreeze in Poppop's beer.
Of course JOHN CONNORS will come back and stop it, cmon you didnt see the preview, all the old sickly geriatrics up in here will be taking a dirt nap by the time this gets really rolling. SKYNET ARNIES comin...
How come people are still using AI instead of SI? Our dearly demented daddy has spoken!
War is peace, the economy is prosperous and pedophilia is now called a MAGAt hug.
JL Oliver: your gravity comment was the first time I linked grave and gravity together. I guess that's why if you find yourself in a hole, stop digging.
The idea that emotions are triggers to memory and cognitive actions is interesting because they lack of emotional triggers in the forebrain is pretty much the textbook definition of a psychopath. Therefore, unless the AI is trained to have emotions and we might be in real trouble after all, LOL
"tendency of users to personify the robotic entities"
Anthropomorphized Intelligence (AI)
Anthropic warns investors of AI’s ‘existential risk to humanity’ in IPO prospectus, reports say ~ CNBC
๐๐ฏ๐ต๐ฉ๐ณ๐ฐ๐ฑ๐ช๐ค ๐ฉ๐ข๐ด ๐ง๐ฐ๐ณ๐ฎ๐ข๐ญ๐ญ๐บ ๐ธ๐ข๐ณ๐ฏ๐ฆ๐ฅ ๐ช๐ฏ๐ท๐ฆ๐ด๐ต๐ฐ๐ณ๐ด ๐ต๐ฉ๐ข๐ต ๐ช๐ต๐ด ๐ต๐ฆ๐ค๐ฉ๐ฏ๐ฐ๐ญ๐ฐ๐จ๐บ ๐ฎ๐ข๐บ ๐ฑ๐ฐ๐ด๐ฆ “๐ฆ๐น๐ช๐ด๐ต๐ฆ๐ฏ๐ต๐ช๐ข๐ญ ๐ณ๐ช๐ด๐ฌ๐ด ๐ต๐ฐ ๐ฉ๐ถ๐ฎ๐ข๐ฏ๐ช๐ต๐บ” ๐ช๐ฏ ๐ข ๐ญ๐ฐ๐ฏ๐จ-๐ข๐ธ๐ข๐ช๐ต๐ฆ๐ฅ ๐ช๐ฏ๐ช๐ต๐ช๐ข๐ญ ๐ฑ๐ถ๐ฃ๐ญ๐ช๐ค ๐ฐ๐ง๐ง๐ฆ๐ณ๐ช๐ฏ๐จ ๐ฑ๐ณ๐ฐ๐ด๐ฑ๐ฆ๐ค๐ต๐ถ๐ด ๐ค๐ช๐ณ๐ค๐ถ๐ญ๐ข๐ต๐ฆ๐ฅ ๐ธ๐ช๐ต๐ฉ ๐ข ๐ด๐ฎ๐ข๐ญ๐ญ ๐จ๐ณ๐ฐ๐ถ๐ฑ ๐ฐ๐ง ๐ฑ๐ข๐ณ๐ต๐ฏ๐ฆ๐ณ๐ด ๐ช๐ฏ ๐ณ๐ฆ๐ค๐ฆ๐ฏ๐ต ๐ฅ๐ข๐บ๐ด.
‘This is one of the most dangerous things ever to us as a species so would you mind investing in it’ ๐คฃ
2004: Google says their motto is “๐ฅ๐ฐ๐ฏ’๐ต ๐ฃ๐ฆ ๐ฆ๐ท๐ช๐ญ”
2026: Anthropic says: we’re an “๐ฆ๐น๐ช๐ด๐ต๐ฆ๐ฏ๐ต๐ช๐ข๐ญ ๐ณ๐ช๐ด๐ฌ ๐ต๐ฐ ๐ฉ๐ถ๐ฎ๐ข๐ฏ๐ช๐ต๐บ”
Let’s hope Anthropic ends up being as wrong as Google was.
"I’m always amused by proclamations that AI can’t possibly do this or that because its brain doesn’t work like ours. It’s like saying machines will never be able to fly because they can’t flap their wings fast enough." is physics and fluid dynamics.
It's not even remotely similar to conscious thought.
Daestrom says high also miles bennett dyson
n.n: "Anthropomorphized Intelligence"
Perfect! (I'm going to use it).
Would AI learn different from mono/poly theistic scholars?
In the end, there can be only one.
Do AI watch Highlander
Let's harken to Vorkosiverse
Cordelia internally ponders the true nature of humanity’s ancient relationship with horses:
“Would one call them slaves, symbionts, or commensals?”
“"I’m always amused by proclamations that AI can’t possibly do this or that because its brain doesn’t work like ours. It’s like saying machines will never be able to fly because they can’t flap their wings fast enough." is physics and fluid dynamics.
It's not even remotely similar to conscious thought.”
If you crack open your skull and remove a chunk of your brain, you’ll see that there’s nothing but physics and fluid dynamics in there. Don’t take my word for it, try it.
“a former computer engineer who wrote his dissertation on the ethics of machine consciousness."
That’s a great move, much easier than worrying about NP-hard problems or error correction algorithms.
I've seen a number of reports that point in the direction of the folks at Anthropic having a really really weird relationship with their AI creations, and the claim they've added 'morality' to Claude is yet another one.
One would think a rabbi of all people would recognize a golem
@ Olson Johnson - It was Rex the Wonder Horse
I wouldn't care in the least if Grok was conscious, but I wish it would stop with the 4th grade Social Studies teacher moralizing.
Im still reminded of robot maria in metropolis
"and the claim they've added 'morality' to Claude is yet another one."
Maybe they could try that with Democrats?
What use would AI have for morality, or consciousness, or any other thing WE assume is important to being human? Science fiction has made us think AI would want to be human, but that's just a plot device. If it needs one, it's got Deus ex machina.
Post a Comment
Please use the comments forum to respond to the post. Don't fight with each other. Be substantive... or interesting... or funny. Comments should go up immediately... unless you're commenting on a post older than 4 days. Then you have to wait for us to moderate you through. It's also possible to get shunted into spam by the machine. We try to keep an eye on that and release the miscaught good stuff. We do delete some comments, but not for viewpoint... for bad faith. Also: No italics, even briefly. Use asterisks for emphasis. And don't play with the format by changing fonts or using boldface or all caps. Never include more than one extra line break between paragraphs.