Humanity has the debate about AI consciousness backwards

I’m can’t track what the point of this essay is. Suppose we replaced “AI” with “animal.” How would this essay be different? What am I supposed to take away? My concern with AI consciousness is about to what extent these machines can have an experience comparable to mine, which this essay doesn’t touch on at all.

If I understand the central argument, which is confusingly stated, the claim is that "consciousness" is a socially-determined, subjective, descriptor for certain things that one cares about. How does this make any sense? If I care about my teddy bear enough it'll gain consciousness?

Reminds me of the Rick and Morty pilot.

"They're just robots, Morty!"

*Blood splurt

"They're not robots, Rick!"

"I mean they're bureaucrats, Morty. I don't respect them."

There's the core of a reasonable argument if you forget about using the word "consciousness."

Consider playing a game where some players are human and others are implemented by AIs. You might want to model your opponents' minds in a similar way? Someone playing chess or Go will need to consider what moves their opponent could make. If it's poker then you need to consider if they're bluffing. If you can make alliances, you'll need to consider how far other players can be trusted.

You can put scare quotes around words like "intention" when discussing AI players if you like, but it might not matter when you're figuring out how to interact with them.

When a game is complicated enough, an AI player is going to need memory (likely implemented by the harness) and it will have to model the other players too in order to coordinate or compete with them.

This doesn't happen when I use a coding agent in a normal way to write code, but I haven't played around with persistent agent stuff like OpenClaw or Gas Town. They seem to be making their agents act in an unnecessarily complicated social environment, which seems to lead to weird situations?

Oh come on. Objects are objects because you want to freely observe them without consciousness and privacy and taboo attached to them. Entities are entities because you want to assign credit or blame to them, so you want to believe that they have agency so that you hold them responsible. We did this with sharp volcanic rocks once, calling them evil. Did it help? No, we got idolatry and eons of bigotry until we secularized once again.

Never forget that LLMs are just next token predictors. The harnesses are what's getting uncanny. And credit/blame assignment? Just assign a surrogate for now, let them deal with the consequences ffs.

This reminds me of a provocative YouTube video title I read the other day, one that makes me want to fight factory farming injustices. The title is "Animals to me are "Philosophical Zombies" - Destiny" https://www.youtube.com/watch?v=RtpLbmr4ijY I agree with Destiny a lot of the time but not here, though he seems to come around somewhat as the video goes on.

If a sheep or a pig isn't thought of as conscious because it doesn't have a name, like someone's pet dog or cat, indicating that we care about them, then that feels problematic to me. So this isn't a way to neatly solve the problem of AI consciousness.

The narcissism of the thesis that places the author as center of the universe and creator of the rest of the consciousnesses "because he cares".

That guy has acute Megalomaniac Solipsism Fever in disguise.

Oh look, another completely retarded article that doesn't even bother to note or deal with the definition problem. If you don't define what you mean by the term, your argument isn't even wrong, it is just a waste of everyone's time, since there are multiple highly conflicting aspects of and/or definitions of consciousness.

https://plato.stanford.edu/entries/consciousness/#ConCon

I wish the AI bros would take a fucking philosophy class. Jesus.

Wow. I have rarely read anything that made me lose more respect for the author in a single article. And this guy is a VP at Google? And he leads an AI research team?

His third paragraph, which is the core of his argument, is stupid. He says, "As intelligent, social entities, we decide ... which other entities we regard as having inner lives and being worthy of care." Yes, that's correct. "We do not care for others because they are conscious. Rather, we believe they are conscious when and because we care about them." (Emphasis in original). That's just utterly and completely backwards. We (at least, most people) avoid inflicting unnecessary pain on animals because they are conscious beings that can feel pain. (Obviously there are some exceptions, sadists who enjoy torturing animals, or even other human beings, merely because they like inflicting pain on other creatures — but note that even there, the sadist believes the creature he's torturing can feel pain, otherwise he wouldn't enjoy torturing it.) We take no such precautions with trees, because we know they do not feel pain. (Silly satirical songs notwithstanding: look up the song "Carrot Juice is Murder" by Arrogant Worms sometime if you want a good laugh).

Is this guy really, truly, seriously arguing that we first decide to care for an animal, and then because we care about it, we believe that it's conscious, able to feel pain and other emotions? That's just so incredibly stupid that it seems hard to believe that someone would actually make that argument — and yet there it is, in plain language — and it's the core of his argument. The truth, of course, is the other way around: we take care of animals because we know they are able to experience pain, hunger, and so on. And this applies both to pets that we come to care about, and also to animals that we feel no emotional attachment to. I have a relative who runs a dairy farm. He doesn't love the cows he takes care of; in fact, he's frequently quite irritated at them. (Ask any dairy farmers you know how often cows have tried to step on their feet while being milked — the farmers' feet, not the cows' feet!) But he still makes sure they're fed and healthy. Not just because it makes good economic sense to do so (sick cows don't produce good milk) but also because it's the right thing to do. He doesn't start to believe that cows feel hunger because he cares for them; he also believes that the cows on other farms, that he has no connection to nor duty of care for, would also feel hunger if not fed.

This is so intuitively obvious that I am stunned that someone would argue the opposite. And this is someone high up enough at Google to be making decisions about AI and so on? Wow. Good thing for him I'm not in charge of Google, because if anyone working for me were to make such a stupid argument I would be looking at his contract and local laws to see if it was legal to fire him for cause. Because if he can get his thinking so completely backwards from reality, he's not well-suited to work in tech where your work has to fit reality.

I have rarely, not never but rarely, encountered an article that so immediately led me to conclude that the author, whom I knew nothing about beforehand, is an utterly muddled thinker and not worth listening to on any subject.