AI Will Never Be Conscious: Michael Pollan’s Argument Against Machine Sentience

In his new book, A World Appears, Michael Pollan argues that artificial intelligence can do many things—it just can’t be a person.

The Blake Lemoine incident is remembered today as a high-water mark of AI hype. It thrust the whole idea of conscious AI into public awareness for a news cycle or two, but it also launched a conversation, among both computer scientists and consciousness researchers, that has only intensified in the years since.

The Turning Point

The turning point came in the summer of 2023, when a group of 19 leading computer scientists and philosophers posted an 88-page report titled “Consciousness in Artificial Intelligence,” informally known as the Butlin report. Within days, it seemed, everyone in the AI and consciousness science community had read it.

The draft report’s abstract offered this arresting sentence:

“Our analysis suggests that no current AI systems are conscious, but also suggests that there are no obvious barriers to building conscious AI systems.”

The authors acknowledged that part of the inspiration behind convening the group and writing the report was “the case of Blake Lemoine.”

“If AIs can give the impression of consciousness, that makes it an urgent priority for scientists and philosophers to weigh in.”

But what caught everyone’s attention was that single statement in the abstract of the preprint: “no obvious barriers to building conscious AI systems.”

The Identity Crisis

When Pollan read those words for the first time, he felt like some important threshold had been crossed, and it was not just a technological one. This had to do with our very identity as a species.

What would it mean for humanity to discover one day in the not-so-distant future that a fully conscious machine had come into the world? I’m guessing it would be a Copernican moment, abruptly dislodging our sense of centrality and specialness.

We humans have spent a few thousand years defining ourselves in opposition to the “lesser” animals. This has entailed denying animals such supposedly uniquely human traits as feelings (one of Descartes’s most flagrant errors), language, reason, and consciousness.

In the last few years, most of these distinctions have disintegrated as scientists have demonstrated that plenty of species are intelligent and conscious, have feelings, and use language and tools, in the process challenging centuries of human exceptionalism.

The New Adversary

With AI, the threat to our exalted self-conception comes from another quarter entirely. Now we humans will have to define ourselves in relation to AIs rather than other animals.

As computer algorithms surpass us in sheer brainpower—handily beating us at games like chess and Go and various forms of “higher” thought like mathematics—we can at least take solace in the fact that we (and many other animal species) still have to ourselves the blessings and burdens of consciousness, the ability to feel and have subjective experiences.

In this sense, AI may serve as a common adversary, drawing humans and other animals closer together: us against it, the living versus the machines.

This new solidarity would make for a heartwarming story and might be good news for the animals invited to join Team Conscious. But what happens if AI begins to challenge the human—or animal, I should say—monopoly on consciousness? Who will we be then?

The Unsettling Prospect

Pollan finds this a deeply unsettling prospect, though he’s not entirely sure why. He’s getting comfortable with the idea of sharing consciousness with other animals (and possibly even with plants, in his case) and he’d be happy to admit them into an expanding circle of moral consideration.

But the prospect of sharing consciousness with machines? That’s a different story entirely.

Key Takeaways

The Bottom Line

Pollan’s book excerpt raises profound questions about what consciousness means and whether machines can ever truly possess it. As AI systems become more sophisticated, the line between simulation and genuine experience becomes increasingly blurred.

The Butlin Report’s conclusion—that there are “no obvious barriers” to conscious AI—suggests that the question is not if, but when. And when that day comes, humanity will face a reckoning not just technological, but existential.

Who will we be when the machines can feel?

—

Sources: [Wired](https://www.wired.com/story/book-excerpt-a-world-appears-michael-pollan/), [Butlin Report](https://arxiv.org/abs/2308.08708)

Tags: AI Consciousness, Michael Pollan, AI Philosophy, Sentience, Book Excerpt