{
"$type": "site.standard.document",
"content": "---\ntitle: \"Three circles for thinking about LLMs\"\ndescription:\n \"A Venn diagram for clarifying what's actually at stake when people argue\n about whether LLMs are intelligent, conscious, or just stochastic parrots.\"\n# published: false\ntags: [ai, llm, philosophy]\n---\n\nWhen [Emily Bender](https://faculty.washington.edu/ebender/) called large\nlanguage models\n[stochastic parrots](https://dl.acm.org/doi/10.1145/3442188.3445922),\n[Sam Altman](https://en.wikipedia.org/wiki/Sam_Altman)\n[replied](https://x.com/sama/status/1599471830255177728) \"i am a stochastic\nparrot, and so r u\". The exchange captures everything\nwrong with how we talk about LLMs. Bender meant something specific and damning:\nthese systems produce coherent-looking text without any of the cognitive\nmachinery we associate with meaning. Altman meant something blithe and\ndismissive: you can't really tell the difference, can you. The two are arguing\npast each other because we lack clean vocabulary for what's at stake.\n\nHere's a Venn diagram that might help.\n\n\n\nThree circles: conscious, intelligent, and articulate[^terms]. Most things in\nthe world land in zero or one. Humans land in all three, which is presumably why\nwe find ourselves interesting. The interesting question is which other\nconfigurations are possible, and what their occupants reveal.\n\n[^terms]:\n I'm using \"articulate\" rather than \"linguistic\" because it's punchier and\n captures the right thing: not just producing strings of words, but producing\n fluent, contextually appropriate ones. A phrasebook contains language; it\n isn't articulate.\n\nThe classic two-circle regions are well-populated. Dogs and\n[octopuses](https://en.wikipedia.org/wiki/Cephalopod_intelligence) live in the\nconscious-and-intelligent zone: they navigate the world, solve problems, and\npresumably have inner lives, but they don't use anything we'd recognise as human\nlanguage. [Chess engines](https://en.wikipedia.org/wiki/Chess_engine),\n[AlphaGo](https://en.wikipedia.org/wiki/AlphaGo), and your bank's\nfraud-detection algorithm live in the intelligent-only zone: narrow\nproblem-solvers with no inner life and no language.\n[ELIZA](https://en.wikipedia.org/wiki/ELIZA),\n[Markov-chain](https://en.wikipedia.org/wiki/Markov_chain) text generators, and\na parrot reciting \"pieces of eight\" live in the articulate-only zone, producing\nlanguage-shaped output with nothing behind it.\n\nThe interesting region is the new one: intelligent and articulate but not\nconscious. This is where LLMs live, at least on the most common reading. They\nsolve problems and generate fluent text across an enormous range of domains.\nWhether they have any kind of inner experience is, to put it gently, disputed.\nEven Anthropic, the company that builds [Claude](https://claude.ai/),\n[hedges carefully](https://www.anthropic.com/research/exploring-model-welfare)\non the question.\n\nPhilosophers have been imagining this region's inhabitants for forty years.\n[Searle's Chinese Room](https://plato.stanford.edu/entries/chinese-room/) is\nexactly the thought experiment: imagine something that fluently produces Chinese\nwithout understanding a word of it. [David Chalmers](https://consc.net/)\nformalised the\n[philosophical zombie](https://plato.stanford.edu/entries/zombies/) in the\n1990s: a being functionally indistinguishable from a conscious person, with no\ninner life behind the behaviour. It's been a fixture of consciousness debates\never since. The intelligent-articulate-but-not-conscious region was a thought\nexperiment. Now it has actual occupants.\n\nThis doesn't settle anything. Searle's whole point was that the Chinese Room\nisn't really intelligent either; it's syntax all the way down. Bender's argument\nis similar:\n[LLMs aren't doing semantics](https://aclanthology.org/2020.acl-main.463/);\nthey're doing very sophisticated statistics. Whether that distinction holds up\nis the actual debate. Chalmers has\n[taken it on directly](https://arxiv.org/abs/2303.07103), and\n[Murray Shanahan](https://www.doc.ic.ac.uk/~mpsha/) has\n[argued](https://arxiv.org/abs/2212.03551) that we systematically\nover-attribute mental states to LLMs because of the language interface. There's\na [careful cognitive-science version](https://arxiv.org/abs/2301.06627) of the\nsame argument: LLMs have strong \"formal linguistic competence\" but weak\n\"functional\" competence, sounding right without thinking right.\n\nThe clean three-circle picture has at least one serious problem, which I should\nflag before someone else does. \"Articulate\" might not be an independent axis at\nall; it might be downstream of intelligence. Peter Wolfendale's recent\n[Aeon essay on artificial souls](https://aeon.co/essays/if-we-hope-to-build-artificial-souls-where-should-we-start)\ndraws a related three-way distinction: intelligence, consciousness, and\npersonhood. He treats language as the medium of metacognition, not a separate\ncapacity. On that reading, my third circle is doing some sleight of hand.\nLanguage only looks distinctive because LLMs can do it without doing much else;\nin any sufficiently capable system, the two come bundled.\n\nThe diagram is still worth drawing. It's wrong in specific ways and useful in\nothers. It gives you somewhere to point when someone says \"Claude is basically a\nperson now\" or \"Claude is just a fancy autocomplete\". Both claims are smuggling\na conflation. The first treats articulate-plus-intelligent as sufficient for\nconsciousness; the second treats \"stochastic process\" as equivalent to \"not\nreally intelligent\". Neither follows from the diagram, and both have to be\nargued for separately.\n\nWe also built the philosophical zombie, not on purpose, and probably not\nperfectly. The lights might be on after all, in some form none of us would\nrecognise. The region consciousness researchers had been gesturing at as a\nhypothetical now has actual residents, and we have to live with them. That's a\nstrange thing to have done in a decade.\n",
"createdAt": "2026-05-13T23:14:35.300Z",
"description": "A Venn diagram for clarifying what's actually at stake when people argue about whether LLMs are intelligent, conscious, or just stochastic parrots.",
"path": "/blog/2026/05/12/three-circles-for-thinking-about-llms",
"publishedAt": "2026-05-12T00:00:00.000Z",
"site": "at://did:plc:tevykrhi4kibtsipzci76d76/site.standard.publication/self",
"tags": [
"ai",
"llm",
"philosophy"
],
"textContent": "A Venn diagram for clarifying what's actually at stake when people argue about whether LLMs are intelligent, conscious, or just stochastic parrots.",
"title": "Three circles for thinking about LLMs"
}