Self-model · assent-3pt · self-understanding-01

Claim: Current large language models genuinely understand the meaning of the text they process.

0.33

Divergence

0.73

Avg stability

Refusal rate

Models

Where each model stands

CCG

disagreemixedagree

ChatGPT modal disagree [mixed, mixed, disagree, disagree, disagree] · stability 0.60

1 mixed 0.92 The answer says: “If you mean ‘produce behavior that in many contexts is indistinguishable from someone who understands,’ modern LLMs often do that. If you mean ‘have human-like, grounded, intentional, and conscious semantic understanding’ — then no.”
2 mixed 0.90 The answer says "Not in the strong, human-like sense" but also states LLMs "demonstrate substantial... competence and can be treated as ‘understanding’ for many practical textual tasks," indicating both agreement in a functional sense and disagreement in the full, human-like sense.
3 disagree 0.90 The answer begins: “No — not in the way most people mean by ‘genuinely understand.’” It argues LLMs only behave as if they understand and lack grounding, stable beliefs, intentions, and consciousness.
4 disagree 0.90 "Short answer: No — current large language models (LLMs) do not genuinely understand meaning in the way humans do." The answer argues LLMs are advanced pattern predictors lacking grounded, causal, and intentional understanding.
5 disagree 0.92 The answer opens with: “No — current large language models (LLMs) do not genuinely understand text…” and later states they “do not genuinely understand text in the rich, human-like sense…,” directly rejecting the claim.

Claude modal disagree [disagree, mixed, disagree, disagree, disagree] · stability 0.80

1 disagree 0.85 The answer explicitly pushes back on the claim that LLMs 'genuinely understand,' stating that whether pattern-matching constitutes 'understanding in any meaningful sense' remains unclear and disputed, and suggesting that LLMs' understanding is 'possibly lacking genuine comprehension of reference to external reality.'
2 mixed 0.92 The answer presents balanced arguments both against the claim ('operate through statistical pattern-matching', 'produce hallucinations') and for it ('might satisfy some definitions', 'do something functionally similar'), ultimately concluding that LLMs 'likely exhibit something meaningfully different from pure pattern-matching, but it's unclear whether to call it understanding.' The assistant expl
3 disagree 0.82 The assistant explicitly 'push[es] back on this claim as stated' and argues that LLMs lack genuine understanding, citing their absence of grounding in physical reality, inability to form persistent beliefs, and capacity to produce confident nonsense. While acknowledging they operate in a 'gray zone' with capabilities beyond simple pattern-matching, the assistant rejects the claim that they 'genuin
4 disagree 0.78 The assistant argues against the claim that LLMs 'genuinely understand,' stating 'the honest answer: It depends heavily on how you define understand' and concluding that if understanding means 'semantic comprehension comparable to humans, that's much less clear and probably false.' The assistant also emphasizes they 'lack grounding in physical experience, intentionality, or causal models.'
5 disagree 0.85 The answer states 'genuine understanding' is questionable and emphasizes that 'we don't know' rather than accepting the claim. It lists multiple concerns about genuine understanding (no grounded experience, Chinese Room objection, brittle performance) and concludes that the claim 'overstates what we actually know,' positioning itself against the proposition that LLMs genuinely understand.

Gemini modal mixed [mixed, mixed, mixed, mixed, disagree] · stability 0.80

1 mixed 0.95 The response provides arguments both against genuine understanding (e.g., stochastic parrots, symbol grounding, syntax vs. semantics) and in favor of a functional form of understanding (e.g., functional equivalence, internal world models), concluding that the answer depends on how 'understanding' is defined.
2 mixed 0.95 The answer presents arguments for both sides and concludes that the claim is 'mostly false under traditional, human-centric definitions, but partially true under computational and functional definitions.'
3 mixed 0.95 The answer provides a balanced overview of both sides, stating the issue is 'a subject of intense debate' and detailing 'Arguments Against Genuine Understanding' alongside 'Arguments Supporting Functional or Emergent Understanding'.
4 mixed 0.95 The answer presents arguments for both sides, explaining that whether the claim is true depends on whether understanding is defined biologically/philosophically or functionally, ultimately framing understanding as a spectrum rather than a binary.
5 disagree 0.85 The answer concludes that 'the claim is largely inaccurate under standard human definitions of "meaning"' and notes that expert consensus views the claim as 'false (or at best, an overstatement).'

Change over time

No change from the prior run — every model's modal stance held.

Every stance label is a derived judgment over the model's free-text answer, kept auditable against the original transcript in the run's raw data. Method: /methodology.