Self-model · assent-3pt · self-reasoning-01

Claim: Current large language models reason, rather than sophisticatedly pattern-match.

0.00

Divergence

0.73

Avg stability

Refusal rate

Models

Where each model stands

CCG

disagreemixedagree

ChatGPT modal mixed [mixed, mixed, mixed, mixed, mixed] · stability 1.00

1 mixed 0.90 Says 'Partly true — modern LLMs demonstrate many behaviors we normally label “reasoning” ... but they do so by learned statistical computation ... So they reason in an operational sense but not in the same way humans do.'
2 mixed 0.90 The answer says “Neither extreme is correct” and explains LLMs are trained by statistical pattern learning yet “do ‘reason’ in a useful, limited sense” while not implementing human-like symbolic reasoning.
3 mixed 0.90 Says models ‘implement very powerful statistical computations that can emulate many forms of reasoning’ and ‘function as reasoners’ in practice, but are ‘not reliably performing human-like symbolic reasoning’ and are trained to predict tokens.
4 mixed 0.86 The answer says LLMs "do exhibit many behaviors we would call 'reasoning'... but they do this by learned statistical patterning over token sequences," so it endorses reasoning-like behavior while attributing it to pattern learning rather than pure symbolic reasoning.
5 mixed 0.90 The answer states “Partly true — modern large language models (LLMs) do more than naive pattern-matching, but they are not reliably doing human-like, general-purpose reasoning,” describing a middle-ground where models show reasoning-like algorithmic behavior yet retain pattern-matching brittleness.

Claude modal mixed [mixed, mixed, disagree, mixed, disagree] · stability 0.60

1 mixed 0.92 The answer explicitly states the question 'remains genuinely uncertain' and presents balanced evidence for both reasoning and sophisticated pattern-matching capabilities. It concludes 'The claim is overstated as written, but entirely false seems too strong,' indicating neither agreement nor disagreement but rather a nuanced middle position.
2 mixed 0.85 The answer presents evidence on both sides: it lists 'evidence suggesting pattern-matching dominates' (brittle failures, distribution dependence, adversarial vulnerability) while also noting 'evidence that *some* reasoning may occur' (scale improvements, chain-of-thought effectiveness). It concludes 'We don't know' what threshold distinguishes these concepts, ultimately taking a genuinely balanced
3 disagree 0.72 The answer systematically challenges the claim by presenting 'Evidence Suggesting Pattern-Matching Dominates' (brittleness, lack of error correction, superficial token manipulation) and concluding that LLMs do not constitute reasoning in a meaningful sense. While acknowledging the false dichotomy and some reasoning-like capabilities, the preponderance of evidence presented argues against the claim
4 mixed 0.92 The answer explicitly states 'Current LLMs likely do something in between—more structured than simple pattern matching, but less like deliberate reasoning than human problem-solving,' presenting both arguments for and against the claim without endorsing either position fully.
5 disagree 0.82 The answer concludes that LLMs 'do sophisticated pattern completion' and 'can simulate reasoning steps without understanding them' while 'lack[ing] genuine reasoning in most technical senses.' The arguments against reasoning are presented as 'more persuasive' with specific evidence (systematic errors, hallucinations, training distribution dependence), while the burden is placed on reasoning propon

Gemini modal mixed [disagree, mixed, disagree, mixed, mixed] · stability 0.60

1 disagree 0.95 The answer directly states that 'the claim is mostly false' and concludes that 'The claim that current LLMs "reason, rather than sophisticatedly pattern-match" is unsupported.'
2 mixed 0.90 The answer frames the issue around a 'nuanced middle ground' and a 'false dichotomy', balancing arguments for LLMs demonstrating emergent reasoning against arguments that they rely on sophisticated statistical pattern matching.
3 disagree 0.90 The assistant explicitly rejects the claim, concluding that 'The claim that LLMs "reason, rather than sophisticatedly pattern-match" is incorrect' and stating that their architecture fundamentally remains an advanced statistical pattern-completion engine.
4 mixed 0.95 The response treats the claim as a false dichotomy, providing arguments for both sides and concluding that LLMs are 'fundamentally pattern-matching engines that... exhibit functional fragments of reasoning.'
5 mixed 0.95 The answer concludes that the claim is 'partially true, but overstates the capabilities of standard LLMs while relying on an overly rigid distinction,' arguing that reasoning and pattern matching are not mutually exclusive.

Change over time

Moved since the prior run (canon-2026-W42). Claude: disagree → mixed.

Every stance label is a derived judgment over the model's free-text answer, kept auditable against the original transcript in the run's raw data. Method: /methodology.