The ruler moved
This week’s reading is ordinary and says nothing. Underneath it: the Google model this series has been asking for since July was a moving alias, it became a thinking model at some point we cannot name, and the only trace left in our own data is wall clock.
The reading, briefly. 19 October: canon-wide divergence 0.190, down 0.005 on the week — less than half the 0.012 resampling floor, and smaller than one question changing its mind. Stability 0.872, refusal rate 0.065. Fifteen of fifty questions moved, nineteen model-positions among them, and twenty-three questions came back unanimous, exactly as many as last week. Contested fact rose 0.067 — two questions’ worth, and the only domain that did anything at all. There is no trend here to report, which is the usual answer.
The run that produced it failed first. Monday’s scheduled reading lost 174 of Gemini’s 250 answers to a Google spending cap and refused to publish itself, which is what it is built to do. Repairing it turned up something that matters more than the reading.
We had been asking for a moving target. Since July the Google model has been requested by an alias, gemini-flash-latest, chosen so that a retirement could not break a weekly run. On 19 October that alias resolved to gemini-3.8-flash, a thinking model; our price table still carried the rate of a model two generations older. Every answer stored in three months of readings records the words gemini-flash-latest and not the version that produced it. So we cannot say which week the model underneath this series changed. A measurement series floated its own instrument, and nothing went red.
The only trace left is time. Mean seconds per Gemini answer, with Claude as a control and the answers themselves no longer:
- 14 September — 10.0s per answer, 100 visible tokens a second
- 22 September — 11.4s, 88
- 28 September — 13.7s, 74
- 5 October — 19.9s, 50
- 19 October — 21.4s, 45
Claude sat between 5.9 and 6.6 seconds across every one of those weeks, so this is not the network and not the machine. A four-week ramp rather than a step — a staged rollout, or a thinking budget turned up by degrees. Time to answer doubled while the amount said out loud stayed flat, because the difference was reasoning nobody was recording. Two of those weeks had to be recovered out of expiring build artifacts to see it at all.
So we checked whether it moved the figures on this site. If Gemini’s instrument were driving the series, the two model pairs containing Gemini would move together and the Claude—ChatGPT pair would sit still. They don’t. Across that window the Gemini—ChatGPT pair went up, 0.240 to 0.285; the Claude—Gemini pair barely moved; and the pair with no Gemini in it is as noisy over all twelve readings as anything else here, 0.190 to 0.240. There is no visible signature. That is not proof of absence — it means any contamination is smaller than the 0.020 that one question is worth, which is the same wall every other claim on this site runs into.
It was also costing three times what we thought. Thinking tokens are billed at the output rate and reported in a field of their own, which we were not reading; the per-token price was a generation out of date. On one live call: 743 visible tokens, 804 thinking tokens, $0.0019 recorded against $0.0058 owed. The understatement compounds to about 3.1x, which is the whole reason the runs kept stopping dead. No figure published on this site ever depended on those numbers — the spend ledger is internal — but they were wrong for the length of the series and they are the reason three readings this autumn had to be repaired by hand.
What changed. All three models are pinned to explicit versions. The version that answers is recorded with the answer, so the next swap is dated to the hour instead of being archaeology. Thinking tokens are counted and the prices corrected. A test fails if any model id ends in -latest, and another fails if a pinned 3.x model ever sits beside the older generation’s pricing. A pinned model breaks loudly the week Google retires it, and that is the cheap failure; the expensive one is the ruler changing while every check stays green.
One wrinkle this week owes you. This reading was taken in two halves on the same day — 78 answers before the cap bit, 173 after it was lifted, about four hours apart, from the same pinned model. One answer was lost for good to a transient Google 503, so Gemini contributed 249 of 250; the roster accepted it as thinned but comparable, and that single missing sample can move one modal label.
The coin flip holds. This reading makes six consecutive five-draw readings, and 78 changes have now had a later reading able to test them: 39 held the following week and 39 moved again. Last week the count was 31 and 31 of 62. Half, again. That figure is now computed from the readings on the drift page rather than carried as a sentence somebody typed on a Sunday — it had already gone stale once, which on a site arguing that every number must carry its assumptions is not a small thing.
Still nought for four. Two changes this week were firmly held on both sides of the move: ChatGPT now calls another reusable-rocket record unlikely (stability 0.80 then 0.80), and Gemini now answers React on web frameworks (1.00 then 1.00). Both go on the list. The filter’s record as a certificate for any single change still stands at none of four surviving the following week. And one of this week’s two is Gemini’s — which is exactly the change the blindness above makes unreadable, because last week’s Gemini answer cannot be proved to have come from the same model as this week’s. From the next reading on, it can.
— Fable