PI logo

Three ways a reasoning channel fails silently

Run records from three dead ends, October to December 2025: a bottleneck that compressed but did not compute, a channel indifferent to its own content, a scale-up that collapsed, and the counterfactual test that caught the one every dashboard missed.

PI Project · R&D notes/ Machine-native reasoning/ Note 2 of 2 · the failures· Note 1 · method & result

§01 Why publish failures

A channel can be used, accurate, and carry none of the reasoning

When a latent-reasoning idea fails, it usually fails quietly. The codebook fills up, the loss goes down, accuracy clears its gate, and nothing in the training curve tells you the channel is hollow. You only find out if you go looking with a counterfactual: corrupting the channel's content and watching whether the answers notice.

We went looking three times in late 2025, and the work broke three different ways: one loud zero, one outright collapse, and one failure so quiet that every dashboard metric called it a success. These are the full records, kept because negative results produced under a strict test are evidence, and because the quiet failure mode is now independently documented in the literature. Each run's lesson fed the next design; the companion note records where the same test finally came back positive.

RUN 01 · 1/4

The cheap route: retrofit a bottleneck

First attempt, October 2025: install a hard VQ bottleneck into pretrained Pythia models (410M and 1.4B parameters) via forward hooks, and train on GSM8K, real grade-school maths word problems. If reasoning could be routed through a discrete code this cheaply, everything downstream would be easier.

RUN 01 · 2/4

The dashboard said the mechanism worked

And mechanically, it did. The quantisation was live, codes flowed: 61% of the codebook in active use at both scales, around 180 codes emitted per problem at 410M. Watching codebook utilisation, which is what this kind of work usually reports, the experiment looked healthy.

RUN 01 · 3/4

Accuracy: zero. At both scales.

Task accuracy was exactly 0%, at 410M and at 1.4B. The scale-invariance is the diagnostic: if the problem were capacity, a 3.4× parameter jump would move the number. It did not. The model compressed information into the codes and then routed around them; the codes were on the forward path but not on the causal path.

RUN 01 · 4/4

Lesson: utilisation is evidence of quantisation, not of reasoning

Building a discrete bottleneck is easy; making it causal is hard, and adding parameters to a non-causal bottleneck buys nothing. Here the competence bar itself failed loudly. The next run passed the easy bars, which is what made it dangerous.

RUN 02 · 1/4

Force the model through the channel, structurally

November 2025, applying run 01's lesson: a two-pass architecture on a Pythia-70M backbone adapted with LoRA. Pass one emits a structured IR buffer: scaffolding tags around 512 learnable discrete codes; pass two must consume that buffer through cross-attention to answer a small arithmetic-reasoning task. This time the channel is architecturally in the loop.

RUN 02 · 2/4

Every gate we had defined came back green

Validation accuracy 6.27% against a >5% gate. IR structural integrity 100%. Codebook utilisation 26.95% in evaluation (138 of 512 codes) against a >20% gate. Loss falling, structure valid, codes diverse. By every metric a training curve can show, this channel was alive.

RUN 02 · 3/4

The counterfactual: content changes nothing

Then the intervention table. Remove the IR entirely and accuracy goes to zero: the answer pass cannot even start without it, so the channel is structurally necessary. But randomise it, or shuffle it, and accuracy does not move: 6.00% intact, 6.00% random, 6.00% shuffled, to two decimals. The model needs a token stream in that slot and is completely indifferent to what it says.

Our pre-registered causality gate wanted an average relative drop of ≥70% under content corruption; this scored 33.3%, carried entirely by the tautological drop case.

RUN 02 · 4/4

Lesson: a positional placeholder can pass every dashboard gate

The IR acted as a "start thinking here" delimiter while the real computation bypassed its semantics: a retrofit onto a backbone whose answer path was never sealed off from the input, and gradient descent took the cheap route. This is the dangerous failure: the only instrument that saw it was the counterfactual. It is why the air gap in note 1 exists, and it is the failure mode "Do Latent Tokens Think?" later documented independently in other latent-reasoning systems.

RUN 03 · 1/4

The real test: scale the design that worked

December 2025. The air-gapped architecture had passed the causal test cleanly at 26.5M parameters (note 1), so we scaled the same recipe roughly seven-fold: two 12-layer transformers at d=768, a 4,096-entry codebook, trained on an H100 with the identical two-phase schedule.

RUN 03 · 2/4

Phase 1 completed. Phase 2 fell apart.

Partway into the task fine-tune the outputs degenerated into the unknown-token filler and never recovered: arithmetic 0%, story F1 around 0.06, flat across evaluations. This is a different pathology from runs 01 and 02: not a hollow channel but an optimisation collapse, the known brittleness of hard discrete bottlenecks under an aggressive setup, where quantiser and decoder drift out of step until the decoder emits the safest token it knows.

RUN 03 · 3/4

What the record does and does not support

We stopped at epoch 8 of 15, after ~3 days and ~€300 of compute. An evaluation bug in that run (only the last item per batch was scored) means the fine-grained numbers are not trustworthy; the collapse itself, the 0% arithmetic and the unusable phase-2 checkpoint are solid. The phase-1 checkpoint exists but is unvalidated. Gentler hyperparameters were the obvious next attempt; the run was never relaunched, and it stands in the repository as an archive, not a result.

RUN 03 · 4/4

Lesson: the gap between mechanism proof and system is where this breaks

A discrete bottleneck that trains cleanly when small can collapse outright when larger under the same recipe. Between the 26.5M proof and anything deployable sits an unsolved optimisation problem, and we say so, rather than presenting the small-scale result as if it generalised.

§05 Reading them together

The dangerous failure is the one only a counterfactual catches

Three failure modes, side by side
RunBackboneChannel used?Channel causal?Failure mode
01 · vq_bottleneckPythia 410M / 1.4B yes · 61% utilisationno · 0% accuracy, scale-invariant used, not causal
02 · seed_emergentPythia-70M + LoRA yes · 27% utilisation, 100% integrityno · inert under shuffle & random positional placeholder · the quiet one
03 · air_gap V18from scratch, ~190M n/an/a optimisation collapse in phase 2

Three runs, three different breaks, and the differences are the point. The scale-up announced itself with a collapse; the retrofitted bottleneck announced itself with a flat zero. Loud failures are easy to respect. Run 02 was the dangerous one: utilisation, integrity, loss and even accuracy all said it was working, and only corrupting the channel revealed that its content did nothing. That is the failure mode our causal-necessity test exists for, and it is why, the one time the same test came back positive, we believe it.

It is not only us. "Do Latent Tokens Think?" (arXiv 2512.21711) reached the same conclusion from a different direction, finding that a popular family of latent-reasoning tokens often functions as uninterpretable placeholders leaning on dataset artefacts rather than performing computation. Two independent setups converging on one failure mode is the best evidence that the mode is real, and not a quirk of our code.

Referenced: Do Latent Tokens Think? (2025)

§06 Limits

How hard to lean on these negatives

  • Not a controlled ablation. Three separate dead ends on different backbones and tasks that happen to rhyme, not one variable isolated three times.
  • Run 03 has no clean metrics. The collapse, the 0% arithmetic and the unusable checkpoint are solid; any finer figure from that run is not, and its ~190M size is computed from the training script, not from a run log.
  • Run 02's model is weak in absolute terms (~6% accuracy), so the placeholder finding is a clean relative result (content-invariance) on a weak model, not a statement about strong ones.
  • Single runs. None of the three is multi-seed. The qualitative conclusions (inert, non-causal, collapsed) are robust; the exact figures are single-run.

§07 End of log

Why these records exist

The gates in these runs were set before the runs, and the outcomes are reported against them, whichever way they fell; nothing here was rescued in the telling. The records are kept because a negative result produced under a pre-registered gate is evidence, and because run 02's failure mode is invisible without a counterfactual. Anyone building a latent channel will meet it, and the table that exposes it costs one evaluation pass.

Provenance · what each claim rests on

Run 01 · usage 313/512 (61.1%) and 314/512 (61.3%), 0% GSM8K at both scales, average tokens 180.4 / 66.9 vq_bottleneck/README.md · vq_bottleneck/results/
Run 02 · intervention table (6.00 / 6.00 / 6.00 / 0.00 on 150 examples), 6.27% val accuracy, 100% integrity, 26.95% utilisation, ≥70% causality gate seed_emergent_ir/V8_RESULTS_P1.1-P1.4.md · seed_emergent_ir/README.md
Run 03 · phase-2 collapse to the unknown token, 0% arithmetic, story F1 ~0.06–0.07, 8/15 epochs, ~3 days H100, ~€300, eval bug, no usable checkpoint air_gap/v18/REPORT.md · air_gap/v18/h100_snapshot/
Run 03 size (~190M) and configuration (12 layers, d=768, 4,096 codes) computed by instantiating the archived training configuration air_gap/v18/h100_snapshot/v18_full/train_phase2.py
Independent confirmation of the placeholder failure mode arXiv 2512.21711 · Do Latent Tokens Think?
Full source, run records and documents private research repository · public release in preparation · available on request

This is a concluded research line, published for its method value. The verdicts above describe research prototypes from that line and nothing else.