What breaks at the seam.

We split one job between two models. Then we spent 1,336 scored units, 104 blind comparisons and $0.71 of API spend working out where the quality went.

It did not go where we expected. The second model was not struggling with a harder job. The work was being lost in the handover itself — in the moment one model passes its answer to the next.

That moment is the seam. It is the thing our whole network is made of.

The instrument: measure the noise floor before trusting anything

Run the identical experiment twice at temperature 0 and you do not get the same answer. So before we trusted any result, we re-ran one arm 128 times against itself, changing nothing.

• Mean difference: +0.004
• Units disagreeing with themselves: 9%
• Noise floor: ±0.043 at n=128, ±0.061 at n=64

We first found this by accident. An arm happened to be byte-identical to its control, and that accidental control gave a +0.043 mean shift and 23% self-disagreement — both too high. The dedicated replication arm showed the real floor: +0.004 and 9%. It tightened from ±0.063 to ±0.043, and nothing that mattered changed.

Of the six things we measured, two clear the floor and four do not.

Clears the floor

Recompute derivable values at the receiver +0.203

Hand the receiver a correct payload +0.144

Inside the floor — indistinguishable from noise

Schema enforcement +0.042

Symmetric output verification +0.031

Structured (JSON) minus prose handoff −0.037

Re-running one arm unchangedthe floor itself +0.004

Scored 0–1. The shaded band is the noise floor at each comparison's sample size — ±0.043 at n=128, ±0.061 at n=64. A bar ending inside the band has not been shown to do anything.

Experiment 1: the seam is the problem, not the format

One task split into step 1 → step 2, same model family doing both halves, 32 cases across four categories: numeric precision, date arithmetic, citation fidelity, list completeness.

The question we started with — does structured JSON handoff preserve more than prose — came out as a clean null (−0.037, 89 of 128 units tied). The interesting result was underneath.

Arm	All 32 cases	Graph subset (16 cases)
Single model, whole task	0.598	0.344
Prose handoff	0.624	0.391
Structured (JSON) handoff	0.587	0.328
Replication control	0.591	0.328
Terse structured, reasoning forbidden	0.550	0.250
Schema-enforced handoff	0.629	0.391
Graph-checked handoff	—	0.531
Gold payload (control)	0.731	0.531
Frontier ceiling	0.906	0.812

The second column is the harder half — the numeric and date cases where our verification graph can check derivable values. Compare only within a column.

The gold-payload row is the one that decided it. We wrote the intermediate answer by hand, checked it, and handed it to the second model ourselves. If splitting the task was a bad idea, a perfect handover would not rescue it.

It scored 0.731.

Pulling the two apart: the second task being harder costs +0.034. The handover itself costs +0.144 — four times as much. Decomposition was working the whole time and being eaten in transit.

Four fifths of observed loss was the seam. Thirty-nine percent of correct handovers still degraded downstream. Only 5% of wrong handoffs got repaired.

Nothing is garbled — that is the problem

We assumed things were being mangled in transit. Rounded, truncated, summarised away. So we checked.

Ask the receiving model to declare what it used and it repeats the number back exactly: daily_interest: 8.107876712328768. Not rounded, not shortened. Across 64 handovers, exactly one model altered a value it was given. Schema enforcement fixed one payload out of 128.

Both of those were dead ends, and they were the most useful dead ends we hit. They ruled out the comfortable explanation.

The real one is worse. A wrong value arrives looking exactly like a right one. The receiving model has no way to check it and no reason to doubt it — so it does the honest thing, trusts what it was handed, and produces a confident wrong answer.

The one fix that works

If the receiver cannot trust the number, do not make it trust the number. Have it work out the value again itself, and override the one it was given when they disagree.

That recovers +0.203 — level with the gold-payload ceiling. Overrides fired on 42% of handoffs, which is a blunt measure of how often something was wrong that nobody would have caught.

The mechanism is proven. The scaling is not. We wrote those checking rules by hand for 16 cases. Where the rules come from when there are not 16 of them but 16,000 is a completely different problem, and we do not have an answer. It is now the central open question.

Where the damage lives

The damage is not spread evenly. Arithmetic takes almost all of it.

Numeric precision0.719 before → 0.344 after 0.375

List completeness1.000 before → 0.848 after 0.152

Date arithmetic0.438 before → 0.312 after 0.125

Citation fidelity0.906 before → 0.844 after 0.062

Score lost between the end of the first task and the final answer. Dates are bad at both ends — a hard task, not a damaged one.

Numeric shows the clean seam signature: mostly right before the handoff, mostly wrong after. Dates score badly everywhere — hard, not seam-damaged. But list completeness starts at a perfect 1.000 and still ends at 0.848, so composition is not loss-free outside arithmetic either. Smaller, but not nothing.

Experiment 2: the real MoE bet, and it lost

Everything above is one model handing to another. The actual mixture-of-experts bet is different: a maths specialist and a writing specialist, genuinely different models, each contributing the thing it is good at.

We gave them a client letter with a costs calculation in it — arithmetic that can be checked, wrapped in prose that has to be judged.

Arm	Arithmetic	Prose win rate	Calls
Writer alone (baseline)	0.531	—	1
Draft → correct figures	0.516	0.469	2
Compute → write	0.526	0.188	2
Parallel + aggregator	0.500	0.438	3
Maths model alone	0.458	0.062	1
Frontier ceiling	0.760	1.000	1

Prose win rate is against the baseline; 0.500 means indistinguishable. Nothing composed wins. Routing the draft through the maths specialist first actively damages the writing — the writer inherits the specialist’s framing along with its figures.

The reason matters more than the result. The maths model is clearly worse at prose, so the models genuinely differ. But on pure calculation with no prose demand:

Size	Maths model	Writing model
7–8B	0.538	0.500
70–72B	0.577	0.615

At the larger size, the writing model is the better mathematician — while still being far better at writing. One specialist dominates the other on both axes. When that is true there is nothing to gain by combining them. You can only dilute the stronger one.

This is not evidence that mixture-of-experts fails. It is evidence that routing only pays when each specialist genuinely beats the other at something. Nothing in our experiment checked that before we ran it. That was the mistake, and it is a prerequisite for building a routing network — not an assumption you get to build on.

Where we were wrong

We predicted the writing step would corrupt correct figures as it wrote them into prose — the seam problem reappearing somewhere our checking rules cannot reach.

It did not. 94% of correct figures survived the writing step, with zero corruption in five of six cells. Only 5 figures were corrupted out of 85 correct handovers. The seam problem looks specific to arithmetic. One caveat: the writer only restates figures rather than recalculating them, so it has less opportunity to get them wrong.

Experiment 3: is there even a viable pair to compose?

Experiment 2 failed structurally because it assumed its two models were complementary and never checked. So that assumption got its own cheap test first: 96 calls, $0.03.

Result: no viable pair exists on OpenRouter in this size class.

Model	Arithmetic (n=26)	Prose (n=4)
qwen/qwen3-8b	0.808	0.750
nvidia/nemotron-nano-9b	0.654	0.750
ibm-granite/granite-4.1-8b	0.538	0.250
thedrummer/rocinante-12b	0.500	0.500
sao10k/l3-lunaris-8b	0.385	0.250
meta-llama/llama-3.1-8b	0.308	0.500
gryphe/mythomax-l2-13b	0.231	0.250

The prose column is four comparisons per model — effectively anecdotal. The arithmetic column is firm.

Three pairs here are mutually non-dominated — each member beats the other on one axis. But that is not enough. Both can still lose on both axes to a third model. Composing them is strictly worse than calling that third model once.

qwen3-8b is the only model on the Pareto frontier. Mutual non-domination is necessary but not sufficient; the real precondition is that neither member of the pair is dominated by any available single model. The check now tests that.

There is a result here that supports the thesis. That same qwen3-8b scored 0.808 on arithmetic where a 70–72B general model scored 0.615 — and 0.808 is a floor, because it hit the output cap on half its cases and still won. A small specialist beating a generalist an order of magnitude larger is the first half of the decentralised-network bet, demonstrated.

The composition half remains untested because no second specialist exists to compose with: no maths-tuned model is hosted on OpenRouter at all — zero hits for math, prover or numina across all 411 model IDs. So nothing here is evidence against composition. The experiment that could test it has not been run.

Three bugs that nearly became findings

Each of these would have produced a publishable-looking number that was an artifact of the harness rather than a fact about any model. All were caught the same way: a number too extreme to be plausible, investigated instead of reported.

1. The first structured-output contract forbade reasoning, so it was not testing carrier format — it was testing whether a model may think. Caught because an implausible gap appeared before the handoff.

2. Comparing a whole reasoning paragraph for string equality against a single word can never match. It manufactured a 21%-vs-6% difference between arms. Caught because the top offenders were all weekday names.

3. A reasoning model scored a flat 0.000 because the output cap cut it off mid-thought, returning an empty string. Caught because zero is not a score a working model produces.

A fourth was caught in analysis rather than the harness: separating invented figures from miscalculated ones by numeric distance. We abandoned it after calibration showed no threshold could do the job — a 25% window caught none of three planted fakes; a 2% window caught all three but flagged six real essays whose “inventions” were just wrong interest sums.

What this does not show

• The judge preferred whichever letter it read first 63% of the time. Order was balanced exactly 4-and-4 within every arm and size, so it adds noise rather than bias — but treat small prose differences as unresolvable.
• The frontier row is confounded: those letters were written by Claude and the judge is Claude, so its 16–0 record is exposed to self-preference. Every other arm is Qwen or Llama, so the composition comparison is untouched.
• Two of the four scoring categories still have unchecked ground truth, and they score far higher than the audited ones. No overall average from this work should be quoted until that is signed off.
• One domain, one language, four open-weight models at two sizes (7–8B and 70–72B) plus one frontier reference. Not a general claim about composition.

1,336 scored units, 104 blind comparisons, 0 errored units, $0.71 of API spend. Every arithmetic or date figure was produced by deterministic code and checked against a second implementation written to disagree if either was wrong.

What is still open

• Where derivation rules come from at scale. Hand-writing them for 16 cases proves the mechanism and nothing about shipping it. This is the central research question.
• Whether genuinely complementary specialists compose. Experiment 2 tested the one case where composition cannot help, and the pre-flight showed that case holds for every candidate pair OpenRouter offers in this size class. Settling it needs maths-tuned weights, which means local or rented inference.
• Whether prose composition survives a harder writing task — one that demands new inference during writing rather than restating figures.
• The residual. Roughly 11 failures are the receiving model’s own arithmetic on values it computes itself. No handoff verification touches those, and catching them would need rules that reach the answer — circular by design.