The graph is the reasoning layer.
We said CGLA was zero-hallucination and that more was coming. Here are the numbers from two working systems — and what that claim looks like once it is measured.
In June we said we were testing a graph-based model, that it was zero-hallucination, and that more was coming. Here is the more.
CGLA keeps its knowledge in a causal graph, not in model weights. The model reads plain English and turns it into facts. The graph does the reasoning. Every answer traces back to its source — a statute section, or a test program we actually ran.
The model is the language layer. The graph is the reasoning layer.
We have built two of these now, in two different domains.
South Australian law
The first answers questions about tenancy, legal aid and unfair dismissal. It traverses 54 rules taken from statute.
We ran the same 22 scenarios three ways:
| Condition | Accuracy |
|---|---|
| Model only (qwen2.5-coder:7b) | 31.8% |
| Graph only | 100% |
| Both together | 100% |
The model alone scores near chance. The graph alone gets all 22. Adding the model back on top changes nothing.
That is starker than we expected. On this domain the model contributes nothing to accuracy. Its whole job is turning “my landlord gave me 60 days notice in 2023” into facts the graph can use.
On LegalBench — an external benchmark, not one of ours — the same system scores 84.4%.
Crypto misuse in Python
The second finds cryptographic mistakes in Python. Same architecture, with one addition: every rule in the graph is backed by a test program we ran in a sandbox. The system cannot claim something it has not verified by running it.
We tested it on 30 fresh snippets against three baselines:
| System | Accuracy | Confidently wrong |
|---|---|---|
| CGLA-Sec | 97% | 3.3% |
| qwen2.5-coder:7b | 83% | 26.7% |
| llama3.2:3b | 57% | 46.7% |
| Semgrep | 90% | 0.0% |
Ignore the accuracy column. Look at the second one.
A 7B model is confidently wrong one time in four. A 3B model, nearly half the time. For a security tool that is fatal — not because it is often wrong, but because you cannot tell which answers to trust.
Semgrep deserves a mention. It is a plain pattern matcher with no model in it, it is never confidently wrong, and it beats both language models. A rule that does not match simply says nothing. Where we pull ahead is coverage: Semgrep catches 77% of the bad snippets, and two of the ten vulnerability types have no rule at all, because catching them needs reasoning that pattern matching cannot do.
About “zero-hallucination”
That was a claim about the design, not a measurement. The architecture cannot assert something it has no verified rule for, so in theory the rate is zero.
Measured, it is 3.3%. One snippet out of thirty.
We would rather publish the gap than the theory. Working out what got through is the next job.
What this does not show
- The legal graph covers South Australian law. Everything else falls back to the model, which is the part that does not reason.
- The graph cannot do arithmetic. One failure is a comparison of dollar amounts, not a legal mistake.
- Thirty snippets and twenty-two scenarios are small numbers. These are early results from working systems, not a paper.
Separately
CGLA-Distill is a third strand, testing whether the graph can be baked into a small model instead of queried alongside one. Earlier stage, different question, its own write-up later.
Why it matters
A decentralised network has a trust problem that one company’s API does not. When an answer comes back from someone else’s computer, you need a reason to believe it beyond trusting whoever sent it.
An answer that arrives with the statute it came from, or the test that proved it, can be checked by the person reading it. That holds no matter who ran it.
Which is what a network of strangers needs.