Independence

Two of those buyers sell the models being tested. That is not an accusation of bad faith — it is a structural fact, and a buyer who needs a verdict that did not come from a supplier now has fewer places to get one.

• Ownership. A one-person practice. No investors, no parent company, no model vendor holds any stake
• Funding. Self-funded. No grant, no vendor programme, no credits from a model provider
• Referral fees. None taken, in either direction. We are not paid to recommend any tool, model or platform
• Implementation. We do not build agents. There is no version of this where we audit our own work, and no upsell that a finding could be pointed toward
• Payment. Never tied to the number of findings — no bonus, no success fee. Paying per finding produces findings
• Licence. Harness, graders and reproduction gate are Apache 2.0 and public

Independence does not mean using nothing. It means the things we depend on are named, and the dependency is bounded.

• Inspect AI — the eval framework, from the UK AI Safety Institute, MIT licensed. Our adapters, graders and reporting sit on top of it
• A frontier model as the system under test, whichever the client uses. We do not select it and we have no preference to defend
• A second model as judge, only where a rule needs judgement — roughly 2% of cases — and only after it agrees with human labels at 96.7% (Cohen's κ 0.95). An uncalibrated judge's output is never published

We chose Inspect AI over promptfoo in August 2026, before writing a line of the harness, because promptfoo had been acquired by OpenAI five months earlier. The tool is good. It is simply not the right foundation for a practice whose entire product is a verdict that does not come from a model vendor.

• Re-run our filter. The reproduction gate is evals/reproduce.py. Every audit hands over the raw run artifacts, so you can apply your own filter to our data
• Read what we got wrong. The finding register carries a correction: we published a mechanism for F-3 that the artifact did not support, and the correction is still there rather than edited away
• Read the negative result. Our own taxonomy predicted that multi-turn conversations degrade accuracy. We measured it, found no degradation, and published that instead of the finding we expected
• Read what we did not measure. 44 limitations are documented, each with the direction it biases the result — whether it hides failures or invents them

Any of the following, and this page changes on the same day the arrangement starts — not when someone asks:

• Taking investment, credits or a grant from a model vendor or an eval tooling company
• Taking a referral fee for recommending any tool, model or platform
• Doing implementation work for a client we also audit
• Any payment arrangement that varies with the number of findings

If you find this page out of date, that is itself a finding. Tell us and it gets corrected in public, the way the F-3 correction was.