Independent AI evaluation, delivered under your contract.
Who it's for
- Primes delivering AI programs to government agencies
- Internal-audit co-source firms that need an AI system tested for a client
- Law firms advising a client on an AI system or an AI vendor
- Systems integrators whose client wants the testing done by someone other than the builder
What the workstream delivers
- Tests of one AI system for one use case, on the client's own questions, in the client's environment
- Thresholds agreed before testing, with false answers and false refusals reported separately
- Findings rated to the client's risk matrix, with an explicit list of failures
- A reproducible evidence bundle, with a replay script that runs in the client's own subscription
- Harness handover and training for the client's risk and data teams
How it fits under your contract
- The evaluation runs as a workstream under your existing contract, scoped to one system and one use case.
- For a government client, week 0 runs approvals in parallel, and testing starts on public or synthetic data.
Independence policy
- We never issue an independent evaluation of a system we built, configured or advised on for the same client.
- We take no resale margin or referral fees from vendors we evaluate, and we disclose any vendor relationship.
- Acceptance tests on systems we build are labelled "acceptance test (not independent)".
- Our sample report on our own demo system is always captioned as not independent.
- We stay vendor-neutral: any vendor technical certification is disclosed, and we don't resell under a vendor partner tier.
Method and harness
Every evaluation follows one written method: a sample-size rationale with confidence intervals, a ground-truth method with inter-rater agreement, calibration of any LLM judge against human labels, thresholds agreed before testing, false answers and false refusals reported separately, and an explicit list of failures.
Questions
What kind of report does the client get?
Findings rated to the client's risk matrix, with an explicit list of failures and the evidence behind each one. It is not an assurance opinion, and not an ISAE 3000 or ISRE engagement (ASAE 3000 or ASRE in Australia).
Do you do adversarial testing or red-teaming?
No, never in-house. Where a system needs adversarial testing, it's done by a CREST-accredited tester the client engages. Our Agentic AI Control Evaluation reviews identity, permissions, approval gates and logging; it doesn't attack the system.
Where does the testing run?
In the client's environment, on the client's own questions. The replay script runs in the client's own subscription, so the tests can be re-run there after we finish.
How long does a workstream take?
3–5 weeks for one system and one use case, from agreeing the thresholds to handing over the harness.
Can the client's team re-run the tests without us?
Yes. The harness and question set are handed over, with a training session for the client's data, risk or second-line team.