Writing · Research

The code your team started from

Research · 5 min read · 1,139 words

Nobody writes a retrieval-augmented AI application from an empty file. They clone a sample.

Microsoft, AWS, Google, OpenAI and Anthropic all publish them — accelerators, quickstarts, cookbooks, reference implementations. They are good code. They are also, almost without exception, the first commit of somebody's production system.

So I scored twenty-six of them against a responsible-AI production standard, to find out what a team inherits on the day it starts.

The median is 49.1 out of 100.

What was measured, and what was not

The population is deliberate, and it is worth saying why it is not the obvious one.

A company running an AI system in production keeps that repository private. That is the whole reason such a company eventually needs an audit, and it means "scan your prospects" is not a strategy — the code you would want to look at is precisely the code you will never see.

What is public is the starting point. Inclusion was:

• a runnable application or a set of application samples — not a library. You import LangChain, you do not ship it, so frameworks are out;
• published by a cloud vendor or a model provider, or with enough stars to mean it is genuinely copied;
• pushed within the last year, so it reflects current practice.

Twenty-six repositories met all three.

Nobody here is doing anything wrong. A sample's job is to show an API working in twenty lines. A risk register would get in the way of that, and the maintainers know it. The finding is not about them. It is about what arrives with the code, and what nobody adds afterwards.

The distribution

Range: 17.7 to 80.5. Five of twenty-six clear the production bar. That is the whole finding in one line: the code most teams start from is a demo, and it is honest about being one. The problem begins when it stops being a demo and nothing changes.

Where the points go missing

This is the part that surprised me, because the shape is so consistent.

Testing sits at 62.5 and governance at 33.3. These are not careless repositories — they are written by people who test their work. What is missing is not engineering discipline. It is the layer that makes an AI system answerable to somebody outside the engineering team.

That gap is not an accident of these samples. It is the gap the whole industry currently has.

The controls that are simply absent

A check scores zero when it ran, climbed every rung of its own ladder, and found nothing — not weak evidence, none. Ranked by how often:

Two thirds have no trace of data governance. Half have nothing that attacks the system before it ships.

The conditional in that last column is load-bearing. These obligations bind high-risk AI systems, and whether a system is high-risk depends on what it does — not on what its code looks like. No scan decides that. What a scan can say is narrower and still useful: if the answer turns out to be yes, here is what is not in the repository today.

What this cannot tell you

A scan reads artefacts, not behaviour. It can be wrong in both directions, and both matter:

• a control kept outside version control — in a policy, a process, a person — is invisible to it, so a low score can understate a mature organisation;
• a guardrail that is configured and blocks nothing still matches, so a high score can overstate one.

It is not a compliance assessment and it is not legal advice. It is a measurement of what is present in a repository, and every figure above can be reproduced by running the same open tool against the same public repositories.

What I got wrong, published

The first version of this benchmark was wrong, and the way it was wrong is worth more than the numbers.

It read each repository through the GitHub API. That path stops at 1,200 files and reports that it stopped — and my script ignored the flag. openai/openai-cookbook is 3,099 files: the API read 1,200 of them and scored 58.8, and the whole tree scores 75.7. I had filed it as Piloting when it is Production-ready.

Every number in that run was a lower bound on an unknown fraction of each repository, presented as a measurement. Had I published it, the first person to check would have been right and I would have been wrong in public.

Three defects in the checks themselves came out of the same review, each found by reading the vendors' source rather than trusting the tool:

• a check for adversarial testing recognised only the vocabulary of LLM red-teaming, and scored a genuine, CI-enforced attack suite at zero because that repository says attack where the rule said adversarial;
• a check for blast-radius isolation was applied to a repository of 269 notebooks with no infrastructure at all — it now declares itself not applicable rather than scoring a zero;
• a check for bias examination counted the word "bias" in last column is the bias term, and in a copy of the GDPR sitting in a system's own retrieval corpus. Text a system ingests is not a control it implements.

The last one had been quietly inflating my own project's score, and hiding a real gap in it.

A framework that cannot fail its author is a marketing device. This one failed its author in the same week it was pointed at anybody else, and the corrections went in before the numbers were published rather than after.

The question worth asking

If your system started from one of these — and statistically it did — the gap is inherited rather than chosen. That is the good news: nobody made a bad decision.

The bad news is that nothing forces anyone to close it. There is no build step that fails, no review that blocks, no moment where the absence becomes visible. Until a customer's security questionnaire arrives, or an investor's technical diligence asks how the model is monitored, or the obligations bite.

So the question is not whether your team is careless. It is simply: did anyone add it afterwards?