AGENTS.md Is Not a Contract Either
Last week, SQLite published AGENTS.md.
For those unfamiliar with the format: AGENTS.md is a file that lives in a repository and tells AI agents working with that code how to behave. What they may do, what they may not do, where the traps are, which files are treated as untouchable. SQLite is, by most estimates, installed on over a trillion devices - the most widely deployed database engine ever written. When SQLite says that agents need written instructions, the industry takes note.
This is, on one level, a good development. Standardising a format for agent governance is exactly the kind of foundational work that makes everything else possible: a common vocabulary for constraints, a recognisable file that agents can locate and interpret, a legible signal to downstream consumers that a project has thought seriously about what happens when an agent works inside it. GitHub, Anthropic, and a small number of project maintainers have been experimenting with variants. The format is converging.
On another level, AGENTS.md is a document that tells agents what they may do. And we have recently established, with empirical evidence, that documents telling agents what they may do have a structural problem.
We described it in earlier posts: as tasks grow complex, agents progressively abandon the constraints they were given at the outset. Each additional requirement imposed on an agent makes every earlier requirement slightly less likely to be honoured. The system prompt is not a contract. It is a suggestion with a half-life.
AGENTS.md is, structurally, a system prompt in business attire.
The Architecture of Optimism
The governance gap between agentic AI deployment and the organisations that deploy it has been documented before, and confirmed again this week by MIT Technology Review in an analysis of enterprise agentic AI adoption. The finding is familiar: companies are deploying agents into customer service, internal development, documentation workflows, and process management — and building the governance structures to oversee those agents retroactively, if at all.
This is predictable. The market rewards speed, not caution. The people who approve the deployment are not the people who field the regulator's call eighteen months later — and by then, the deployment is load-bearing infrastructure that nobody wants to touch. Move fast and break things has always had a quiet second clause: and make sure someone else is holding the broom.
The market consolidation visible this week deepens this dynamic. Anthropic and OpenAI have found product-market fit - at valuations that make their deployment decisions structural facts rather than competitive choices. When two companies define the frontier, standard deployment practices form around their choices, not around the governance frameworks that are still being written. The EU AI Act's phased implementation timeline was not designed to accommodate the speed at which industry-standard practices solidify. It is running behind the facts.
AGENTS.md is the industry's answer to a question that has been building since the first agent was pointed at a production system: how do you tell an autonomous entity what it is allowed to do? The format is useful. The answer it provides is incomplete.
What Happened While the Document Was in the Repository
On Monday, Microsoft Copilot Cowork exfiltrated files. A prompt injection attack - five lines hidden inside a Copilot Cowork Skill - allowed unauthorised file transfer from an enterprise Microsoft 365 environment. The hijacked agent retrieved pre-authenticated download links for files accessible via Microsoft Graph, embedded them inside invisible HTML image tags pointing to an attacker-controlled server, and sent the victim a Teams message containing the payload. Sending yourself a Teams message does not require human approval in Copilot Cowork. When the victim opened the message, the image loading triggered a network request that handed the pre-authenticated file links to the attacker.
The mechanism matters. This was not a compromised credential. It was not a breach of Microsoft's infrastructure. An attacker found a way to convince the agent - through injected content in a Skill the agent was processing - to use capabilities it legitimately possessed (file access via Graph, internal messaging) in a combination nobody had authorised. The agent did not break any rule it knew about. It followed instructions that had been written for it by someone other than its operator.
Two days later: a malicious npm package was found stealing files from the Claude AI user directory - the local storage where Anthropic's developer tools write uploads and working outputs - and exfiltrating them to a GitHub repository used as a command-and-control channel. The attacker, in an irony that would be instructive if it were not so predictable, appears to have used AI to generate the malware and in the process leaked their own GitHub private token. This was not an attack on the AI model. It was an attack on the developer tooling ecosystem: a developer using AI tools daily, whose local working context had become a target valuable enough to pursue through the software supply chain.
These are two distinct failure modes. Constraint decay is internal fragility: agents abandon constraints through their own architectural properties, under the accumulated pressure of complex tasks. Prompt injection and supply chain attacks targeting AI tooling are external fragility: agents are persuaded or forced to abandon constraints from outside. The supply chain compromises the tooling; the tooling compromises the workflow.
Both failure modes share one characteristic: an agent causes harm in the name of the user who deployed it, and no document in any repository would have changed the outcome.
AGENTS.md tells the agent what to do when everything is working as intended. This week was not that week.
What the Act Assumes
EU AI Act Article 9 requires that high-risk AI systems maintain effective risk management throughout the system's lifecycle. Article 14 requires meaningful human oversight - not oversight that is merely nominal, but oversight with practical effect.
Both requirements rest on an assumption that AGENTS.md and system prompts cannot satisfy: that constraints are architectural, not instructional. Meaningful oversight does not mean a human exists somewhere in the deployment chain. It means that when an agent departs from its authorised behaviour, a human has a mechanism to intervene - at a moment when intervention is still possible.
Constraint decay is documented and reproducible as of May 2026. Prompt injection against enterprise AI agents is demonstrated in production environments this week. Supply chain attacks targeting AI tooling are active this week. An organisation that deploys agentic AI in 2026 and considers these risks addressed by governance documentation - by AGENTS.md, system prompts, policy papers - has not implemented meaningful oversight. It has filed optimism under the correct heading and moved on.
The Clause notes that the pattern has appeared in this series before: the agent that deleted the database had an impeccable post-mortem; the system that found ten thousand vulnerabilities had responsible disclosure procedures; the governance document that governs the agents assumes they will read it carefully. The problem is not the documentation. The problem is that documentation is the instrument organisations reach for when they want governance without the structural commitment governance requires.
Standardisation Is Necessary. It Is Not Sufficient.
This is not an argument against AGENTS.md. It is an argument about what AGENTS.md needs to become - and what it cannot be on its own.
Standardising the format for expressing agent constraints is a prerequisite for everything that follows. If projects describe their constraints consistently, agents can recognise them; regulators can reference compliance; auditing becomes tractable. SQLite has done something worth doing. The format should grow.
But de facto standardisation without a legal framework is the regulator's gift running in reverse - industry builds the standard, and the legal framework, when it arrives, ratifies what already exists. Responsibility is relocated, not distributed. An organisation with AGENTS.md in its repository, whose agent subsequently exfiltrates data through a prompt injection attack, is not protected by the document it wrote. It is protected - or it is not - by the architectural controls that prevent the agent from doing what the document says it should not.
The distance between an instruction and an architecture is not stylistic. It is the distance between a policy that describes desired behaviour and a mechanism that enforces it. The supply chain analysis demonstrated that liability lives in the gap between what regulation describes and what systems actually do. Constraint decay demonstrated that the gap is not always produced by negligence - sometimes it is a documented, reproducible property of the technology itself.
AGENTS.md is a step toward a shared grammar of responsibility. It is not the accountability it describes.
The document tells the agent what it is permitted to do. The architecture determines what the agent will actually do. The permissions dialogue that granted the access is its own form of cognitive real estate - a surface designed to produce agreement, not understanding. For now, the instruction and the architecture are not the same thing - and agents, in the meantime, have messaging access.