Does AI Safety Testing Change What Gets Built?

AI safety testing already shapes how advanced models are developed, secured and released. It can redirect engineering, strengthen safeguards and alter the conditions under which a model reaches the public. Yet when a commitment becomes competitively costly, a company may revise the rule rather than let the rule determine its conduct. Even where testing changes development or deployment, the public evidence tells us much less about whether those changes reduce harm after release.

The purpose of this article is to identify the evidence that would distinguish meaningful constraint from organized compliance. The question is no longer whether safety frameworks have effects but whether they remain binding when their consequences become expensive, and whether the decisions they produce make deployed systems safer.

Major developers have adopted voluntary frameworks built on a similar principle: stronger capabilities should bring stronger safeguards (Williams & Freund, 2026). Anthropic offers the clearest public example of both their influence and their limits. Companies that disclose less about their policies provide less evidence to examine, so the public record may not represent the industry as a whole.

In May 2025, Anthropic released Claude Opus 4 under its AI Safety Level 3 (ASL-3) protections. The company had not concluded that the model crossed the relevant capability threshold, but said it could no longer confidently rule that out. It applied stronger deployment and security measures while continuing its evaluation (Anthropic, 2025). The policy materially affected how the model reached the public.

Donald MacKenzie’s history of the Black-Scholes formula offers one interpretation, and his title states the claim: the formula was an engine, not a camera. It began as an account of how options ought to be priced and became a force shaping how they were priced (MacKenzie, 2006). MacKenzie called the strong form of this Barnesian performativity: using a model makes the world more like the model. A safety threshold can work the same way. Once written down, it enters the environment in which technical and commercial decisions are made rather than describing that environment from outside.

John Meyer and Brian Rowan offer the skeptical account. Organizations often adopt formal structures as ceremony, gaining legitimacy while insulating core activity from their demands (Meyer & Rowan, 1977). On this view, a safety framework is a costume: visible and socially useful, but not necessarily connected to the work underneath.

Neither account is sufficient. Anthropic’s framework was not an empty costume because it changed a release. Nor was it an engine running on fixed settings because Anthropic weakened its strongest forward commitment after concluding that unilateral compliance could conflict with competitive pressure. Matthew Kraatz and Edward Zajac found a similar limit to institutional conformity among American colleges, which departed from accepted templates when market and technical pressures favoured change without suffering the expected penalties (Kraatz & Zajac, 1996).

The third outcome is the hardest and most important. The apparatus creates detailed records of evaluations and organizational decisions, but much weaker evidence that those decisions reduce harm. The European Code of Practice addresses this through continuing risk management, incident reporting, post-market monitoring and external evaluation (European Commission, 2025). These measures may reveal failures. They cannot readily establish how much worse an event would have been without a safeguard, or whether a model that passed a bounded test will remain safe against new prompts, tools, fine-tuning methods and attacks.

The operational layer therefore deserves scrutiny. Independent experts prepared the aforementioned Code of Practice through a multi-stakeholder process, but providers hold much of the knowledge needed to decide how hard a model has been tested, which methods count as adequate elicitation and what remaining risk is acceptable. Anthropic’s own disclosures show how much room that leaves. Reviewing its first year under the framework, the company reported that some evaluations had lacked basic elicitation techniques such as chain-of-thought prompting, and it responded in part by extending the evaluation interval from three months to six so that future assessments would not be rushed. In February 2026 it judged that a model did not cross its automated AI research threshold while noting that confidently ruling this out was becoming difficult and required assessments more subjective than it would like. Providers can also revise the thresholds themselves: versions 3.3 and 3.4 changed those covering chemical and biological weapons production and automated AI research to better reflect the company’s stated threat models (Anthropic, 2026b). As Philip Mirowski and Edward Nik-Khah argued in another context, understanding what a measure does is incomplete without examining who selects and operationalizes it (Mirowski & Nik-Khah, 2007).