Don’t assume composability in AI safety. Deployments need their own testing.

Practitioners know well that deployed AI systems behave very differently from foundation models. With modifications such as system prompts, retrieval layers, and fine-tuning, behavior keeps changing. But safety evaluations often incorrectly assume composability—that is, directly citing safety scores from foundation model system cards.

System Composability Is Not a Given

Composability refers to whether evaluation results from an upstream system can apply to a system built downstream. In classical software, this assumption usually holds, but in AI safety, the situation is often the opposite.

Safety Scores Change Unpredictably

Research by CDT and MIT shows that compared with foundation models, safety scores of medically and legally fine-tuned models are sometimes higher and sometimes lower, and no single factor predicts the direction or magnitude. Adding runtime guardrails usually improves scores, but the specific effect needs to be tested in practice.

Measurement Methods Must Improve in Tandem

Many commonly used benchmarks are too noisy to reliably compare systems before and after deployment. MLCommons' Candela benchmark-as-a-service offering can run AILuminate tests on a complete deployed system that includes prompts, guardrails, retrieval, and tools, and provides error margins.

Making Composability Testable

When measurement uncertainty is known, researchers can test which modifications preserve results and which break them. This helps teams decide whether to skip testing, streamline testing, or conduct a full review, and can also support regulators in dividing responsibilities between developers and deployers.

Get Involved Now

Deployment teams can immediately use Candela to test systems. Researchers can join the AIRR Analytics working group to jointly advance research on AI safety composability.