Skip to content
Free since 2015. Built for the engineers who keep things running.

cat /articles/tests-as-data-ensuring-ai-generated-code-does-what-we-want.md

CI/CD

Tests as data: ensuring AI-generated code does what we want

Matteo Vaccari argues that when AI writes the code, developers should treat customer-level tests as the main contract, and that storing those tests as data makes them harder for coding agents to change.

Martin Fowler11 Nov 2026 · 1 min read · summary

article --summary

The summary

The piece looks at a worry about AI coding assistants: if agents write both the production code and the tests, people can lose confidence that the tests will catch functional mistakes. Vaccari's reasoning starts from the idea that, once AI writes the code, developers effectively become its customers. From that view, customer tests rather than developer tests become the primary agreement between people and AI. Keeping that agreement as test data instead of test code, he suggests, makes it harder for agents to alter it without notice. He also notes that custom parsers and test runners were once too costly to build and maintain, but AI has made them cheap enough to be practical.

Editorial note: This is a summary of an article first published by Martin Fowler. The full piece is on their site.

Reading now

Topic
CI/CD
Source
Martin Fowler
Published
11 Nov 2026
Format
Curated summary

Have a take on this?

Write it up and submit it. It goes on the list once an editor approves it.

Submit an article

articles --latest

Latest articles

New from the blogs we follow and the community.

All articles

newsletter --weekly

DevOps Weekly, minus the noise.

The best reads, new jobs and upcoming events, in one email a week.