Three rules for adding AI to a codebase that already ships
Adding AI capabilities to a system already running your business is a different discipline than building a greenfield AI app. Three rules we keep coming back to.
Most “add AI to our product” projects we see treat the AI layer as if it were a greenfield app bolted on the side. It rarely works. The codebase running your business has already paid for its scars — error handling, audit trails, edge cases that no one wrote a ticket for — and AI bolted on top of that without discipline tends to undo a decade of slow-earned reliability.
Three rules we keep coming back to.
1. Scope the agent narrowly, on purpose.
The instinct is to build the most capable agent you can — give it the database, the API, the document store, and a long system prompt. That’s how you ship a Swiss Army knife that no one trusts in production.
Better: pick one task the agent owns. Make its inputs and outputs structured. Write down what it is not allowed to do. The narrower the scope, the easier it is to evaluate, monitor, and explain to the people who already maintain the system.
2. Build the eval before the agent.
If you can’t measure whether the agent is doing the job, you don’t have an agent — you have an opinion. Even a small eval set, manually curated, beats the alternative. We routinely see teams spend a quarter tuning prompts without a way to tell whether version 14 is better than version 4.
The eval doesn’t need to be sophisticated. It needs to exist before the agent, not after.
3. Make it operable by the people on call.
The on-call rotation didn’t sign up for an AI system. They signed up for the system you had a year ago. If the agent goes weird at 3am, the operator needs to be able to read what it did, why, and turn it off without paging an ML engineer.
That means logs in human-readable format, not blob telemetry. It means a kill switch that doesn’t require a deploy. It means dashboards that show what the agent decided this week — not just whether the endpoint returned 200.
The summary: AI inside an existing business system is an integration problem first, a model problem second. Treat it like the boring infrastructure work it actually is, and the AI part gets a lot less risky.