Your AI Automation Has an Expiry Date: What Model Deprecation Actually Costs You

The email arrived on a Tuesday afternoon in June. One line: think the invoice thing is broken, it's been picking up the wrong VAT number since about the 14th.

Nobody had deployed anything. No prompt had changed. Nobody had touched that repository in eleven weeks. The pipeline was doing exactly what it had done every working day for fourteen months, and it had quietly started getting one field wrong.

The model it called had been retired on the 15th.

Here is the part that still bothers me. The system didn't fail. Had it failed, we would have known within the hour and fixed it before lunch. Instead it kept running, kept returning confident well-formed JSON, and kept writing a number from the wrong corner of the page into a finance system that had no reason to doubt it. Six weeks of invoices went through before a human noticed.

This is the one structural difference between an AI automation and the rest of your IT stack, and almost nobody prices it in. So let's price it in: what actually changes underneath a working automation, what the change costs, and what we now build into every pipeline as a default.

Software rots. AI software rots on someone else's schedule.

Normal software has a comforting property. If you don't touch it, it mostly keeps working. You pin your dependencies, you skip the upgrade, and the thing runs. Upgrading is a decision you make, at a time you choose, for a reason you can explain to your CFO.

A hosted model is not that. You have no agreement with the vendor about how long the thing you built on will exist. They publish a policy, they send an email, and the clock runs whether or not anyone on your side is reading it.

The published policies in 2026 varied more than most people realise:

Read the third row again, because it is the one that catches people. xAI's retirement notice said the old requests would stop working. The documentation later clarified that the retired slugs would keep resolving, routed to a newer model at that model's prices. Both are defensible engineering choices. Neither produces a line in your error log.

And it isn't only the model. Langfuse is removing its legacy batch ingestion endpoint from Langfuse Cloud on 16 November 2026, which strands the older SDK versions that talk to it. A pipeline we built last year has four upstream clocks running on it: a model snapshot, an SDK major version, an observability endpoint, and a vector database API version. I set none of them. Neither did the client.

The three ways it actually breaks

Numbers two and three are the expensive ones, and not because the repair is hard. In both cases the actual fix is a day of work. What you pay for is the window: every wrong record you shipped before anyone noticed, the remediation, and the conversation where you explain to the operations team that the system they were told to trust has been wrong since the middle of last month. That last cost is the one that lingers. As I've written about failed pilots before, adoption is what actually kills automations, and adoption runs on trust.

Why your monitoring didn't catch it

Because your monitoring was built for the wrong kind of failure.

Uptime, latency, error rate, queue depth. Those four catch every infrastructure problem you will ever have, and they stay a reassuring green straight through a quality regression. There is no HTTP status code for technically valid, subtly wrong.

Aggregate accuracy doesn't save you either, and this is the part teams underestimate. A behaviour change rarely degrades everything at once; it degrades the slice where the changed behaviour applies. Multi-page invoices. Handwritten amendments. The one supplier who puts the tax number in the footer. If that slice is eight percent of volume, a real and serious regression moves your weekly average by a point or two, which is indistinguishable from an ordinary bad week.

So detection defaults to a human. Someone downstream notices an odd value, assumes they misread it, sees it again a week later, mentions it to nobody in particular, and eventually a customer complains loudly enough to start a thread. Across the cases we've been brought in to clean up, that path takes three to six weeks.

Forrester's 2026 work on agent deployments puts some weight behind this. They report that 22% of deployments show negative ROI at the twelve-month mark, and when they break down the causes, 26% of those trace back to drift in evaluation coverage. Not bad models. Not bad use cases. Nobody watching closely enough to notice the thing had changed.

What this costs, in numbers

These are our own figures from our own projects, so treat them as ranges rather than benchmarks. Your pipeline is more or less complicated than the ones I'm averaging.

A planned migration on a mid-complexity pipeline runs four to six working days: two days building or extending the evaluation set if one doesn't already exist, one to three days retuning prompts and output schemas against the new model, a day of shadow running and diffing, half a day for rollout and a rollback path. Call it €3,500–€6,000. Expect one to two of those a year, per model you depend on.

That's the good version. Here's the June one, priced out.

The diagnosis line is the one that surprises clients. Four days of a senior engineer's time went into proving a negative: no deploys, no config drift, no data change, no infrastructure event, before anyone thought to ask whether the model was still the model. Once you have seen it happen, you check that first. We do now.

In our post on implementation cost there's a line recommending you budget 15–25% of build cost annually for maintenance. This is the bulk of what that number is for. It is not a retainer for feature requests.

The four things we now build into every pipeline

None of these are clever. They are the AI equivalent of pinning your dependencies and writing a test suite, and the only reason they need saying is that the field is young enough that they aren't yet default.

Together these add roughly 10–15% to build cost, and I won't pretend that's always worth it. On a genuinely low-stakes internal automation, something that drafts a summary a human reads and edits anyway, skip the shadow runs and the weekly schedule. Keep the pinned snapshot and keep the evaluation set, even a small one. Those two are cheap, and they are the difference between knowing and guessing.

What to put in the contract

If you are buying an AI automation from anyone, us included, five questions belong in the conversation before you sign:

1. Which exact model ID does this run on, and what is its published retirement date?
2. When that date arrives, who pays for the migration? In scope, retained, or time-and-materials?
3. Do I own the evaluation set, and is it delivered with the system?
4. Where do the prompts live, and can I read them without you?
5. What is the rollback path if the replacement model turns out to be worse than the one it replaced?

A good implementer will have answers ready. What you are really testing is whether they have been through this before, and the third question is the tell. Anyone who has survived a forced migration knows the evaluation set is the asset that made it survivable, and will hand it over without being asked twice.

Thirty days of post-launch support and a bug-fix warranty do not cover a vendor-initiated retirement fourteen months in. That isn't a bug in anyone's code. No warranty covers it, and if no name is attached to that day, the day is yours by default.

You'll find out on a Tuesday.

An automation isn't finished when it works. It is finished when someone owns the morning the model underneath it changes, and both of you know who that someone is.

The four defaults above cost a couple of weeks, once. Not having them costs a couple of weeks a year, indefinitely, at a moment you don't get to choose.