A demo is not a system

Almost every AI demo works. That is not a compliment, it is the problem.

A demo is built to work. It runs once, in front of a friendly audience, on inputs the person demoing chose. Production is the opposite of all three: it runs every day, in front of no one, on inputs nobody chose. The distance between those two states is what I call the Demo-to-Ship Delta, and it is where most AI projects quietly die.

The numbers say so plainly. MIT’s 2025 study of enterprise generative AI found that only about 5 percent of pilots ever reach production (MIT, via Forbes, 2025). Gartner expects close to a third of generative AI projects to be abandoned after the proof of concept (Gartner, 2024). The demo is not where projects fail. The demo is where they look like they are winning, right before the part that is hard.

The model is the small part

The most useful thing I ever read about this is a decade old. In 2015 a team of Google engineers published a paper called Hidden Technical Debt in Machine Learning Systems (Sculley et al., NeurIPS 2015). Its famous point is a picture: in a real machine learning system, the box that holds the actual model is tiny, surrounded by a much larger sprawl of data pipelines, configuration, monitoring, serving infrastructure, and the glue that holds it together. The clever part is a rounding error in the total work.

That is the delta, drawn to scale. The demo is the small box. This is also why “the model scored 94 percent in testing” tells you almost nothing about whether the thing will ship. The score measures the small box. The delta is everything around it.

What the invisible ninety percent is

It is worth naming what lives in that larger space, because you cannot budget for work you cannot see, and the whole trouble with the delta is that it is invisible until you are in it. In a system that actually ships, the unglamorous majority is roughly this:

• The data path. Where inputs come from, how they are cleaned, and what happens when they arrive late, malformed, or not at all. Most production incidents are data incidents, not model incidents, and none of them appear in a demo that ran on a tidy file someone prepared.
• Real evaluation. A test set that includes the ugly cases, not just the representative ones, and a way to re-run it automatically every time anything changes. More on this below, because it is the piece people skip most.
• Monitoring. Something watching the live output so that when quality slips, a human finds out in hours, not from a complaint months later.
• The human loop. Who reviews the output, who is allowed to overrule it, what they actually see, and how their day changes. This is where this essay meets the management gap: the technical work and the ownership tend to fail in exactly the same spot.
• Rollback and retraining. A plan and a trigger for the day the system has to be pulled back or refreshed, decided before you need it, not improvised during the incident.
• Integration and access. How it plugs into the tools people already use, and who is permitted to see what. In regulated settings that last point is a project of its own.

None of that shows up in a demo. All of it decides whether the demo becomes a system.

A demo is a best case. Production is a worst case.

There is a second reason demos mislead, and it is about the inputs.

A demo runs on inputs someone selected, which means it shows you the system’s best case. Production runs on inputs reality selects, which means you eventually meet its worst case, on an ordinary day, when no one is watching. The number that matters is not the average across the good cases. It is the floor: what the system does on its worst realistic input. In most demos the floor is never shown, and in most procurement it is never asked about.

How you actually measure the floor

Finding the floor before production does is a discipline, and it is most of what I spend my research time on. A few principles separate a real evaluation from a demo dressed up as one.

Test on inputs you did not choose. Pull a real, messy sample from the actual environment, including the cases that embarrass the system. If your test set only contains inputs the model handles well, you have measured your own optimism, not the system.

Decide what “works” means before you look. Write down the threshold the system has to clear, and the cases it must never get wrong, before you run the numbers. Deciding after you have seen the results is how a mediocre score quietly becomes “good enough.”

Report the distribution, not the headline. One average hides the tail. Break the result down by the segments that matter, the rare case, the vulnerable user, the edge that is a sliver of the volume and most of the harm, and then look hard at the worst bucket, because that is the one some real person is going to live in.

Re-run it, always. An evaluation is not a certificate you earn once. Models age, as we will see in a moment, so the floor moves, and the only way to know where it sits now is to keep measuring it.

Most teams do none of this, because it is slow and it makes the system look worse than the demo did. That is exactly the point. It is the difference between meeting the worst case in a test, on your terms, and meeting it in production, on its own.

It gets worse after launch, on its own

Here is the part almost no demo will ever tell you. Even if you ship a system that genuinely works on the day it goes live, it will get worse on its own.

A peer-reviewed study in Scientific Reports tracked model quality over time across healthcare, transport, finance, and weather data and found that more than 90 percent of models degraded as time passed since their last training, a phenomenon the authors call AI aging (Vela et al., 2022). The world the model learned from keeps moving, and the model does not move with it unless someone makes it. So the demo is not just the best case on the best inputs. It is also the best the system will ever be, on day zero, before reality starts pulling it apart.

That reframes what you are buying. A demo sells you a peak. Production hands you a slope, and the slope points down unless someone owns keeping it up.

Why the timeline always slips

If you have ever watched an AI project run to twice its estimate, this is usually why. The estimate was built around the visible box, the model, because that is the part everyone can see and reason about. The delta, the data plumbing, the honest evaluation, the monitoring, the retraining, was either forgotten or waved at with the word “productionise,” as if it were a single step at the end rather than most of the mountain. Gartner names escalating cost and unclear value as top reasons projects get abandoned after the proof of concept, and both are the delta arriving late and uninvited. The work does not go away because you did not plan for it. It just shows up as the overrun.

Why regulated work cannot paper over the delta

In a consumer app the delta is survivable. You ship something rough, watch the graphs, and patch. In healthcare, public services, or law you do not get that grace. “We will fix it in production” is not a sentence you are allowed to say when the worst case is a patient, a benefits decision, or a legal exposure.

The EU AI Act makes some of this explicit. For high-risk systems, whose core obligations apply from December 2027, it requires ongoing human oversight and record-keeping in actual use, not just a good result at approval time (EU AI Act, Article 14). In plain terms, the law assumes what the demo hides: that a system’s behaviour in production is a live thing that has to be watched, not a number you certify once and forget. The delta is not an engineering footnote in regulated work. It is the compliance surface.

How to size the delta before you buy

You cannot see the delta in a demo, by design. But you can ask about it, and the questions are not technical.

• What does it do on its worst realistic input, and can you show me that case, not your best one?
• Who watches it in production, how would they even notice it had quietly gotten worse, and how fast?
• When it drifts, and it will, whose job is it to retrain or roll back, and on what trigger?
• What is the plan for the day the data arrives late, malformed, or not at all?

Take an illustrative case. A team demos a document-classification tool that sorts incoming case files, and in the demo it is fast and accurate. Ask the questions. The demo ran on last quarter’s clean sample; nobody has tested this quarter’s messier files. No one is assigned to notice when accuracy slips, so the first sign of trouble will be a misfiled case someone catches by hand months later. There is no retraining trigger, so the model ages quietly on the slope. The model in the demo was genuinely good. The system around it did not exist yet, and the system is the job.

If a vendor can only show you the best case, you have not seen the product. You have seen the brochure.

What I have seen

None of this is theoretical for me. I build and ship AI in settings where the worst case is expensive, and the pattern is consistent: the part everyone celebrates, the demo, the model, the score, is the part that turned out to be nearly done, and the long tail of work nobody scoped, the evaluation of the floor, the monitoring, the plan for when it drifts, is what actually decided whether the thing survived contact with real use.

What to do about it

Before you buy or build, treat the demo as the beginning of the estimate, not the end of it. Assume the visible ten percent is done and the invisible ninety percent is not. Ask to see the worst case, not the best. Decide what “works” means in writing before you measure it. And budget for evaluation, monitoring, and the ongoing work of keeping a system from ageing, because that spend is not overhead on top of the project, it is most of the project.

A demo is a promise about the best day. Production is a question about the worst one. The teams that ship are the ones who spent their time on the second question while everyone else was still applauding the first.

---

Sources

• MIT Project NANDA, The GenAI Divide: State of AI in Business (2025), as reported by Forbes.
• Gartner, Predicts 30% of Generative AI Projects Will Be Abandoned After Proof of Concept by End of 2025 (2024).
• Sculley, D. et al., Hidden Technical Debt in Machine Learning Systems, NeurIPS (2015).
• Vela, D. et al., Temporal quality degradation in AI models, Scientific Reports (2022).
• Regulation (EU) 2024/1689 (the EU AI Act), Article 14.