Most AI demos work. You feed in a few hand-picked examples, the model gives a sharp answer, and everyone in the room is impressed. Then it meets real users, messy data and edge cases nobody planned for, and things start to break.
We've seen that gap often enough that our whole process is built around closing it. Here's how a typical AI project runs at JRC Software.
1. We start with the problem, not the model
In the first conversation, we rarely talk about models. We ask about the work. Which decision or workflow is slow, expensive or error-prone today? How would you know if it got better? What data do you have? And what happens when the system gets it wrong?
That last question shapes everything. A tool that drafts internal summaries can afford the occasional mistake. One that flags transactions for review can't. The answers tell us how much to automate, where a person stays in the loop, and whether AI is the right tool at all. Sometimes it isn't, and a simple rule or a better form solves the problem faster.
On a booking platform we built for service businesses, a wrong answer from the AI agent meant a real appointment in someone's diary, or a customer told they were booked when nothing had been reserved. So the model never confirms anything itself. It proposes; the server decides. And the booking is refused outright if the customer hasn't verified their identity first.
2. We look hard at the data early
A model can only work with what it's given. If customer records live in three systems that disagree with each other, no model will fix that.
Before we build anything, we check where the data lives, how current it is, who owns it, and whether we're allowed to use it the way we plan to. Our data engineering team often works alongside the AI build. Fixing a pipeline in week two is far cheaper than discovering the problem in month three.
3. We pick the simplest architecture that works
Most problems don't need a custom-trained model. Depending on the use case, we might use a general-purpose AI model that looks things up in your documents before it answers, a classic machine learning model, plain business rules, or a mix.
We decide based on accuracy, cost per request, response time, security requirements and how easily your team can maintain it afterwards.
On the booking platform, we skipped the document lookup altogether. Each business has a handful of documents at most, so we pull the key facts out of each one when it's uploaded and give those to the model directly. A separate search system would have been another thing to run, and no more accurate.
4. We build the whole system around the model
In most AI products, the model is a small part of the code. Around it sit the application, data pipelines, integrations with your existing systems, access controls, logging, and a deployment process that lets us ship updates safely.
This is where many AI projects stall. It's also where our experience in cloud, platform and quality engineering does most of its work.
5. We test what the AI produces
Traditional tests check that software does what it's told. AI needs more, because the same input won't always give the same output.
Early in each project, we build an evaluation set. On the booking platform it's fifteen complete customer journeys, written out step by step: a new customer booking from scratch, a returning one, an expired reschedule link, two people going for the same slot. Before a release, a script plays them through a working copy of the system. We check what the system actually did, such as whether a time slot was held and a payment was started, and only loosely check the wording, because wording shifts between model versions and the outcome shouldn't.
Every change is checked against it before it ships. We also test how the system handles unexpected or hostile input, such as attempts to trick it into ignoring its instructions, sensitive personal details typed into a conversation, and questions it was never meant to answer. Security and quality reviews run throughout the build instead of the week before launch.
6. We design for the people who have to trust it
A feature can work perfectly and still go unused if people don't trust it. Design makes a big difference here: showing the source behind an answer, flagging when the system is unsure, and making it easy to correct or override a result.
This matters most where a person is still accountable for the outcome, such as approvals, financial decisions and anything customer-facing.
7. We keep watching after launch
Launch is when you finally learn how people use the system. Inputs drift, usage patterns change, and model providers update the models underneath you.
So after go-live, we log every turn of every conversation: which model answered, how long it took to start responding, how much text went in, and what was said on both sides.
Those logs have already changed our decisions. Another product we work on is a voice agent that answers the phone for service businesses. A customer rings to book an appointment, the AI agent picks up, and the two of them talk it through. Every reply has to come back quickly. On a phone call, a pause longer than a second or two feels like the line has gone dead.
The agent sends the same background instructions to the model on every turn, so we switched on a feature that lets the model provider store those instructions instead of receiving them fresh each time. In theory that's cheaper and quicker, and usually it is.
The logs disagreed. Some turns sat for close to 12 seconds before the agent started speaking, while calls without the feature stayed between 0.8 and 1.5 seconds every time. We didn't need to know the exact cause to make the call. On a phone line, an occasional 12-second silence costs far more than the feature saves, so we switched it off. Without the logs, the first sign would have been callers hanging up.
We keep adding real cases to the evaluation set and review cloud and model spend as usage grows. An AI system that isn't maintained gets worse quietly.
The short version
Getting AI into production takes clear problem definition, data work, solid engineering, careful testing, thoughtful design and ongoing care. The model is just one piece.
If you have an AI idea, or a prototype that hasn't made it past the demo stage, we'd be glad to talk it through.
Thinking about where AI fits in your product roadmap?
Let's talk about what an end-to-end AI system would take.