How much does an AI project cost, from POC to production?

An artificial intelligence project rarely costs what people expect. The question that comes up in every meeting is the price of the model, when that is the smallest line and the easiest one to steer. The three budgets that actually decide the cost lie elsewhere: scoping, data preparation and integration, and running the thing over time.

This article gives verifiable orders of magnitude, the method for calculating your own crossover point between an API and a dedicated server, and the five mechanisms that make a budget slip. The goal is not to give you a price, which would mean nothing without your use case, but to let you challenge any quote you receive.

Token pricing, and why it is almost never the problem

Let us start with the number everyone looks at. Models are billed per million tokens, separately for input and output, with output always costing more. On OpenAI’s public pricing, a light model such as GPT-4o-mini sits at 0.15 dollars per million input tokens and 0.60 for output, while a top-tier model reaches 10 dollars for input and 50 for output. At Mistral, Mistral Large is listed at 0.50 dollars for input and 1.50 for output.

The ratio between the two ends of that range exceeds a factor of sixty on input and eighty on output. In other words, the choice of model weighs more on your bill than anything you can optimise afterwards. Classifying an email, extracting a date or rephrasing a paragraph do not require the most capable model on the market. Reserving that one for the tasks that genuinely need it cuts the bill without degrading the result.

Two mechanisms complete the picture and are widely underused. Context caching drops the price of already-seen input tokens to a tenth of the standard rate at OpenAI, which changes everything as soon as a system resends the same instructions or the same documents on every call. And batch processing, for anything that does not need an immediate answer, applies a fifty percent discount. A project that processes documents overnight therefore pays half the headline price.

The second lever is context size. Many implementations resend a system prompt of several thousand words and documents pasted in without need on every call. It is that volume, multiplied by the number of calls, that builds the bill, not the unit rate.

The three phases, and what each really costs

Scoping and the proof of concept

This is the most profitable phase and the one most often skipped. Its purpose is not to build, it is to answer three questions: does the use case have measurable value, do the necessary data exist in a usable state, and can a model perform the task at an acceptable level.

Confusing a proof of concept, a prototype and a minimum viable product costs months. We covered it in our article on what to build first. The useful distinction: a proof of concept is allowed to be ugly, slow and manual, but it must settle an uncertainty. If it settles nothing, it is worthless whatever it cost.

The budget for this phase is counted in scoping days, not licences. And it is the last moment at which you can still decide not to do the project at all, which is often the most profitable decision of the year.

Going to production

This is where most of the cost, and most of the surprise, sits. The model is one component among many. Around it you have to build data retrieval, chunking, indexing, access management, logging, the interface, and the handling of cases where the system does not know the answer.

On an assistant connected to your internal documents, quality depends far more on that architecture than on the model chosen. That is the subject of our article on RAG architecture over internal documents, and it is also why the question of RAG versus fine-tuning belongs in scoping rather than afterwards.

One proportion from our own projects: on a corporate document assistant, the cost of model calls rarely exceeds a tenth of the first-year budget. The rest is development, integration and data preparation.

Running it

The forgotten line. An AI system in production needs supervision, because it fails differently from conventional software: it does not crash, it answers badly. That means logging exchanges, watching costs, setting spending caps, and handling user feedback.

On top of that comes a workload nobody anticipates: models change. A provider deprecates a version, publishes a better and cheaper one, adjusts its rates. What worked six months ago has to be re-checked. Budgeting zero for that maintenance means letting quality degrade silently.

The line nobody budgets: evaluation

This is what separates a system that holds up from one that dazzles in a demo and then disappoints. Without evaluation you have no way of knowing whether a change of model, prompt or chunking improves or degrades the system. You are navigating on impressions.

The good news is that it costs less than people think. Fifty to a hundred representative examples, annotated by hand with the expected answer, are enough to compare two configurations reliably. Two measurements are enough to steer by: recall, which tells you whether the right information is among the retrieved passages, and faithfulness, which measures the share of statements not supported by the sources provided.

On that second measurement, Vectara’s public leaderboard gives a useful reference point: even the best models remain at a few percent hallucination on a task where the summary must stay faithful to a supplied document. That is not zero, and it is exactly why you should measure on your own documents rather than trust a general score.

Allow one to two days to build that evaluation set. It is the highest-return investment in the whole project, because it makes every subsequent decision verifiable.

API or dedicated server: the crossover can be calculated

The question comes up systematically, often driven as much by sovereignty as by cost. It deserves a calculation rather than an intuition, and the calculation is simple: divide the monthly cost of your server by the price per million tokens of the API you would be replacing.

One order of magnitude to anchor the reasoning. At Scaleway, an instance with an L4 GPU is billed at 0.79 euros per hour, roughly 575 euros a month running continuously. Against a light model billed at a few tens of cents per million tokens, you need billions of monthly tokens before the server becomes the economical choice. Against a top-tier model, the threshold falls to a few hundred million. In both cases, that is a volume most projects never reach.

Two caveats complete the calculation. First, the model has to fit in the GPU’s memory, which rules out entry-level cards for large models. Second, the inference engine matters as much as the card: an engine optimised for throughput such as vLLM serves far more requests per second than sequential execution, as we detail in our article on hosting an open source LLM.

Finally, cost is not always the real motive. When the constraint is confidentiality, the reasoning changes in nature. It is then worth checking what the provider actually says: OpenAI states in its API data documentation that data sent through the API is not used to train its models, but that content passes through abuse-monitoring logs retained for up to thirty days. That is the trade-off we set out in our comparison of Mistral against OpenAI.

The five mechanisms that blow a budget

• A scope that was never closed. An assistant meant to answer everything has no success criterion, and therefore no end. The first scoping task is to write down what the system will not do.
• Data discovered along the way. Scattered documents, mixed formats, contradictory versions, unclear access rights. This is the line that most often explodes, and it can be measured before starting rather than during.
• No evaluation. With no test set, every change is judged on impressions, iterations multiply and the development budget doubles without quality improving.
• An agent with too many rights. As soon as a system can act, prompt injection becomes a real risk, ranked first in the OWASP Top 10 for large language model applications. Restricting tools and requiring human approval on irreversible actions costs less than the incident.
• No spending cap. A badly closed loop or a growing context can multiply consumption overnight. A cap per key and an alert prevent the end-of-month surprise.

Three markers for reading a quote

When a proposal lands, three elements immediately say whether it was built or improvised. The first is the presence of an evaluation line. A quote with neither a test set nor a quantified success criterion is selling a demo, not a production system.

The second is how data is treated. If source preparation appears as a flat line without having been examined, it will either be badly underestimated or billed as a change request. A serious supplier asks to see a real sample before quoting, and declines to commit on a corpus they have not opened.

The third is the running cost, separate from the build cost. A proposal that says nothing about the monthly budget for calls, hosting, supervision and model maintenance leaves the most durable line out of scope. That is precisely the one discovered three months after launch. We approach the same logic in our article on AI transformation in the enterprise.

How to price before committing

The method we apply fits in four questions, and it is answered in a workshop rather than in a quote. What decision or task does the system replace, and how much human time does that represent today? What data is needed, where is it, and in what state? What does a good answer look like, phrased precisely enough to be annotated on fifty examples? And what happens when the system gets it wrong, who sees it and who corrects it?

Once those four answers are written, pricing becomes mechanical, and above all comparable between suppliers. Without them, any quote is a finger in the air, including ours. It is the same logic that governs the cost of building a SaaS: it is not the technology that sets the price, it is the precision of the scope.

One last marker, counter-intuitive but consistent: the cheapest AI projects are the ones that start with the narrowest use case. An assistant that answers one family of questions well ships in a few weeks, produces a measurable result, and becomes the base for the next extension. An assistant that has to know everything on day one never ships.

What we take from it

The cost of an AI project is steered by scoping, not by the choice of model. Token pricing is real but marginal, going to production concentrates most of the spend, and evaluation is the only investment that makes the others verifiable. As for the dedicated server, confidentiality justifies it far more often than economics do.

If you have an assistant, agent or intelligent automation project and you are trying to price it seriously, that is exactly what our AI scoping is for. We come out of it with a written scope, an evaluation set and a defensible estimate. Let’s talk.