The invisible costs of keeping machines compliant, accurate, and sane.
Corporate leaders will know that standing up a proof of concept in order to satisfy the desire from above – or below – to “do something with AI” is relatively easy, and relatively cheap. With a small team, a pre-trained model and a credit card to pay for API access and in a few weeks you can have a demo that is more than ready to present at a board meeting.
The problem is what comes next. While entry – or ‘developer’ – level APIs are affordable, more advanced systems can cost thousands of pounds a month. During the training of an AI model, algorithms perform millions – or billions – of calls a day for weeks or even months, and the major providers – Google, Microsoft, and OpenAI – charge per million calls.
But experts say that responsibly scaling AI doesn’t have to translate to runaway costs. Indeed, the most successful organisations treat governance and retraining as operational disciplines that do not take them by surprise. With the right architecture, forecasting, and model-lifecycle management, costs can be controlled, predicted, and even reduced.
“We’ve had to start treating AI like cloud,” says Mike Downing, CIO of WPA. “It’s not a one-off investment. It’s a continuous service. You forecast for it. You monitor it. And if you don’t, you’re in trouble — because the costs are going to keep coming whether you plan for them or not.”
Executives have learned that the model itself is the part everyone gets excited about, but it is only a fraction of the total cost. The real cost is everything around it: the infrastructure that runs it, the processes that validate it, and the governance that stops it going off the rails.
That is particularly true in Financial Services and insurance, where a single bad output could trigger a regulatory investigation, which could end up costing financially, not to mention reputationally. Highly regulated industries struggle to step outside their regulatory loop and share data. Responses need to be watertight before they ever go out to a customer.
That insistence on accuracy means every answer, inference, and document summary generated by an AI system must be tested, audited, and explained. Each of those steps consumes compute power, tokens, and human time, none of which are free. Executives say that boards constantly underestimate how much effort it takes to keep systems compliant.
“You expect all this AI stuff to deliver in six months, but you’ve got to go back and demand the benefit later. It’s not enough to build it once. You’ve got to maintain it,” said John Crichton, CIO of Gallagher Re.
“If you’re in a highly competitive space where every minute that passes is a missed opportunity, the challenge is presenting AI use cases to the C-suite that have that duality of time versus money and speed to gain a competitive advantage,” added Rodnei Connolly, Head of Data, AI and Digital CX Delivery Arteffect Digital.
The token trap
Then there’s the economics of tokenisation, an unseen meter ticking away in the background of every prompt. Most large-language models charge by token, the atomic unit of text fed into and out of the model. What looks like a simple query can be hundreds of tokens long, and when multiplied across an enterprise, the numbers become eye-watering.
“It’s a completely different cost curve,” Downing says. “Most of the models charge you based on the number of tokens you consume, input as well as output.”
“That changes the total cost of ownership entirely. You have to forecast token usage the way you forecast cloud compute. If you don’t, you’ll be surprised by the bills.”
Drift, the slow bleed of accuracy
Even if you can get token and call cost under control, a second invisible bill soon arrives: drift. Models and context change and very quickly the system that you spent time and money building is not fit for purpose. By this point, the company is reliant on AI, so going back is not an option.
Every time a data source is updated, or a regulatory rule changes, the model needs to be retrained. That means new data ingestion, validation, and redeployment, which are all labour-intensive. Drift isn’t just about accuracy, either. A model trained on last year’s market data may not simply be wrong; it may be dangerous. “If you don’t put the resources into it, you’ll end up with systems that stop working as promised,” says Thomas Boltze, CTO, Standard Chartered Ventures Portfolio Company.
Compliance never sleeps
Regulators, too, are learning fast. Many financial authorities now expect firms to prove that their AI systems are explainable and properly monitored, even if the system doesn’t make final decisions. From a regulator’s perspective, Boltze warns, “‘The AI made the mistake’ is not an acceptable excuse. You’re still accountable. Especially if you’re dealing with client money, the scrutiny is ten times higher.”
That scrutiny has financial implications. Firms must keep audit trails, human oversight, and compliance testing in place for every AI-driven process, even internal copilots. Each additional safeguard means more data processing and more compute hours.
But automation can reduce that effort, while lifecycle management tools can track performance and drift automatically. Choosing the right-sized models, instead of defaulting to the biggest or newest, also keeps spending proportionate to value. And teams that develop the literacy to understand how AI works can focus on innovation instead of firefighting.
Scaling AI responsibly is about turning discipline into an advantage: knowing where every token goes, what each model contributes, and when to retrain or retire it. The result isn’t just lower cost, it’s a system that stays accurate, auditable, and trusted as it grows.
The economics of maturity
The paradox of AI cost management is that the more responsible you are, the more expensive it can look, because strong governance, model retraining, and human review all add up. But cut those corners, and the eventual price can be far higher.
That’s why forward-looking firms are starting to budget for AI as an operational cost, not a one-time capital spend. They forecast usage, monitor drift, and even set aside a budget for retraining cycles the way they do for software upgrades.
CIOs interviewed said they are now educating CFOs and boards on token consumption, model lifecycle costs, and retraining schedules, a level of financial literacy that simply didn’t exist a year ago. As one data leader put it, “The real innovation isn’t the model — it’s teaching finance to think in tokens.”
When pilots become programmes
The transition from pilot to production is where the majority of AI projects fail. The proof-of-concept stage is intoxicating: a small success, fast turnaround, and the sense of being on the frontier. But scaling responsibly — with compliance, monitoring, and cost controls — is where most initiatives stall.
“We’re learning to be disciplined. You can’t just build it and move on. You have to measure the benefit, report it, and be honest about the cost of keeping it that way,” said Crichton.
Connolly warned that companies often overspend by defaulting to the most powerful models.
The instinct to buy the “Rolls-Royce” version of AI, he said, ignores the reality that many business problems don’t need frontier-scale models. Smaller models can deliver the same outcome at a fraction of the cost, especially once token use and energy draw start adding up.
He also predicted that CFOs will become key in AI strategy in Financial Services and Insurance as usage grows. Executives will start asking hard questions about value, efficiency, and how much was spent versus how much was saved.
“People that are just throwing things against the wall and see what’s going to stick, and it would be interesting to see that though the lens of a CFO.”
But he is also optimistic about the prospects for human workers even in an AI-driven world.
“My personal opinion is that if you’re going to lose your job it is because you did not evolve yourself as a professional. If you start using this new wave of technology as a tool, and you become proficient with it, no one’s going to take your job away,” he said.
“The differentiation will be the ones who are curious, who roll up their sleeves, test, and make it a better practice within their day-to-day, versus the ones who just watch the ducks fly.”
