Most enterprises discover their AI bill the way households discover a mobile data overage: after the fact, as a single number with no story attached. A pilot that cost little in its first month becomes a production system whose spend tracks usage, prompt length, retries and the habits of every developer who touches it. AI cost optimisation is the discipline of putting that story back into the number, so that spend can be explained, forecast and tied to a business result before it becomes a line item that the finance team wants cut.
The problem is not that models are expensive. On a per-token basis they have become dramatically cheaper. The problem is that the cost of an AI system is the product of a falling unit price and a rising, poorly observed quantity, and most organisations only manage the first term. The quantity side, which covers how many tokens each task consumes, how often a request is repeated, which model handles it and whether anyone needed it at all, is where the money goes and where the controls are.
This piece sets out why AI spend behaves differently from conventional cloud spend, how an inference bill is actually built, and a table of cost levers with illustrative impact ranges derived from published pricing mechanics. It is written for the CIO, CTO or finance partner who has to sign off the next phase of an AI programme and wants to know which questions to ask first.
Why AI spend behaves differently from cloud spend
The FinOps Foundation, the industry body that codified cloud financial management, now treats AI as a distinct cost domain. Its State of FinOps 2026 survey of 1,192 respondents, representing more than $83 billion in annual cloud spend, found that 98% of practitioners now manage AI spend, up from 63% in 2025 and 31% in 2024. FinOps for AI was named the top forward-looking priority, and the most common challenge reported was visibility into AI costs. The most requested tooling feature was granular monitoring of AI spend across tokens, LLM requests and GPU utilisation.
The Foundation's FinOps for AI overview explains why visibility is hard. The basic equation of price multiplied by quantity still holds, but the meters differ by provider, SKUs change frequently, and costs can rise after preview or trial periods end. Spend arrives through several channels at once: token-billed API calls, GPU compute for self-hosted or fine-tuned models, storage and data transfer, managed services priced by call or volume, and third-party licences. A single product feature can draw on four of these.
Three behaviours make AI spend harder to govern than a virtual machine fleet. First, cost is set at design time by people who rarely see the bill: a prompt engineer who adds two thousand tokens of instructions to every call has made a recurring financial decision. Second, consumption is driven by end users and, increasingly, by agents that call models in loops, so the quantity term can multiply without any change in headcount or infrastructure. Third, quality and cost are coupled. Switching to a cheaper model can save money while quietly raising error rates, which then generates retries, escalations and human rework that never appear on the AI line.
Falling unit prices, rising bills
The Stanford AI Index 2025 reports that the inference cost of a system performing at the level of GPT- 3.5 on a standard benchmark fell from $20.00 per million tokens in November 2022 to $0.07 per million tokens by October 2024, a reduction of more than 280-fold in about 18 months. At the hardware level, the same report finds costs declining by 30% per year and energy efficiency improving by 40% annually.
These are real gains, and they are the reason many AI business cases now clear their hurdle rates. They do not, however, guarantee falling bills. Cheaper tokens invite more tokens: longer context windows, retrieval that stuffs more documents into each prompt, multi-step agents that call a model a dozen times per task, and features rolled out to every employee rather than a pilot group. An organisation that budgets on the assumption that unit prices will keep falling, without governing quantity, is likely to find that its total spend rises anyway.
The practical reading is that the unit-price curve is a tailwind to be captured, not a strategy. Capturing it requires an architecture that can move workloads to cheaper models as they appear, which is an argument for abstraction layers rather than hard-coded model calls.
The anatomy of an inference bill
For a token-billed workload, the monthly cost of a single feature can be written as a simple model. Stated assumptions: the provider bills input and output tokens separately, output tokens are priced higher than input tokens (as is common across major providers), and some requests are retried or repeated.
Monthly cost = requests × (1 + retry rate) × [(input tokens × input price) + (output tokens × output price)]
Each term is a lever. Requests can be reduced by not calling a model when a rule, a search or a cached answer would do. The retry rate falls when prompts are robust and outputs are validated. Input tokens are inflated by long system prompts, verbose retrieval and conversation history that is resent on every turn. Output tokens are inflated by unconstrained generation. The two prices depend on which model is used, whether the request is synchronous or batched, and whether repeated prompt content is cached.
For self-hosted models, the equation changes form but not logic. Cost becomes GPU hours multiplied by hourly rate, divided by useful throughput, so utilisation becomes the dominant variable. The FinOps Foundation's own illustrative metrics include cost per inference, cost per token and GPU utilisation; a fleet running at 40% utilisation costs roughly twice as much per useful token as the same fleet at 80%. The model also shows why a single cost-per-call figure misleads. Two features with identical call volumes can differ in cost by an order of magnitude because one sends a short query and receives a one-line answer while the other sends a forty-page contract and asks for a full redline.
AI cost optimisation levers and their impact
The table below is the core analytical contribution of this piece. It lists the levers available to an enterprise, the mechanism by which each reduces cost, and an illustrative impact range on the cost of the workload to which the lever applies. The ranges are not survey findings. Where a provider publishes a pricing rule, the range is derived arithmetically from that rule under stated conditions; where no pricing rule exists, the range is a judgement labelled as such. Impacts do not add linearly, because each lever acts on a cost base already reduced by the others.
The caching arithmetic illustrates how a published pricing rule becomes an impact range. Assume a workload in which 80% of input tokens per request are a stable prefix and 20% are unique. Without caching, input cost is 1.0 unit. With caching, and once the cache is warm, the cost is 0.2 for the unique portion plus 0.8 multiplied by 0.1 for the cached portion, or 0.28 units, a reduction of 72% on input. The cache write premium of 1.25 times, paid when the prefix is first written or after it expires, erodes this saving when traffic is sparse. At a 50% shared prefix the warm-cache saving falls to 45%. Anthropic's documentation also states that cache multipliers stack with the batch discount, so a cached, batched workload compounds both.
Routing deserves particular care because it is where the largest savings and the largest hidden quality losses sit together. A router that sends 70% of traffic to a model one-tenth the price of the frontier tier cuts cost on that traffic by 90%, but only if the evaluation confirms the cheaper model is good enough for that slice. A separate PanScience Research piece covers evaluation before deployment; the same discipline applies to every routing rule.
A worked illustration
Consider an internal assistant for a compliance team. This is an illustrative model, not an observed deployment. Index its baseline monthly cost at 100, split 60 for input tokens and 40 for output tokens, with every request going to a single frontier model synchronously.
Step one: move the overnight document classification jobs, assumed to be 30% of spend, to batch. Saving: 15, leaving 85.
Step two: cache the long policy prefix that every request carries. Assume this applies to the remaining input cost and achieves the 45% warm-cache saving calculated above for a half-shared prefix. Input cost on the synchronous and batch paths combined is now about 51 of the 85; a 45% saving on that is about 23, leaving roughly 62.
Step three: route simple lookups, assumed to be half of remaining traffic, to a model one-fifth the price. Saving on that half: 80%, or about 25, leaving roughly 37.
Step four: cap output length on the remaining frontier traffic, saving an assumed 20% of its output cost, worth perhaps 2 to 3 points.
The illustrative result is a bill between a third and two-fifths of the baseline, with no change to user- facing functionality, provided each change passes evaluation. Change the assumptions and the result moves: a workload with little batchable work, short prompts and no simple traffic might see savings of 15% to 25%. The value of the exercise is less the final number than the discipline of attaching each saving to a specific mechanism that can be tested.
Governance: who owns the meter
Levers are engineering choices; using them consistently is an organisational one. The FinOps Foundation's guidance converges on a few practices that translate directly to AI.
Tagging and showback come first. Every model call should carry metadata for team, product, environment and use case, so that spend can be attributed. Showback, which makes teams aware of their costs without charging them directly, is usually the right starting point because it changes behaviour without the political cost of internal billing.
Baselines come next. The Foundation recommends establishing cost and functionality baselines from past invoices, usage reports and measures such as request volumes and accuracy. Without a baseline, an optimisation cannot be distinguished from a drop in usage.
Unit economics turn cost into a business conversation. The Foundation's illustrative metrics include cost per inference and cost per API call. More useful for an executive is cost per business outcome: per resolved ticket, per contract reviewed, per claim processed. A separate PanScience Research piece sets out a unit-cost model for enterprise AI; the point here is that cost per outcome is the metric that survives a model change, a price change or a vendor change.
Maturity sets the tolerance. The Foundation's crawl, walk, run model accepts fast and loosely governed experimentation early, within cost limits, and shifts to tracking, anomaly analysis and return on investment as workloads mature. Applying production controls to a two-week experiment wastes effort; failing to apply them to a feature used by ten thousand employees wastes money.
Finally, the State of FinOps 2026 notes that many organisations are being asked to fund AI investments from optimisation savings elsewhere. That makes AI cost optimisation a funding mechanism as well as a control, which raises the stakes on getting attribution right.
Where the control point sits: gateways and on-premise capacity
Most of the levers in the table require a place in the architecture where every model call passes through and can be inspected, routed, cached, limited and logged. That place is an LLM gateway. A separate Cluster 4 article on LLM gateways and AI control planes covers the capability map in detail; for cost purposes the essential functions are per-request cost attribution, routing rules that can be changed without redeploying applications, caching, and budget enforcement.
Within the PanScience portfolio, Proteus is described as an LLM gateway that routes, secures and monitors model traffic (company-reported). The relevance here is architectural rather than promotional: without a gateway of some kind, whether bought, built or open source, routing and quota levers have to be reimplemented in every application, and most teams will not bother.
The second structural decision is where inference runs. For steady, high-volume, predictable workloads, owned or reserved capacity can undercut per-token pricing, but only at high utilisation. In India, the IndiaAI Mission has made more than 38,000 GPUs available at about ₹65 per hour, according to the PIB backgrounder of February 2026, which changes the arithmetic for organisations that qualify. For regulated industries the decision is often driven by data residency as much as cost. PanScience reports that its venture Eigen provides on-premise AI for regulated industries. A separate PanScience Research piece covers on-premise economics in depth, including the utilisation thresholds at which owned capacity pays back; the cost-optimisation point is simply that the API-versus-owned decision should be revisited as volumes stabilise, not made once at pilot stage.
What to do in the next ninety days
A practical sequence for an enterprise that already has AI in production:
1. Instrument before optimising. Ensure every call is attributed to a team and use case, and that input tokens, output tokens, model, latency and cost are logged. 2. Rank workloads by spend. In most portfolios a small number of features account for most of the bill; optimise those first. 3. Apply the mechanical levers. Batch whatever can wait, cache stable prefixes, and cap output lengths. These rely on published pricing rules and carry the least quality risk. 4. Evaluate, then route. Build a test set for each high-spend workload and trial cheaper models against it before changing production routing. 5. Set budgets and alerts. Per-team quotas and anomaly detection catch runaway agents and test scripts before month end. 6. Report cost per outcome. Move the executive conversation from total spend to cost per resolved task, compared with the cost of doing the task without AI.
Conclusion
AI cost optimisation is less about finding a cheaper model than about making cost visible at the point where it is created. Unit prices are falling fast, and enterprises that architect for portability will capture that decline. But the quantity of tokens consumed, driven by prompt design, retrieval habits, agent loops and user growth, can outrun any price cut. The levers that control quantity are well understood and, in the case of batch and caching, priced explicitly by providers. What most organisations lack is the attribution and governance to use them consistently. Building that capability early, while AI spend is still a modest share of technology budgets, is considerably cheaper than retrofitting it after the bill has become the story.
Methodology and sources
This article combines published survey data, provider pricing documentation and an original illustrative model. Observed figures come from the FinOps Foundation State of FinOps 2026 report (data.finops.org), the FinOps Foundation FinOps for AI overview, last updated 17 February 2026 (finops.org/wg/finops-for-ai-overview), the Stanford AI Index 2025, chapter 1 (hai.stanford.edu/assets/files/hai_ai-index-report-2025_chapter1_final.pdf), OpenAI's Batch API guide (developers.openai.com/api/docs/guides/batch), Anthropic's prompt caching documentation (platform.claude.com/docs/en/build-with-claude/prompt-caching) and pricing page (platform.claude.com/docs/en/about-claude/pricing), and the PIB IndiaAI Mission backgrounder of February 2026 as verified by the editor. Provider pricing rules were read on 25 October 2026 and change frequently; readers should confirm current terms. The cost lever table's ranges are illustrative: the batch and caching ranges are derived arithmetically from published pricing rules under the stated assumptions, and all other ranges are the author's judgement, labelled as such. The worked compliance- assistant example is a hypothetical model with stated assumptions, not an observed deployment. Descriptions of Proteus and Eigen are company-reported by PanScience Innovations (panscience.xyz).