The argument for on-premise AI in India is usually made in the language of compliance and answered in the language of cost. A bank's compliance head says customer data cannot leave the country, or the premises, or the bank's own key management. The chief financial officer asks what a rack of GPUs costs against a monthly API bill, and the conversation stalls because nobody in the room has a unit-cost model that puts both on the same line.
This article builds that model. It takes a published GPU-hour price from the IndiaAI Mission, a published commercial GPU-hour price, and published API token prices from a frontier provider's pricing page, states every assumption about throughput, utilisation and team cost, converts to rupees at ₹95 to the US dollar, and produces a cost per million tokens for each path at several annual volumes. Every figure that is not a published price is labelled as an assumption, and the model is illustrative: it is a way of thinking, not a quotation.
The conclusion is less flattering to either camp than their advocates would like. At the volumes most Indian enterprises actually run, on-premise inference is more expensive per token than API access to a comparable small model, and the gap is dominated not by GPUs but by people. On-premise becomes the cheaper path only at high, well-utilised volume against mid-tier API pricing, or when the fixed cost of the platform team is already being paid for other reasons. For regulated enterprises the decision is therefore a compliance decision first and an economics decision second, and the economics tell you how to make the compliance decision affordable rather than whether to make it.
What on-premise AI in India means in 2026, and why regulated sectors ask for it
On-premise, in this article, means inference that runs on hardware the enterprise controls, whether in its own data centre, a colocation facility in India, or a dedicated private cloud tenancy where the enterprise holds the keys and the model weights. It excludes the public API, in which prompts and documents travel to a provider's infrastructure. The grey zone, a hyperscaler's India region running a provider's model under the enterprise's account, is a separate case that this model does not price.
The regulatory pull towards on-premise AI in India comes from several directions. The Reserve Bank of India's circular of 6 April 2018 requires payment-system data to be stored in India. SEBI's framework for adoption of cloud services, issued on 6 March 2023, governs how regulated securities-market entities may use cloud infrastructure. The Digital Personal Data Protection Act 2023 and its Rules, notified on 14 November 2025 with an 18-month phased window, place additional obligations on significant data fiduciaries, including local storage of specified categories of data, and Rule 15 governs cross-border transfers. The RBI's FREE-AI committee report of August 2025 goes further in direction if not yet in force: among its 26 recommendations under six pillars are indigenous financial-sector AI models and sector data infrastructure. These are recommendations, not binding regulation, but a bank reading them knows which way its examiner is leaning.
None of these instruments says "AI must run on premises". Each narrows the set of acceptable architectures, and the narrowing compounds: a bank that is a significant data fiduciary, subject to payment-data localisation, and anticipating FREE-AI's direction, finds that the simplest architecture that satisfies all three is one in which the model runs where the data already is. That is the demand side. The supply side is that open-weight models in the 7 to 8 billion parameter class, and increasingly larger ones, now perform well enough on bounded enterprise tasks (classification, extraction, retrieval-augmented question answering) that running them locally is a real option rather than a compromise. The question is what it costs.
The unit-cost model: inputs and sources
The model has five inputs. Two are published prices, one is a published programme rate, and two are stated assumptions.
Two caveats govern everything that follows. First, the comparison is of the cost of tokens, not of equivalent quality: an 8-billion-parameter open model served locally is not the same product as a frontier API model, and the enterprise's task determines which model class is adequate. The model therefore compares on-premise small-model cost against both the small and the mid API tiers, and the reader chooses the relevant line. Second, the IndiaAI programme rate is a subsidised rate within a government mission; whether a particular enterprise can obtain capacity at that rate depends on the Mission's terms, and most private deployments should plan on the commercial or committed rate.
Step one: GPU-only cost per million tokens
At full load one H100 produces 3.6 million tokens per hour under the throughput assumption. Dividing the hourly rate by the tokens actually produced at each utilisation gives the GPU-only cost per million tokens.
For comparison, the API prices convert as follows. A blended price is shown for a 3:1 input-to-output token ratio, which is typical of retrieval-augmented workloads where long context is read and short answers written; the ratio is an assumption.
The first lesson is already visible. GPU-only on-premise cost for a small model, at ₹28 to ₹351 per million tokens, brackets the small API tier at ₹19 only at the very best case (programme rate, 90 per cent utilisation) and otherwise exceeds it. Against the mid API tier at ₹380, on-premise GPU cost is lower in every cell except commercial on- demand at 30 per cent utilisation, where the two are roughly equal. Utilisation is the swing variable: moving from 30 to 90 per cent utilisation cuts cost per token by two-thirds, which is a larger effect than the difference between the programme rate and the commercial rate at the same utilisation.
Step two: adding the people
GPU-only cost is the figure on-premise advocates quote and the one that misleads. The hardware does not run itself. Someone must serve the model, patch the stack, manage keys, monitor drift, handle the retrieval pipeline and answer the auditor. The model assumes a platform fixed cost of ₹1.5 crore a year for a small dedicated team and supporting infrastructure, and it assumes GPUs are rented continuously (the enterprise pays for the hour whether or not a token is produced), sized at 60 per cent sustained utilisation with a minimum of two GPUs for redundancy. Annual capacity per GPU at 60 per cent is 8,760 hours multiplied by 2.16 million tokens, or about 18.9 billion tokens.
The table resolves the CFO's question. Against the small API tier, on-premise inference does not win on cost at any volume in the table; the fixed cost of the team alone exceeds the entire API bill until volumes far beyond what most enterprises run. Against the mid API tier, break-even sits at roughly 45 billion tokens a year at the programme rate and roughly 100 billion at the commercial on-demand rate. Algebraically, break-even volume equals the on-premise annual cost divided by the API blended price: at the programme rate with three GPUs, about ₹1.74 crore divided by ₹380, or about 46 billion tokens; at the commercial rate the GPU rental itself is large enough that the fully loaded cost per million only falls below ₹380 somewhere above 100 billion tokens, at seven or more GPUs. Below those volumes, API is cheaper; above them, on-premise is, and the advantage widens as volume grows because the team cost is spread across more tokens. Note how far the fully loaded figures sit from the GPU-only figures in the previous table: at 80 billion tokens on the programme rate, the GPU accounts for ₹43 of the ₹238 per million and the team for the rest.
For scale, 10 billion tokens a year is about 27 million tokens a day, which is on the order of 20,000 to 30,000 retrieval-augmented queries a day with long contexts. A large bank's internal knowledge assistant might reach it; a mid-sized manufacturer's maintenance copilot will not. Most Indian enterprise deployments in 2026 sit in the first two rows of the table, where the economics favour API by a wide margin and on-premise can be justified only on grounds other than cost.
Where the model is most sensitive
Four assumptions move the result most, and a reader applying the model should test each against their own case.
The platform team. The ₹1.5 crore fixed cost is the dominant term at low volume. If the enterprise already runs a data platform team whose marginal cost of adding model serving is a fraction of this, or if the team is shared across many AI workloads, the fixed cost per workload falls and on-premise break-even moves sharply left. This is the strongest economic argument for on-premise in large institutions: the team is already there.
Model size. The throughput assumption is for an 8-billion-parameter-class model. A 70-billion-parameter-class model needs several times the compute per token, roughly multiplying the GPU-only cost per million by four to eight depending on quantisation and hardware. If the task requires the larger model, the on-premise GPU column rises accordingly while the API column, for a comparable capability, is already the mid or flagship tier.
Utilisation. A single-tenant on-premise deployment serving one business unit during Indian office hours will struggle to exceed 30 per cent utilisation. Pooling workloads across units, running batch jobs overnight and routing low- priority traffic to fill idle capacity are the operational levers that make 60 per cent achievable, and each requires a gateway that can route and queue, which is part of the platform the fixed cost pays for.
API discounts. The pricing page shows Flex and Batch tiers at half the standard price and cached-input prices at a tenth or less of standard input. A workload that tolerates latency and reuses long system prompts can cut its API blended cost well below the figures above, moving the break-even further right.
The compliance variable, priced
The model so far prices tokens. The regulated enterprise's actual question is the cost of compliance under each architecture, and the model can be extended to answer it by treating compliance as a fixed cost that differs by path.
Under an API architecture, the enterprise must establish that the provider's data handling, residency, retention and sub-processor arrangements satisfy its regulator; must maintain the vendor-risk file; and must accept that a change in the provider's terms or a regulator's interpretation can make the architecture non-compliant on a date the enterprise does not control. The cost is low in cash and high in residual risk.
Under an on-premise architecture, the enterprise bears the cost of securing the stack itself but holds the evidence: the data never left, the keys are its own, the model weights are versioned and auditable. The cost is high in cash, which the model has already counted in the platform team, and low in residual risk.
The honest comparison therefore adds the expected cost of a compliance failure to the API column. For a payments business under the 2018 circular or a significant data fiduciary under the DPDP Rules, that expected cost can be large enough to decide the question regardless of the token economics. This is why the decision is a compliance decision first: the economics determine how to make the compliant architecture affordable, principally by sharing the platform team and raising utilisation, rather than whether to choose it.
Practice: how on-premise is being made affordable
Two patterns are visible among builders of on-premise enterprise AI, and PanScience's portfolio offers company- reported examples of each. Eigen, described by PanScience as on-premise AI for regulated industries, and Insituate AI, described as an on-premise enterprise RAG and agent platform, both address the fixed-cost problem from the supply side: a platform that arrives with serving, retrieval, security and monitoring already built replaces a part of the team the model prices at ₹1.5 crore, which is the term that dominates at low volume. Proteus, described as an LLM gateway that routes, secures and monitors, addresses the utilisation problem: a gateway that routes each request to the cheapest compliant destination, local for regulated data and API for everything else, lets the on-premise capacity run full while the long tail of general tasks goes to the API tier that the model shows is cheaper. These are descriptions of the product categories from panscience.xyz, not performance claims, and the article makes no statement about any client.
The broader lesson is that the pure on-premise and pure API architectures are both economically wrong for most regulated Indian enterprises. The architecture the model favours is hybrid by data class: regulated data to local inference on shared, well-utilised capacity, everything else to the API, with a gateway enforcing the split and producing the audit trail. The IndiaAI Mission's compute, at ₹65 to ₹92 per GPU-hour where an enterprise can access it, lowers the local column further, but the model shows that even at that rate the people, not the GPUs, decide the result.
Conclusion
On-premise AI in India is cheaper than the API only under conditions most enterprises do not meet: tens of billions of tokens a year, utilisation above half, a platform team whose cost is shared or already sunk, and a comparison against mid-tier rather than small-tier API pricing. Below those thresholds the API wins on cost, usually by a wide margin, and the GPU rate, whether ₹92 under the IndiaAI Mission or ₹379 on the commercial market, is not the deciding term at the volumes most enterprises run; it moves the break-even from about 45 billion to about 100 billion tokens, but the team cost is what puts both numbers out of reach. What decides the question for regulated enterprises is the expected cost of non-compliance, which the token model cannot see but a compliance head can. The right use of the model is therefore not to choose between on-premise and API but to design the hybrid that keeps regulated data local at the lowest achievable fixed cost and highest achievable utilisation, and sends everything else to the cheaper line.
Methodology and sources
This article presents an illustrative unit-cost model. Observed inputs are: the IndiaAI Mission's subsidised GPU rate of ₹65 per hour and 38,000-GPU target, from the PIB backgrounder "Transforming India with AI", 12 October 2025 (https://static.pib.gov.in/WriteReadData/specificdocs/documents/2025/oct/doc20251012664501.pdf), with the ₹10,371.92 crore outlay; the H100 rate of ₹92 per GPU per hour and 34,381 GPUs from 14 empanelled providers, from IANS via Morung Express, 24 July 2025 (https://morungexpress.com/indiaai-mission-34381-gpus-onboarded- from-14-empanelled-service-providers); the commercial on-demand H100 SXM price of $3.99 per hour from Lambda's public pricing page, read 25 October 2026 (https://lambda.ai/pricing); and API token prices for gpt-6-luna, gpt-6.1-sol and gpt-6-astra at the standard tier from OpenAI's developer pricing page, read 25 October 2026 (https://developers.openai.com/api/docs/pricing). Prices change; readers should re-open both pricing pages before using the figures.
Assumed inputs, all illustrative and stated in the text, are: throughput of 1,000 tokens per second per H100 for an 8- billion-parameter-class model; utilisation of 30, 60 and 90 per cent; a committed GPU rate of ₹200 per hour; a platform fixed cost of ₹1.5 crore per year; a 3:1 input-to-output token ratio; continuous GPU rental sized at 60 per cent utilisation with a two-GPU minimum; and an exchange rate of ₹95 per US dollar. The break-even and cost-per- million figures follow arithmetically from these inputs and change with them. Regulatory references (RBI circular of 6 April 2018, SEBI cloud framework of 6 March 2023, DPDP Rules notified 14 November 2025, RBI FREE-AI committee report of August 2025) are as held in the PanScience Research editorial register; the FREE-AI report's contents are recommendations, not binding regulation. PanScience venture descriptions (Eigen, Insituate AI, Proteus) are company-reported from panscience.xyz; no client is named and no performance or deployment claim is made.