Two years ago the question an Indian CIO asked about language models was which frontier API to buy. The question in 2026 is harder: whether to use a frontier API at all for a given workflow, or to run a smaller, often open-weight model inside the company's own perimeter, tuned to its own documents. The case for small language models in enterprise deployments is usually argued on cost. That argument is weaker than it sounds, and the real argument is elsewhere.
Three facts from the Stanford AI Index 2025 frame the decision. In 2022 the smallest model scoring above 60 per cent on the MMLU benchmark was PaLM, at 540 billion parameters; by 2024 Microsoft's Phi-3 Mini reached the same threshold with 3.8 billion, a 142-fold reduction in two years. The cost of querying a model at GPT-3.5-level MMLU performance fell from $20.00 per million tokens in November 2022 to $0.07 per million tokens by October 2024, a more than 280-fold reduction. And the gap between the best closed-weight model and the best open-weight model on Chatbot Arena narrowed from 8.04 per cent in January 2024 to 1.70 per cent by February 2025.
Read together, those numbers say something uncomfortable for both camps. Small models have become good enough for most bounded enterprise tasks. But the frontier vendors have cut their own prices so hard that beating them on raw token price with self-hosted hardware is not straightforward. This article sets out a five-condition fit test for when a small or domain-specific model wins, with the assumptions stated, and where it loses.
What "small" means, and why domain-specific is a different axis
There is no agreed definition of a small language model. In practice the term bundles three separate properties that enterprises should keep apart.
The first is parameter count. Models in the 1 to 15 billion range run on a single data-centre GPU, and the smallest on a laptop or phone. The Phi-3 technical report describes phi-3-mini as "a 3.8 billion parameter language model trained on 3.3 trillion tokens" that "achieves 69% on MMLU and 8.38 on MT-bench", rivalling "models such as Mixtral 8x7B and GPT-3.5" while being "small enough to be deployed on a phone". The Phi-4 report, released in December 2024, describes a 14 billion parameter model that "substantially surpasses its teacher model on STEM-focused QA capabilities", the teacher being GPT-4.
The second property is domain specialisation. BloombergGPT, published in March 2023, was a 50 billion parameter model trained on 363 billion tokens of Bloomberg's financial data plus 345 billion tokens of general text; the authors report it "outperforms existing models on financial tasks by significant margins" without losing general performance. Fifty billion parameters is not small, but the model is specialised.
The third is deployment control: whether the weights are available to run inside the enterprise's own infrastructure. This is the property that regulated Indian sectors care about most, and it is independent of size. A 70 billion parameter open-weight model self-hosted in a bank's data centre gives more control than a 3 billion parameter model reached through a third-party API.
An Indian example makes the point. OnFinance AI, a PanScience Innovations venture, describes NeoGPT on its website as a "BFSI-specific finetuned LLaMA 3.3 70B", supporting regulators including SEBI, RBI, IRDAI, AMFI, the US SEC and FINRA, the UK FCA, Saudi Arabia's SAMA and Singapore's MAS, with on-premises and private- cloud deployment options. PanScience reports 70+ agents across 40+ regulators. All of this is company-reported. The relevant point for a buyer is that the model is not small by parameter count; its advantage is domain tuning and the fact that it can run where the compliance data already sits.
The lesson of the table is that "small" and "domain-specific" are different levers. An enterprise can pull one, both or neither.
The cost argument, with the assumptions stated
The usual pitch for small language models in enterprise settings is that tokens are cheaper when you run the model yourself. Whether that is true depends on volume, utilisation and which API tier you are comparing against.
Consider an illustrative calculation using one public Indian compute price. The IndiaAI Mission has reported 38,000+ GPUs available at about ₹65 per hour. Assume a 7 billion parameter class model serving a bounded task. Throughput on a single modern GPU depends heavily on batching: a lightly loaded server might produce around one million tokens per hour, a well-batched one several times that. Taking a range of one to five million tokens per GPU hour, the compute cost of self-hosting lands between ₹13 and ₹65 per million tokens, or roughly $0.14 to $0.68 at ₹95 to the dollar. These are assumptions, not measurements, and they exclude the engineering, evaluation and operations staff who make the deployment work.
Set that against the AI Index figure of $0.07 per million tokens for GPT-3.5-level performance from a hosted small model in October 2024. On raw token price, the cheapest API tier is competitive with, and often below, a self-hosted deployment at realistic utilisation. Epoch AI's analysis, published in March 2025, found that the price of constant performance has fallen at rates "ranging from 9x to 900x per year" depending on the benchmark, with the caveat that "the fastest price drops in that range have occurred in the past year", so the trend may not hold.
This yields the first honest conclusion: if the only reason to run a small model is to save on tokens, the case is fragile. The frontier vendors have a cost curve of their own, and they are riding it down fast. The durable reasons are different, and they are the subject of the fit test.
Small language models for enterprise: a five-condition fit test
The test below is an original framework drawn from the sources above and from PanScience's own practice building on-premise AI for regulated clients. Each condition has an observable test, so that a CIO can check it against a real workflow rather than argue in the abstract. The working rule is that a small or domain-specific model is the right default when at least three of the five conditions hold, and the API is the right default when fewer than three do.
Three of these deserve expansion.
Condition 1 is the most important and the most often skipped. The AI Index's 142-fold figure is a statement about a general benchmark. The gap between small and frontier models on a narrow, well-specified task is usually smaller still, and sometimes inverted after fine-tuning. The gap on open-ended reasoning, multi-step planning or tasks that shift week to week remains large. A procurement team classifying invoices into 40 categories is in the first world; a research desk asking unstructured questions of unstructured data is in the second.
Condition 2 is where the Indian case is strongest. The RBI's April 2018 circular requires that "the entire payment data shall be stored in systems located only in India", with processing abroad permitted only if the data is brought back and deleted overseas within one business day or 24 hours, whichever is earlier. SEBI's March 2023 cloud framework requires that regulated entities' data storage and processing is conducted within MeitY-empanelled data centres. The DPDP Rules notified on 14 November 2025 give significant data fiduciaries additional obligations including annual impact assessments and audits and local storage of specified categories, phased in over 18 months. None of these forbids a hosted API in all cases, but each raises the compliance cost of one, and a self-hosted model removes the question. A separate PanScience Research piece in this series maps the full sovereign stack against these requirements.
Condition 5 is the one that kills more small-model projects than any other. An API is a vendor's problem to run. A self-hosted model needs an evaluation harness, a retraining schedule, a vulnerability process and a rollback path. EY India's Outlook 2026 survey of over 200 Indian enterprise leaders found 91 per cent rank speed of deployment as the top factor in buy-versus-build decisions; the Menlo Ventures 2025 enterprise survey found 76 per cent of respondents buying rather than building. Those preferences are rational when condition 5 is not met. Where it is met, the economics of conditions 2 to 4 start to pay.
Where small models lose
Honesty about the failure cases matters as much as the fit test.
Small models lose on open-ended reasoning and on tasks whose definition keeps moving. The AI Index records that OpenAI's o1 reasoning model was "nearly six times more expensive and 30 times slower than GPT-4o", and enterprises chose it anyway for hard problems, because a cheaper wrong answer has no value. Where a task needs that kind of capability, a small model is a false economy.
They lose at low volume. If a workflow generates ten thousand tokens a day, the fixed cost of standing up and operating a model is never recovered, and the data residency question is better solved by contract and a private endpoint than by hardware.
They lose when the organisation lacks evaluation discipline. A fine-tuned model is only as good as the test set that measured it. Without a held-out evaluation that mirrors production, a team cannot know whether the small model is adequate, and will discover the answer from users.
And they lose when the choice is made for identity rather than fit. "We run our own models" is a satisfying sentence for a technology leader to say; it is not a business case.
What the Indian supply side now offers
Two things have changed on the Indian supply side since 2024 that make the fit test more often pass.
The first is compute and data. The IndiaAI Mission, with an outlay of ₹10,371.92 crore, reports 38,000+ GPUs at around ₹65 per hour, and the AIKosh platform lists 7,541 datasets and 273 models. The Mission has selected 12 foundation-model startups, including BharatGen and Sarvam. For an enterprise, this lowers the fixed cost of experimenting with condition 3 and condition 4 deployments, and makes Indic-language fine-tuning a practical option rather than a research project.
The second is the maturity of the open-weight tier. The narrowing of the open-versus-closed gap to 1.70 per cent on Chatbot Arena means the capability ceiling of a self-hostable model is now close to the frontier for general tasks, and the parameter range below 15 billion covers most bounded ones. The RBI's FREE-AI committee report of August 2025 explicitly recommends the development of "indigenous financial sector-specific AI models", including LLM, SLM and non-LLM options, to be offered as a public good. That is a recommendation rather than binding regulation, but it signals the direction of regulatory thinking.
The practice pattern PanScience reports across its on-premise ventures, Eigen for regulated industries and Insituate AI for enterprise retrieval and agents, is consistent with the fit test: the deployments that hold are those where the data could not move, the task was bounded, and the client had or hired an owner for the model lifecycle. Where any of those was missing, a hosted model with a strong contract served better.
Conclusion
The decision between small language models and frontier APIs for enterprise use is not a technology preference. It is a fit question with five observable conditions: task boundedness, data residency, domain distance, volume and latency, and operating capacity. Three or more and the small or domain-specific model is the right default. Fewer and the API is.
The cost argument alone does not decide it. Hosted prices for GPT-3.5-level capability have fallen more than 280-fold in two years, and an honest self-hosting calculation at realistic utilisation often cannot beat them. What self-hosting buys is control: over where data sits, over what the model has learned, over latency, and over the vendor relationship. For Indian banks, exchanges, hospitals and plants under RBI, SEBI, DPDP and sector rules, that control has a price, and increasingly a regulatory floor.
The practical advice is to run the test workflow by workflow, write the assumptions down, and expect the answer to differ across the same company. Most large Indian enterprises will end up with both: frontier APIs for open-ended and low-volume work, and a handful of small or domain-tuned models for the bounded, high-volume, data-sensitive core. The operating model that manages both is the subject of a separate piece in this series.
Methodology and sources
This article combines published benchmark and cost data with an original fit framework. The five-condition test and the three-or-more rule are the authors' analytical contribution, informed by PanScience Innovations' company- reported practice; they are a decision heuristic, not a measured result. The self-hosting cost range (₹13 to ₹65 per million tokens) is illustrative and rests on stated assumptions: the IndiaAI Mission's reported price of about ₹65 per GPU hour, a 7 billion parameter class model, one to five million tokens per GPU hour depending on batching, and a conversion of ₹95 to the dollar. It excludes staff and overhead and should be replaced with measured throughput before any purchasing decision.
Benchmark and cost figures come from the Stanford AI Index 2025, Chapter 1, Research and Development (hai.stanford.edu/assets/files/hai_ai-index-report-2025_chapter1_final.pdf) and Chapter 2, Technical Performance (hai.stanford.edu/assets/files/hai_ai-index-report-2025_chapter2_final.pdf), opened 25 October 2026. Model details are from the Phi-3 Technical Report (arxiv.org/abs/2404.14219), the Phi-4 Technical Report (arxiv.org/abs/2412.08905) and the BloombergGPT paper (arxiv.org/abs/2303.17564). Inference price trends are from Epoch AI, "LLM inference prices have fallen rapidly but unequally across tasks", 12 March 2025 (epoch.ai/data- insights/llm-inference-price-trends). Regulatory references are the RBI FAQ on storage of payment system data (rbi.org.in/Scripts/FAQView.aspx?Id=130), the SEBI framework for adoption of cloud services, 6 March 2023, read via Taxmann's summary (taxmann.com/post/blog/sebi-introduces-framework-for-adoption-of-cloud-services-by- regulated-entities/), the PIB backgrounder on the DPDP Rules 2025 (static.pib.gov.in/WriteReadData/specificdocs/documents/2025/nov/doc20251117695301.pdf) and the RBI FREE-AI committee report (rbi.org.in/scripts/PublicationReportDetails.aspx?ID=1306). Survey figures are from EY India, The AIdea of India: Outlook 2026 (ey.com/en_in/insights/ai/agentic-ai-india/aidea-survey-findings) and Menlo Ventures, State of Generative AI in the Enterprise 2025, both as verified by the editor. IndiaAI Mission figures are from the PIB backgrounder of February 2026. OnFinance AI details are company-reported from onfinance.ai and the PanScience Innovations fact sheet; no client names or deployment figures beyond that sheet are stated.