An enterprise rarely lacks evidence that an AI system works. The vendor has a benchmark score, the pilot team has a demo, the business sponsor has a satisfied user. What the enterprise usually lacks is evidence that the system works on its own data, for its own task, under its own conditions, with its own failure costs. Enterprise AI evaluation is the discipline of producing that second kind of evidence before a deployment decision, and it is the step most often compressed when a pilot has gone well and a budget cycle is closing.
The cost of skipping it is visible in the aggregate surveys. McKinsey's State of AI 2025 found 88 per cent of organisations using AI regularly in at least one function and about a third scaling, with 39 per cent reporting any EBIT impact and most of those under 5 per cent. EY India's 2026 outlook survey found 47 per cent of Indian enterprise leaders with multiple generative AI use cases live in production but only 10 per cent scaling across the business. Systems that pass a pilot and fail at scale have usually been evaluated on the pilot's conditions and deployed into different ones.
This piece sets out what a pre-deployment evaluation should contain, drawing on the measurement function of the NIST AI Risk Management Framework, the management-system structure of ISO/IEC 42001, the holistic evaluation method developed at Stanford for language models, and the research on data leakage that explains why so many reported results do not reproduce. It closes with an evaluation harness checklist that a CIO, a risk officer or a procurement team can apply to any system, bought or built.
What the frameworks ask for
The NIST AI Risk Management Framework 1.0, released on 26 January 2023 and "intended for voluntary use", organises AI risk management into four functions: Govern, Map, Measure and Manage. Govern "is infused throughout AI risk management" and cultivates a risk culture; Map "establishes the context to frame risks related to an AI system"; Measure "employs quantitative, qualitative, or mixed-method tools, techniques, and methodologies" to analyse, assess, benchmark and monitor risk; Manage allocates resources to mapped and measured risks and plans for incidents. The framework names seven characteristics of trustworthy AI: valid and reliable; safe; secure and resilient; accountable and transparent; explainable and interpretable; privacy-enhanced; and fair with harmful bias managed. NIST added a Generative AI Profile (NIST AI 600-1) on 26 July 2024 and has stated that the framework is being revised.
The Measure function is the one that concerns pre-deployment evaluation, and its four categories are worth quoting because they are the skeleton of a good harness. Measure 1: "Appropriate methods and metrics are identified and applied." Measure 2: "AI systems are evaluated for trustworthy characteristics." Measure 3: "Mechanisms for tracking identified AI risks over time are in place." Measure 4: "Feedback about efficacy of measurement is gathered and assessed." The framework states that AI systems "should be tested before their deployment and regularly while in operation", seeks "objective, repeatable, or scalable test, evaluation, verification, and validation (TEVV) processes", and warns that laboratory measurements "may differ from risks that emerge in operational, real-world settings". Measure 2.3 asks that performance be demonstrated "for conditions similar to deployment setting(s)".
ISO/IEC 42001:2023, published on 18 December 2023 and described by ISO as "the world's first AI management system standard", approaches the same problem from the organisational side. It "specifies requirements for establishing, implementing, maintaining, and continually improving" an AI management system, applies to "organizations of any size involved in developing, providing, or using AI-based products or services", follows the Plan-Do-Check-Act methodology common to management system standards, and offers "an integrated approach to managing AI projects, from risk assessment to effective treatment of these risks". Where NIST tells an evaluator what to measure, ISO 42001 tells an organisation to have a repeatable process for deciding, doing and reviewing the measurement. An enterprise that evaluates each AI system differently, by whichever team happens to own it, can satisfy NIST's Measure function for one system and still have no management system in ISO's sense.
The two frameworks are complementary, and the harness below is aligned to both: each row maps to a NIST Measure category and to the Plan-Do-Check-Act stage where ISO 42001 would expect the evidence to be produced and kept.
Why benchmark scores are not evaluation
The Stanford Center for Research on Foundation Models introduced Holistic Evaluation of Language Models (HELM) in November 2022 with a diagnosis that applies well beyond language models. Before HELM, the team found, models had been evaluated on only "17.9% of our scenarios", a coverage the project raised to 96 per cent by evaluating 30 models from 12 providers on 16 core scenarios and 26 targeted ones, measuring seven metrics for each: "accuracy, calibration, robustness, fairness, bias, toxicity, and efficiency". The project's three principles were "broad coverage and recognition of incompleteness", "multi-metric measurement" and "standardization". Among its findings: average accuracy tracked robustness and fairness, but some scenarios and models showed large drops, and the best adaptation strategy was "scenario- and model-dependent". In some cases human evaluation was essential; model summaries were rated above the reference summaries in two standard datasets, which the authors took as evidence that "better summarization datasets are desperately needed".
Three lessons transfer directly to the enterprise. A single accuracy figure on a public benchmark tells the buyer which model was best at that benchmark, not which will be best at the buyer's task. Accuracy and the other properties can diverge, so a system chosen on accuracy alone may be the least calibrated or least robust of the candidates. And the reference data against which a system is scored may itself be wrong, which means the enterprise's own labelled evaluation set is a product that needs quality control, not a by-product of the pilot.
The second body of research explains why even careful internal evaluations overstate performance. Kapoor and Narayanan, in "Leakage and the Reproducibility Crisis in ML-based Science" (2022), surveyed the literature and found "17 fields where errors have been found, collectively affecting 329 papers", attributing the failures to data leakage, for which they offer "a fine-grained taxonomy of 8 types of leakage that range from textbook errors to open research problems". Their proposed remedy is "model info sheets for reporting scientific claims based on ML models", so that a reviewer can detect leakage before publication. In an enterprise setting the analogue of publication is deployment, and the analogue of the reviewer is whoever signs the go-live. Leakage in enterprise evaluation takes recognisable forms: an evaluation set drawn from the same period as training, so that the model has seen tomorrow's outcome in today's features; duplicate records that appear on both sides of the split; labels that were themselves produced with the help of the model under test; and, for systems built on foundation models, evaluation items that the base model encountered during its pre-training.
The enterprise AI evaluation harness checklist
The checklist is an original synthesis. It is organised into the four NIST Measure categories, with the ISO 42001 Plan-Do-Check-Act stage noted, and each row states what evidence should exist before deployment. "System" means the whole deployed artefact, including prompts, retrieval, rules and human review steps, not the model alone.
The checklist is deliberately long, and a proportionate application will weight the rows by the system's consequence. A document summariser for internal use needs the task definition, the evaluation set, the leakage review and monitoring; it does not need a full red-team. A system that affects credit, employment, health or legal outcomes needs every row, and the slice analysis and human review rows become the ones on which the deployment decision turns.
What an Indian enterprise should add
Two features of the Indian context change the weighting.
The first is language and population. A system evaluated on English inputs from metropolitan users will meet Hindi, regional languages, transliteration, and a range of document quality that the evaluation set did not contain. The slice analysis row is therefore not optional, and the evaluation set should be sampled from the branches, plants or courts where the system will actually run, not from the head office pilot. A separate PanScience Research piece treats Indic- language AI at length; here the point is only that the evaluation set must contain the languages the deployment will.
The second is regulatory expectation. The RBI's FREE-AI committee report of August 2025 lists Understandable by Design, Fairness and Equity, and Accountability among its seven sutras and includes Assurance among its six pillars; its recommendations are not yet binding, but a bank building an evaluation harness now should expect the explainability and fairness rows to be asked about. The Digital Personal Data Protection Rules notified on 14 November 2025 give the privacy and lawful basis row a statutory footing. In health, the ICMR's 2023 guidelines ask for validation on "independently collected datasets other than the training dataset" and for local validation at each deployment site, which is a stricter form of the deployment-conditions row. A separate PanScience Research piece covers AI governance for Indian boards; the harness supplies the evidence that governance will ask for.
The labour of evaluation
The checklist's most expensive rows are not the technical ones. Building a labelled evaluation set from the deployment population, measuring annotator agreement, adjudicating disputes, running a shadow-mode comparison and reviewing slices is labour, much of it skilled domain labour, and it is the labour most frequently cut when a pilot is being hurried into production. The research on leakage is, in effect, a record of what happens when it is cut.
PanScience Innovations reports that two of its ventures work in this layer: Indika AI in data-centric AI and model operations, with a company-reported annotation accuracy of about 98 per cent, and FlexiBench as a managed AI workforce of over 30,000 annotation experts. The company-reported figures are cited only to make a structural observation. An evaluation set is itself a labelled dataset with a measurable accuracy, and whoever produces it, in- house or through a managed workforce, should be able to state that accuracy, because an evaluation against labels of unknown quality is a comparison of two unknowns. Whether to build or buy that labour is a commercial question a separate PanScience Research piece addresses; that the accuracy figure must exist is not.
From checklist to decision
A pre-deployment evaluation should end in one of three outcomes, written down. Deploy, with the monitoring specification live and the re-evaluation triggers recorded. Deploy with constraints, where specific slices or confidence bands are routed to human review until a stated re-evaluation. Or do not deploy, with the rows that failed named, so that the build team knows what would change the decision. The third outcome is the one that an evaluation process exists to make possible. If no system has ever failed an enterprise's evaluation, the enterprise does not have one.
The frameworks agree on the destination even where they differ in method. NIST asks for measurement under conditions similar to deployment and for feedback on whether the measurement worked. ISO 42001 asks for a management system that makes the measurement repeatable across every AI system the organisation runs. HELM shows that one metric is never enough and that reference data can be wrong. The leakage research shows that even careful teams fool themselves without a reviewer. An enterprise that absorbs these four lessons will deploy fewer systems in the next year than one that does not. It will also keep more of them running.
Methodology and sources
This article presents an original evaluation harness checklist aligned to published frameworks; the checklist's rows and their mapping are the author's synthesis and are illustrative. NIST AI RMF 1.0 release date, voluntary status, the Generative AI Profile and the revision statement are from the NIST AI RMF page (https://www.nist.gov/itl/ai-risk- management-framework); the four functions, seven trustworthy characteristics, the four Measure categories, the TEVV statement and Measure 2.3 are quoted from the framework document NIST AI 100-1 (https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf). ISO/IEC 42001:2023 publication date, scope and description are from the ISO standard page (https://www.iso.org/standard/42001); the article does not quote the standard's clauses or Annex A, which are not reproduced on that page. HELM figures and quotations are from the Stanford CRFM announcement "Language Models are Changing AI: The Need for Holistic Evaluation", 17 November 2022 (https://crfm.stanford.edu/2022/11/17/helm.html) and the HELM site (https://crfm.stanford.edu/helm/). The leakage figures and the model info sheets proposal are from Kapoor and Narayanan, "Leakage and the Reproducibility Crisis in ML-based Science", arXiv:2207.07048, July 2022 (https://arxiv.org/abs/2207.07048). ICMR validation expectations are from the "Ethical Guidelines for Application of Artificial Intelligence in Biomedical Research and Healthcare", 2023 (https://www.icmr.gov.in/icmrobject/custom_data/pdf/Ethical- guidelines/Ethical_Guidelines_AI_Healthcare_2023.pdf). McKinsey State of AI 2025 (https://www.mckinsey.com/capabilities/tech-and-ai/our-insights/the-state-of-ai-2025), EY India AIdea of India Outlook 2026 survey findings (https://www.ey.com/en_in/insights/ai/agentic-ai-india/aidea-survey-findings), the RBI FREE-AI committee report (August 2025) and the DPDP Rules notification (14 November 2025) were verified by the editor from the sources in the series specification. Descriptions and figures for Indika AI and FlexiBench are company-reported by PanScience Innovations (https://panscience.xyz) and are used to illustrate the structure of evaluation labour, not as evidence of outcomes.