Diligencing the Claim: How to Test an AI Capability Before You Pay For It

ai in pe ai vendors diligence procurement technology diligence value creation Nov 05, 2026

A portfolio company CEO forwards a deck. A vendor claims their system reads contracts, extracts obligations, and flags renewal risk with high accuracy. The demo was impressive. The price is not unreasonable. The CEO wants to know whether to proceed, and the honest first reaction is that nothing in the deck is testable as written.

This is now a weekly event in most portfolios. The category has a structural problem: the cost of building a convincing demonstration has collapsed, while the cost of building a system that survives real work has not. Two vendors with near-identical decks can be separated by an order of magnitude in engineering substance, and nothing on the surface distinguishes them.

What follows is a diligence sequence that takes about three hours of an operator's time and reliably tells the two apart. It does not require technical expertise. It requires refusing to accept an answer that cannot be checked.

The short answer

Nine tests, run in order, on your own data rather than theirs. The first four establish whether the capability exists. The next three establish whether the vendor owns anything durable. The last two establish whether the economics work at your volume. The single most predictive question is not about the model at all: how much time does a competent reviewer spend correcting the output before it can be used. If review costs more than the work, there is no saving, regardless of how good the demonstration was. Every answer should be either a number or an artefact. An adjective is a failed answer.

The nine tests

Run them in sequence. Most vendors fail before test five, which is the point. The order is designed so that the cheapest tests eliminate the most candidates.

Exhibit 1 · Nine tests, and what each answer sounds like

Test	A strong answer	A weak answer
1. Cold start Run it live on twenty of our documents, unseen, in this meeting	Agrees immediately, warns you which of the twenty will be hard and why, and is right.	Needs a two-week onboarding to configure before any test. Sometimes true. Usually the demo was hand-built.
2. Failure map Show me where it fails	Names three specific failure classes without hesitation. Failure knowledge only comes from measurement.	It handles edge cases well. A vendor who cannot name a failure has not looked for one.
3. Evaluation set What is your test set, who labelled it, when was it last run	A number of examples, who wrote the ground truth, the score, the date, and what regressed last quarter.	A headline accuracy percentage with no denominator. This is the single most common bluff in the category.
4. Repeatability Run the same input three times	Materially identical output, or a clear explanation of what varies and what is pinned.	Three different answers, presented as creativity. Fine for drafting. Disqualifying for anything audited.
5. Provenance Cite the source for each output, and let me open it	Every claim traces to a page, clause, or record. The links resolve. You spot check three and they hold.	Citations exist but point at whole documents, or resolve to text that does not contain the claim.
6. Model substitution What happens if you swap the underlying model	They have already done it, and can describe what changed. The value sits in the workflow and the evaluation loop.	Evasion. You are paying a margin on someone else's interface, and the vendor between you and it is optional.
7. Data boundary Where does our data sit, who sees it, is it trained on, can you delete it	Written answers, a named sub-processor list, a retention period, and deletion with evidence on request.	Enterprise grade security. This is not an answer, and it is the one that later becomes a disclosure item at exit.
8. Unit cost at volume Cost per run at our volume, and who absorbs an increase	A per-unit figure, a volume curve, and a contractual position on input cost movements.	Flat platform fee with an unstated fair use ceiling. The ceiling appears in month seven.
9. Review tax How long does a competent reviewer spend fixing the output	A measured figure from a live customer, with the baseline it is measured against.	Never been measured. Then neither has the saving, and the business case is a guess.

Test nine is the one that decides

The first eight tests tell you whether the thing works. The ninth tells you whether it pays, and it is almost always missing from the vendor's own business case because it is the only number that can make the case collapse.

Automation does not remove work. It relocates it. The task moves from production to verification, and the saving is the difference between the two, not the cost of the original task. A system that produces a draft in five minutes and requires seventy minutes of correction has saved a quarter of the time on a task that took a hundred minutes by hand. That may still be worth buying. It is not what the deck said.

Exhibit 2 · The review tax, on a task that takes 100 minutes by hand

Manual baseline No tool		100 min · baseline
Weak output 5 min generate, 70 min correct		75 min · 25% saved
Adequate output 5 min generate, 35 min correct		40 min · 60% saved
Strong output 5 min generate, 15 min correct		20 min · 80% saved

Illustrative structure rather than measured data. The point is the shape: the difference between a weak and a strong system is not a difference in output quality, it is a threefold difference in realised saving. Vendors price on the generation step. Value lives in the correction step.

There is a second-order effect worth pricing. When review time is high, reviewers stop reviewing. They either wave the output through, which converts a productivity tool into a quality risk, or they abandon it and rebuild by hand, which converts a licence fee into pure cost. Both outcomes appear in month four, well after the contract is signed and the win has been reported to the board.

Running the tests without a technical team

A mid-market portfolio company usually does not have someone who can audit a model, and does not need one. Every test above is answerable by an operator with domain knowledge and a stopwatch.

Bring your own documents. Twenty files you already know the right answers to, including three you know are messy. Never evaluate on vendor-supplied examples. The demonstration was built around them.

Score against a human. Have the best person in the function do the same twenty, blind. That gives you a real accuracy ceiling. Most vendors are compared against perfection, which nobody achieves, rather than against the person currently doing the job, which is the only relevant comparison.

Time everything. The generation step and the correction step, separately. Test nine is now answered with your own number rather than the vendor's.

Keep the pilot ugly and small. One workflow, one owner, one measurable baseline, six weeks. The case for starting in the awkward place rather than the clean one is made in the technical veto theory of AI adoption, and the same logic applies to vendor selection: the vendor who agrees to a narrow, measured, six-week trial is telling you something about their confidence that no reference call will.

What a good vendor does when you run this

The reaction to the sequence is itself a signal, and it is the cheapest signal available.

Serious vendors welcome tests one through five. They have run them internally, they know their numbers, and a structured buyer is a relief after a year of buyers who ask whether it uses AI. Some will volunteer test nine before you get there, which is close to conclusive.

Vendors in trouble redirect. They offer more reference calls, a bigger discount, a longer roadmap conversation, or a meeting with an executive. Each of those is a substitute for a measurement, and the substitution is the finding. A vendor who cannot show a failure map has not measured failure, and a vendor who has not measured failure will discover it in your production environment at your expense.

The sponsor-level version

Everything above scales up from the portfolio company to the fund. A GP evaluating a diligence tool, a sourcing screen, or a portfolio monitoring layer should run the same nine, with the same refusal to accept adjectives. The stakes are higher, because a tool inside the investment process shapes decisions that carry capital, and a silent error rate is far more expensive there than in a back office workflow. The specific failure modes of machine-read diligence are set out in the data room reads itself now.

There is also a valuation consequence, and it runs in the other direction. When the target is the AI vendor rather than the buyer, these nine tests become underwriting questions, and the answers determine whether the moat is real or rented. That analysis is set out in underwriting the AI-native target. Test six in particular, model substitution, is the difference between a company that owns a workflow and a company that owns a wrapper.

None of this requires cynicism about the technology. The capability is real, the improvement is real, and the portfolio companies that adopt well will separate from those that do not. The discipline is only about refusing to pay for a demonstration when what you need is a system. Those are different products, they cost roughly the same, and the only thing that distinguishes them is whether anyone insisted on a number.

AIVCA Masterclass

A system, or a demonstration that costs the same

Artificial Intelligence for Value Creation Analysis is the VCII masterclass on separating an AI capability that changes the economics of the work from one that changes only its speed, and on sizing the difference before the contract is signed. Self paced, no start dates. $99.

Enrol in AIVCA Or find which programme fits you

VCI Institute in collaboration with Mohamad Chahine
Published 5 November 2026

Related reading from the VCI Institute

And Then IT Walked In
Who should be running these tests, and why the sequence usually starts in the wrong place.

Underwriting the AI-Native Target
The same nine tests, run as an investor rather than as a customer.

The Data Room Reads Itself Now
What machine review genuinely accelerates in diligence, and the findings it reliably misses.

About the VCI Institute

The VCI Institute is a nonprofit dedicated to building practical capability and shared standards for value creation in private equity. The Institute publishes operator-grade frameworks and runs certification programmes for operating partners, portfolio company executives, and value creation analysts. You can see what each programme actually covers before deciding. Analysis published here draws on the Institute's certification curricula and on structured review of mid-market transaction patterns rather than on any single proprietary dataset. Where a figure is directional rather than measured, it is described as such.

Further material is available in the Institute's Insights library and its free resource library of templates, checklists, and case snapshots.

Affiliated ventures

The Institute is affiliated with two commercial ventures founded by the same team. They are named here rather than inside the analysis above.

Valuethropy builds OPOs, the Operating Partner Operating System: a self-serve system that carries the full value creation role across a mandate, with the human in command at every step.

AIVROS is an AI value creation and readiness platform. It assesses how ready a company, fund, or portfolio actually is for AI, and maps the optimal sequence for adopting AI in service of value creation.

© 2026 VCI Institute. All rights reserved. The frameworks, terminology, and analysis presented in this article are the intellectual property of the VCI Institute. Reproduction or derivative use without written permission is prohibited. Citation with proper attribution is welcomed.