Token Cost Has Become the Binding Constraint on Enterprise AI, Vista's Saroya Argues

What's New

Compute cost is rising rather than falling as AI scales, which makes routing work to the cheapest sufficient model the central engineering problem for enterprise software. Monti Saroya, Senior Managing Director and Co Head of the Flagship Fund at Vista Equity Partners, argues this in a conversation on Alt Goes Mainstream. Vista built a first notice of loss product for its insurance software company Duck Creek on a frontier model and frontier cloud, and found hosting alone would cost roughly $12 million for a small customer. The response was a harness, a software layer that acts as a router and directs queries to small language models where general intelligence is unnecessary.

Why It Matters

Saroya's position is that the majority of tokens consumed today are subsidised by venture investors and large cloud providers, and that this ends as those companies go public and face gross margin scrutiny. He notes price increases have already been passed through this year. The conventional expectation that compute costs decline with scale, which held through the internet buildout Saroya worked in at Cisco, is the assumption he is arguing against, on the grounds that demand is outstripping supply and geopolitical constraints require nations to source compute domestically.

Big Picture Drivers

• General intelligence solving nothing specific: A claims process requires trusted intelligence scaled to that process, and the harness built around the intelligence layer is what performs the work, which is where Saroya locates the software opportunity.

• The system of record as the moat: Agents log into the same enterprise systems built for human users, so the harness plugs the system of action into the system of record, and the accumulated data, compliance and regulatory work is what remains sticky.

• Open source as the cost answer: Saroya argues all software moves to open source over time, citing Unix giving way to Linux, and notes China has produced some of the strongest open source models, with intelligence levels sufficient for most enterprise workflows.

• Inference sited in existing infrastructure: Vista's VC2 venture places inference centres in data centres built for telecommunications in the 1990s, using air cooled SambaNova chips that avoid requiring new power, compliance or buildings.

• Productivity gains concentrated in the most expensive employees: AI is making the best engineers substantially more productive rather than replacing headcount, with engineering teams growing more slowly while the software being built becomes more complex.

• Value accruing at the application layer over time: Saroya maps the current cycle to the internet buildout, where semiconductors captured value first, cloud providers second, and software providers captured the long term value.

By The Numbers

• $12 million: Ongoing hosting cost for the Duck Creek first notice of loss product on a frontier model for a small insurance customer, before build costs, which Saroya judged untenable.

• 5 to 10x: The productivity uplift Saroya attributes to AI for the strongest employees, against 15% to 20% measured engineering productivity growth across Vista companies.

• $103 billion: Vista's assets under management as a specialist enterprise software investor.

• Two and a half years: How long Vista's agentic factory has been running, distilling model releases into deployment guidance for portfolio companies.

• Q1 this year: When token cost first entered the enterprise decision, according to Saroya.

Key Trends to Watch

• Chief financial officers imposing measurement on AI budgets: Saroya expects the question to shift toward what is being automated and how the return is measured, with that measurement occurring in the software layer.

• Workflow specific rather than vertically specific software: The next generation of applications will address individual processes, with agents built for personal claims separately from specialty or commercial claims, rather than for insurance as a category.

• Sovereign AI as the next infrastructure buildout: In a world where access to intelligence matters to a country, Saroya expects national compute capacity to become the theme, with no existing company serving as a clean analogy.

• Token price increases continuing as providers approach public markets: Subsidised consumption ends when gross profit and margin become the measures, which Saroya treats as already underway.

Memorable Quotes

• "general intelligence is general by design" Saroya's compression of why frontier models are the wrong tool for a defined enterprise process.

• "we're using the most intelligent models for the simplest questions which cost the most" The waste the harness layer exists to eliminate.

• "the cost of tokens is going up not down" The reversal of the pattern Saroya observed building internet infrastructure at Cisco.

• "everything goes open source over some period of time" Why he expects open weight models to carry the next phase of deployment.

The Wrap

The argument holds if token prices continue rising as subsidies withdraw, making cost aware routing a measurable differentiator, and if enterprises reward software vendors who can demonstrate accuracy improving while cost falls. It fails if compute supply expands faster than demand and unit costs decline on the historical pattern, which would make the harness layer an optimisation rather than a requirement, and would weaken the case for owning inference capacity. The measurable signal is whether enterprises begin reporting AI spend against automation outcomes. The next several quarters of published token pricing and enterprise budget discipline will indicate which direction this resolves.