A new line on the P&L
Jensen Huang made a prediction at GTC this year that most leaders haven't fully absorbed yet: employees will soon have a token budget alongside their salary.
It sounds like a futurist provocation. It's actually an architectural reality that finance teams are already starting to encounter.
Tokens are the unit of consumption for every AI interaction — every reasoning step, every context retrieval, every decision an agent makes. And unlike software licences or cloud compute, AI inference costs are variable by design. No fixed fee. No predictable monthly bill. Only consumption.
One agent — manageable. One hundred — noticeable. One thousand — a line item the CFO is asking about.
The enterprises that scale agents without governing token costs will discover the economics at the worst possible moment: when adoption is high, spend is compounding, and the architecture to control it was never built.
"Tokens are becoming as fundamental to ERP as headcount or compute. The CFOs who understand this first will ask the right questions. The ones who don't will inherit the consequences."
The frontier model trap
The instinct is understandable. You've invested in AI. You want the best. Put every agent on the most powerful frontier model and let it run.
The problem is that frontier models cost ten to a hundred times more per token than smaller, task-specific models. And the majority of what enterprise agents actually do doesn't require frontier intelligence.
Routing an invoice. Checking a policy. Summarising a status report. Categorising a support ticket. A well-designed smaller model handles all of this — accurately, reliably, at a fraction of the cost.
Frontier models earn their cost when the task genuinely needs them: complex multi-step reasoning, ambiguous decisions with significant business consequences, high-stakes exceptions where the cost of a wrong answer outweighs the cost of expensive inference.
Defaulting everything to frontier models isn't a quality decision. It's an architecture failure masquerading as one.
Three principles that control token costs at scale
1. Route by complexity, not by default
The routing decision — which model handles which task — costs almost nothing to make. The savings compound across thousands of daily agent interactions.
Simple task goes to a small model. Complex reasoning goes to a frontier model. This isn't a compromise on intelligence. It's intelligence applied to the deployment of intelligence.
2. Cache context, don't repeat it
Every time an agent re-ingests the same business rules, approval hierarchies, or historical data — you pay for it again. In a multi-agent environment where context is retrieved hundreds of times a day, that cost accumulates fast.
A persistent context layer means agents retrieve curated, pre-structured context rather than rebuilding it from scratch on every call. Same intelligence. A fraction of the token cost.
3. Budget per agent, not per department
Treating AI spend as a single departmental line item is like giving your entire sales team one shared expense account with no individual limits. Accountability disappears into aggregation.
Every agent should carry a defined token budget — with threshold alerts, escalation triggers, and hard caps. Not to restrict what agents can do. To make what they do accountable, traceable, and controllable before the spend happens rather than after.
How we built this at Vishleshan
When we designed Vidura, token governance was an architectural decision from the start — not something we bolted on after the first unexpected bill.
Every agent runs with a defined budget. Routine tasks route to smaller models automatically. Context is cached and retrieved, not re-ingested on every call. Frontier model access is reserved for the reasoning that genuinely warrants it.
The result is a cost structure that scales with the business rather than against it — one where growth in agent adoption doesn't create a parallel growth in uncontrolled spend.
Deepak's Take
The token budget conversation is coming to every enterprise boardroom. The question is whether your architecture is ready for it when it arrives.
The enterprises that treat token economics as a first-class concern — designing cost governance into the agent infrastructure from the beginning — will scale AI sustainably. The ones that treat it as a finance problem to solve later will find that later arrives faster than expected.
The CFO's question is already forming. The architecture should already have the answer.
Deepak Choithramani is Co-Founder of Vishleshan AI Solutions. He writes about enterprise AI, agentic systems, and what it actually takes to go from pilot to production.
Follow Deepak on LinkedIn