Uber burned through $3.4 billion on AI compute by April 2026. This happened while token prices were in freefall — down 98% from their 2024 peaks. If you're running AI in production, your bills probably look the same: prices dropped, but your total spend went up. Way up.
The Problem
Here's the paradox. Per-token pricing has collapsed across every major provider. GPT-4-class inference costs a fraction of what it did 18 months ago. Open-source models like Nemotron 3 Ultra and Gemma 4 12B have driven competition to the point where some providers are practically giving tokens away.
But enterprises aren't saving money. They're spending more.
A Bain survey published this week found a 68-point gap between individual AI benefit (97% of employees see value) and organizational ROI (only 29% hits the P&L). Translation: people love using AI, but the economics are broken.
The culprit isn't vendor greed — it's uncontrolled consumption. When tokens get cheap, teams stop optimizing. They stop routing. They throw every request at the most expensive model. Usage volume scales 10-50x faster than the price drops.
The Solution
The fix isn't begging your vendor for a discount. It's model routing — and it's the single highest-leverage cost optimization available today.
Model routing means sending each request to the cheapest model that can handle it well. A simple customer FAQ doesn't need a 550B parameter reasoning model. A code generation task might. The key is building intelligence into your routing layer.
The Linux Foundation just launched the Tokenomics Foundation to tackle this exact problem at an industry level. But you don't need to wait for standards.
Here's what works right now:
- Tier your models. Use a cheap/fast model (like Gemma 4 12B or MAI-Code-1-Flash) for 70-80% of requests. Reserve expensive models for tasks that actually need them.
- Set budget caps per agent. If you're running AI agents, every agent should have a hard spend limit. The survey data shows budget loops — agents that spiral into runaway costs — are one of the top three reasons agents get rolled back.
- Measure cost per outcome, not per token. A $0.001 token that gets the wrong answer is more expensive than a $0.01 token that gets it right the first time.
Benchmarks
- Nemotron 3 Ultra (NVIDIA): 5x throughput vs. comparable open models, 30% lower cost to task completion — but only if you route to it selectively
- Gemma 4 12B (Google): Nears 26B MoE performance at half the memory — excellent for high-volume routing tier
- MAI-Code-1-Flash (Microsoft): 5B active parameters, cheaper than Haiku-class models for agentic coding tasks
- JetBrains Mellum2: 2x faster inference than comparable code models, Apache 2.0 — ideal for internal dev tool routing
Caveat: These benchmarks are from controlled environments. Your mileage will vary based on workload, latency requirements, and how well your routing logic is tuned. No benchmark replaces measuring your own production data.
Impact
Let's do the math on a mid-size company running 10M AI requests/month:
- Blind routing (everything to the best model): ~$340K/month
- Smart routing (tiered with budget caps): ~$100K/month
- Savings: $240K/month = $2.9M/year
That's not theoretical. The Tokenomics Foundation's early data suggests model routing alone can cut enterprise AI costs by 60-70%. Combined with prompt optimization and caching, some teams are hitting 80%.
The companies that figure this out now — while tokens are "cheap" — will have a massive advantage when the next pricing cycle hits. Because it will.
The Bottom Line
Cheap tokens are a trap if you're not routing. The 98% price drop didn't save you money — it gave you permission to be sloppy. Stop measuring cost per token and start measuring cost per business outcome. Build routing. Set budgets. Or keep watching your bill triple while your vendor tells you they "cut prices again."
The companies winning at AI right now aren't the ones using the best models. They're the ones using the right models for the right tasks — and they're saving millions doing it.