AI Cost & Unit Economics

Why AI Economics Are Different

Traditional SaaS economics are pretty forgiving. Once you've built the infrastructure, each additional user costs almost nothing. Your database handles 10x more queries with a bigger instance. Your CDN serves 10x more pages with barely any cost increase. That near-zero marginal cost is what gives SaaS businesses their 70-80% gross margins.

AI breaks this model completely. Every inference costs real money. Every API call to an LLM has a per-token price. Every GPU-second of compute for a self-hosted model burns electricity and depreciates hardware. If your product processes 1 million requests per day through an LLM and you double your user base, your inference cost roughly doubles too. There's no magical economy of scale that saves you, at least not automatically.

This has caught multiple companies off guard. They build an AI feature, it takes off with users, and suddenly the cloud bill is growing faster than revenue. The VP of Engineering gets a panicked call from finance wanting to know why infrastructure costs jumped 300% in a single quarter. Understanding AI unit economics before launch prevents that entire conversation.

AI Unit Economics Framework

You need three layers of cost visibility, and each one serves a different audience and drives different decisions.

Cost per inference is your foundation. It's the raw cost of a single AI operation: one LLM call, one embedding generation, one model prediction. For API-based models, you can calculate it pretty directly from token counts and pricing tiers. For self-hosted models, divide your total GPU infrastructure cost by the number of inferences served. This number should live on a dashboard that your ML engineers check weekly.

Cost per feature rolls inference costs up into product-level units. A single "summarize this document" feature might need three LLM calls (chunking, summarizing each chunk, combining summaries), two embedding lookups, and one classification call. The cost per feature is all of those added together. This is what your product managers need to see, because it tells them the true cost of the capabilities they're building.

Cost per user divides total AI spend by active users, broken down by segment. Enterprise users processing 500 documents a month have completely different economics than free-tier users asking 10 questions. This metric feeds straight into pricing decisions. If your cost per enterprise user is $12/month and you're charging $20/month, your AI margin is 40%. If free-tier users cost $3 each per month, your CFO needs to know that number.

Token Optimization Strategies

Token optimization is where engineering effort delivers the best ROI on AI costs. Most teams can cut their token consumption by 50-70% without any meaningful quality drop.

Prompt engineering for efficiency. Most prompts are way longer than they need to be. A system prompt with 2,000 tokens of instructions can often be trimmed to 500 tokens with no measurable quality difference. Run your prompts through an eval suite at different lengths and find the shortest version that still meets your quality bar. This isn't a one-time thing. Every time you update prompts, check the token efficiency again.

Response caching. If people keep asking the same question, cache the answer. It sounds obvious, but a lot of teams skip this because AI responses feel "dynamic." In practice, 20-40% of queries in most applications are either identical or close enough that it doesn't matter. Set up semantic caching with embeddings: if a new query's embedding is close enough to a cached one, just return the cached response. This alone can eliminate 30% of your inference costs.

Tiered model routing. Not every request needs your most powerful (and expensive) model. Build a lightweight classifier that routes requests by complexity. Simple factual lookups go to a small model or a retrieval system. Standard questions go to a mid-tier model. Only the genuinely complex, nuanced requests go to the frontier model. Teams that set this up typically see 60-80% cost reduction with barely any quality impact.

Batching and async processing. If your use case can handle even a little bit of latency, batch requests together. Many model serving frameworks offer batch inference at significantly lower per-token costs. A background job that collects requests in 5-second windows can cut per-token cost by 30-50% compared to processing each call individually in real time.

Model Selection as Economic Decision

Picking a model isn't just about performance. It's a financial commitment that compounds over time.

A frontier model like GPT-4 might cost $30 per million input tokens. A fine-tuned GPT-3.5 might run $1.50 per million tokens. A self-hosted Llama model on your own GPUs might come in at $0.30 per million tokens after you amortize the hardware. The quality gap between these options is real, but it's often narrower than people expect, especially for well-defined tasks where you can fine-tune the cheaper model on your specific domain data.

Do the math before you commit. If your feature handles 10 million requests per month at an average of 1,000 tokens each, the difference between GPT-4 and a fine-tuned open-source model is roughly $290K per month. That's $3.5 million per year. At that kind of delta, you can hire a small ML team to fine-tune and maintain a custom model and still come out well ahead.

The math changes based on volume. At 100K requests per month, just use the best API model and don't overthink it. At 1 million, start evaluating alternatives. At 10 million, model hosting economics should be a regular topic in your engineering leadership meetings.

AI FinOps Reporting

Your AI cost dashboard should answer five questions at a glance:

What are we spending on AI inference, broken down by model and feature? Which features are cost-efficient (delivering high business value per dollar of inference) and which are burning money? How is cost per inference trending over time, and are our optimization efforts actually working? What will our AI costs look like in 3, 6, and 12 months at current growth rates? Where is the biggest opportunity to cut costs without hurting quality?

Build this dashboard before your AI spend gets big enough to cause problems. The teams that wait until the bill is alarming end up making panicked cuts to features users depend on. The teams that build cost visibility early make informed, strategic calls about where AI delivers real value and where it's just an expensive experiment.

Report AI unit economics alongside your traditional infrastructure metrics in monthly engineering reviews. When the CFO sees "cost per AI-resolved ticket: $0.45 vs. cost per human-resolved ticket: $12.00," the conversation shifts from "why is AI so expensive" to "how do we route more tickets through the AI pipeline." That shift in framing is the whole point of good AI FinOps.