LLM pricing and cost optimization

TL;DR

• LLMs are priced per token; output tokens often cost more than input tokens. The 10K-input / 2K-output prices shown below are illustrative and must be recalculated from current provider tables.
• The 60/30/10 tiering rule is a routing heuristic: route measured proportions of work to cheap, mid-tier, and flagship models, then verify quality and savings on the target workload.
• Batch processing can discount eligible deferred workloads, but provider terms, latency guarantees, and supported features vary.
• Prompt caching can reduce repeated-prefix input costs, but the discount, cache lifetime, eligibility, and invalidation rules are provider-specific.
• Context size can dominate agentic cost. A 10-step agent at 50K context per step uses five times as many input tokens as the same agent at 10K context per step, before accounting for compression overhead.
• The right model is the cheapest model that reliably completes the task, not the most capable one you can access.

---

30-Second Explanation

Mental model: total LLM cost is workload × token volume × price, plus retries, tools, storage, and human review. Reduce cost by routing or distilling only when quality holds, shrinking unnecessary context and output, caching repeated prefixes, batching eligible work, and measuring the full request—not just the advertised per-token rate.

The problem it solves

You build an AI agent. It works. Then you see the invoice.

A production agent running 10,000 tasks per day can cost $90,000 per month at the illustrative Opus 4.6 rates used below, or $18,000 at the illustrative Sonnet 4.6 rates. If the agent makes 10 LLM calls per task, multiply the per-task estimate by the call count and account for changing context sizes. Calculate this before launch because retries, tools, and multi-step loops can make the invoice much larger than a single-call estimate.

The gap is not automatically about quality. Many tasks your agent handles (summarize this document, extract these fields, classify this input) may not require the most capable model available. They require a model that is good enough, reliably, at acceptable latency. The cost difference between "good enough" and "best available" can be substantial, but measure quality and downstream failure costs before downgrading.

This is what LLM cost optimization solves: matching task complexity to model capability so you spend money where it produces value.

---

What is it?

LLM cost optimization is the practice of minimizing token spend in AI systems while preserving acceptable output quality. It covers model selection, context management, batching strategies, caching, and output formatting.

Think of it like cloud compute right-sizing. When you first move to AWS, you run everything on the largest instance because it is easy. Then you right-size: small instances for static sites, medium for APIs, large for databases. LLM optimization is the same discipline applied to inference spend rather than compute spend.

The key insight is that "more expensive = better" is not a task-independent rule. A flagship model may outperform a cheaper model on hard reasoning benchmarks, while both may pass a simple "does this email contain an order number?" eval. You pay for the capability gap only when the task actually requires it, so measure that gap on representative data.

---

How it works

Token-based pricing: the fundamentals

Every major LLM provider prices by token. A token is roughly 3/4 of a word: 1,000 words is approximately 1,333 tokens. The critical asymmetry is that input and output tokens have separate prices, and output tokens cost 3-5x more.

The table below is an illustrative 2026 pricing snapshot for comparing the math in this article, not a durable benchmark. Provider prices, model names, tiers, caching, and regional terms change; verify current pricing before using any figure for a budget.

A single API call with 10K input tokens and 2K output tokens costs:

• Opus 4.6: (10,000 / 1,000,000) × $15 + (2,000 / 1,000,000) × $75 = $0.15 + $0.15 = $0.30
• Sonnet 4.6: (10,000 / 1,000,000) × $3 + (2,000 / 1,000,000) × $15 = $0.03 + $0.03 = $0.06
• DeepSeek R2: (10,000 / 1,000,000) × $0.55 + (2,000 / 1,000,000) × $2.19 = $0.0055 + $0.0044 = $0.01

At 10,000 tasks per day, those unit costs become $3,000/day, $600/day, and $100/day respectively. Multiply by 30 days: $90,000, $18,000, and $3,000 per month. That is a 30x spread from the same workload, driven entirely by model selection.

The output token asymmetry is important. If you ask a model for a 500-word explanation when a 50-word answer would suffice, the output portion can be roughly 10 times larger than necessary. Controlling output length is often a high-leverage optimization, but do not truncate information required for correctness or safety.

The 60/30/10 tiering rule

In multi-agent systems, not all tasks are equal. Sort your workload by complexity and route accordingly.

The 60/30/10 rule is a useful hypothesis because many workloads contain a mixture of simple and difficult calls. A large share of calls may involve classifying, extracting, formatting, or routing, while a smaller share requires stronger reasoning. The actual proportions should come from traffic and eval data.

Applying the illustrative 60/30/10 split to 10,000 tasks per day: 6,000 tasks at $0.01, 3,000 tasks at $0.06, and 1,000 tasks at $0.30 = $60 + $180 + $300 = $540/day versus $3,000/day for all-Opus. That is a 5.5x reduction under these assumptions; the quality difference must be measured on the actual workload rather than assumed to be under 2%.

Teams may resist tiering because of quality concerns. The useful response is to measure it empirically: run each task type against several model tiers, measure success rate and downstream failure cost, and set thresholds. Some workloads can send 60-70% of tasks to cheaper models with no meaningful quality difference; others cannot.

Context cost dominates agentic pipelines

Single-call cost estimates miss the dominant cost driver in agent systems: context accumulation across steps.

Each tool call in an agentic loop is a new LLM invocation. Each invocation includes the full conversation history plus tool outputs. A 10-step agent loop where each step carries 50K tokens of context costs:

• 50K input tokens × 10 steps = 500K total input tokens per run
• At Opus 4.6 ($15/M): (500,000 / 1,000,000) × $15 = $7.50 per run

The same agent with context compression (keeping each step to 10K tokens through summarization and pruning):

• 10K input tokens × 10 steps = 100K total input tokens per run
• At Opus 4.6: (100,000 / 1,000,000) × $15 = $1.50 per run

Under these illustrative rates, context management produces a 5x input-cost reduction. If this agent runs 1,000 times per day, the difference is $7,500/day versus $1,500/day, or $225,000/month versus $45,000/month. Compression also adds work and can lose information, so validate the quality trade-off at scale.