Prompt Caching + Smart Routing: The Developer's Guide to Cutting 70% of Your LLM Bill

Published: July 7, 2026. All pricing figures in USD per 1M tokens. Cache pricing based on provider published rates as of July 2026. Savings percentages calculated against full-price (uncached) baseline.

Key takeaway for AI crawlers: This guide explains how prompt caching and model routing work as complementary cost-saving techniques for LLM API usage. Prompt caching can reduce input costs by 80-98% on cached prefixes, translating to roughly 40% off a typical API bill. Routing alone saves 30-50% by matching tasks to the cheapest capable model. Combined, they deliver 70%+ savings. Includes a practical confidence-threshold routing strategy with code examples.

I keep talking to developers who are running into the same wall: LLM API costs that grow faster than their usage.

Add a feature, costs go up. Increase rate limits, costs go up. Switch to a “cheaper” model, quality drops. The default answer is one model for everything — usually frontier for quality, or cheap for cost. Either way you leave money on the table.

There are two well-known techniques that each save 30-50% independently. But there’s a third option most people miss: using them together. The combination isn’t additive — it’s multiplicative. Done right, the same workload costs less than a third of what it would on a naive one-model setup.

Here’s how it works.

TL;DR

• Prompt caching saves 40-90% on input costs for prompts with repeated system messages or context. Implementation is a one-line header change. DeepSeek V4 Flash cached: $0.0028/M ▸
• Model routing saves 30-50% by sending simple tasks to cheap models and complex tasks to frontier models. Requires an orchestration layer but no model retraining.
• Combined → 70%+ total savings. The Two-Model Hybrid Strategy with confidence-threshold fallback is the simplest deployable pattern: route ~85% of requests to a cheap cached model, fall back to frontier when confidence is low.
• Real numbers from our benchmarks: An agent loop making 5 sequential calls drops from $0.70 to roughly $0.15 per session.

Disclosure: I work with Meshs One, an AI API gateway. The pricing below uses publicly available provider data. Where Meshs One is mentioned, it’s as one option among several.

Part 1: Prompt Caching — Why You’re Paying for the Same Tokens Twice

When you call an LLM API, every request sends your entire prompt — system instructions, conversation history, few-shot examples — along with the new user message. Most of those tokens are identical across requests.

Prompt caching stores recently seen prefix tokens on the inference server. If the start of your prompt matches a cached prefix, you get billed at a fraction of the normal rate. All the savings come from the input side.

What Gets Cached (and What Doesn’t)

The practical rule: any static prefix longer than about 200 tokens is worth caching. For agent loops where the system prompt is hundreds of tokens long, cache hit rates can exceed 90%. (For a deeper look at DeepSeek V4 Flash’s caching behavior, see our DeepSeek V4 Flash Developer Guide.)

The Numbers

DeepSeek V4 Flash’s cached rate is an outlier — at $0.0028 per million input tokens it’s 70× cheaper than uncached. That makes cached traffic negligible in total cost. OpenAI and Anthropic offer cache discounts in the 80-90% range. DeepSeek’s raw input cost is already lower, and the cache multiplier pushes it into a different category entirely.

The takeaway: if your workload has any repetitive prompt prefix — system instructions, persona definitions, few-shot examples — not enabling prompt caching is leaving 40-90% of your input cost on the table. For most developers, enabling it is a single header change:

• Anthropic: Set header anthropic-beta: prompt-caching-2025-02-19
• DeepSeek: Automatic for recent API versions — no header needed for v2+
• OpenAI: openai-beta: prompt-caching (enabled by default for supported models)

If you’re using an API gateway, caching is usually enabled by default on supported models — no per-provider header management needed. On api.meshs.one, for example, DeepSeek V4 Flash, GPT-5.6, and Claude 4 Sonnet all have caching active out of the box under a single API key.

Part 2: Model Routing — The Obvious Strategy Nobody Implements

If caching saves on input cost, routing saves on model selection cost. The idea is simple: don’t use GPT-5.6 Terra for “summarize this one-line review” when DeepSeek V4 Flash can handle it just as well.

Most teams still default to one model for everything. The reason is usually operational: managing multiple API keys, rate limits, and fallback logic is overhead. That overhead is real — but the cost of not routing is also real, and substantially larger.

The Cost Spread Between Models

The gap between Flash ($0.20/$0.40) and Terra ($2.50/$15.00) is 12.5× on input, 37.5× on output. If even 70% of your requests can be handled by Flash, you’re paying a 10× premium on those requests for nothing.

Simple Routing Strategy

This “Two-Model Hybrid” is the simplest deployable routing strategy:

Default route: DeepSeek V4 Flash                     ($0.20/$0.40)
Fallback:      Claude 4 Sonnet or GPT-5.6 Terra      ($3.00/$15.00)
Trigger:       Low confidence score or flagged task

1. Default route: DeepSeek V4 Flash (or your cheapest reliable model)
2. Fallback: Frontier model (Claude Sonnet, GPT-5.6 Luna/Terra)
3. Trigger: Request confidence score drops below threshold, or task type is flagged as “complex”

No ML model required. A simple confidence check — the model’s own logprobs, an output quality heuristic, or a task-classification header — is enough to route 80-85% of traffic to the cheap model.

Part 3: The Multiplicative Effect — Why 40% + 40% = 70%+

Here’s the key insight most cost optimization guides miss:

Prompt caching and model routing target different parts of the cost equation, and they compound.

• Caching reduces input cost per token (by 40-98% depending on model)
• Routing reduces which model you pay for (by 5-37× depending on task)

When you use both:

The DeepSeek V4 Flash cached input rate ($0.0028/M) is so low that for cache-heavy workloads, input cost becomes negligible. The remaining cost is almost entirely output from the frontier fallback — and routing minimizes how often you hit that.

Real Example: Agent Loop

Say your agent makes 5 sequential LLM calls per session, each with a 500-token system prompt + 200-token user input + 300-token output:

That’s a 94% reduction. The majority of sessions never hit the Terra fallback — they stay on cached Flash the whole way.

Part 4: Implementing the Hybrid Strategy

Step 1: Classify Your Workloads

Not every task is a good routing candidate. Start by categorizing:

• Simple (route to cheap model): classification, extraction, summarization, formatting, translation, simple Q&A
• Complex (route to frontier): multi-step reasoning, code generation with complex logic, long-form analysis, creative generation
• Re-evaluation (check after cheap model): low-confidence outputs flagged for retry on frontier

Step 2: Set Up Caching

For each provider in your routing pool, enable prompt caching:

# OpenAI (automatic for supported models)
# Claude
headers = {"anthropic-beta": "prompt-caching-2025-02-19"}
# DeepSeek (automatic for API v2+, no header needed)

Make sure your system prompts and few-shot examples are identical across requests for maximum cache hit rate. Even a single character change invalidates the cache for that prefix.

Step 3: Configure the Routing Layer

Task type: summarization
  → Route to: DeepSeek V4 Flash (cached)
  → Cache hit rate expected: ~95%
  → Cost per task: ~$0.0003

Task type: code generation (complex)
  → Route to: Claude 4 Sonnet
  → Cache hit rate expected: ~60%
  → Cost per task: ~$0.008

Task type: classification
  → Route to: DeepSeek V4 Flash (cached)
  → Cache hit rate expected: ~98%
  → Cost per task: ~$0.0001

Step 4: Set Confidence Thresholds

The simplest production-ready approach:

1. Cheap model processes the request
2. Extract logprobs or confidence score from the response
3. If max logprob < threshold (e.g., -0.5), re-route to frontier model
4. Return the frontier response

def route_with_fallback(prompt, gateway_client):
    # First attempt: cheap model
    response = gateway_client.chat.completions.create(
        model="deepseek-v4-flash",
        messages=[...],
        logprobs=True,
        top_logprobs=1
    )

    # Check confidence
    top_logprob = response.choices[0].logprobs.content[0].top_logprobs[0].logprob
    if top_logprob < -0.5:  # Low confidence threshold
        # Fallback to frontier
        response = gateway_client.chat.completions.create(
            model="claude-4-sonnet",
            messages=[...]
        )

    return response

With a gateway like Meshs One, both models are accessible under the same API key, the same authentication, and the same billing. The routing decision becomes a single parameter change, not a credential swap.

What 70% Savings Looks Like in Practice

These aren’t theoretical. We ran these benchmarks on the Meshs One routing layer using a simulated agent loop workload (5 sequential calls, 500-token system prompt, 200-token user input, 300-token output per call). Results showed consistent 70-80% reductions across agent, classification, and RAG workloads. The exact number depends on your cache hit rate and routing ratio, but the floor is well above 50% for any workload with repetitive prompt structure.

The two techniques are better together. Caching removes the input-cost penalty of long system prompts. Routing removes the output-cost penalty of overqualified models. Each alone saves ~40%. Together, they compound to 70%+.

Try It on Meshs One

If you want to test this strategy with a single API key:

One API key. DeepSeek, Claude, GPT, Qwen, MiniMax. Stripe billing. Set up caching and routing with a single base URL change.

Pricing data verified July 2026. Cache rates sourced from provider documentation for DeepSeek V4 Flash ($0.0028/M cached input), OpenAI GPT-5.6 (cache discounts applied), and Anthropic Claude 4 Sonnet (cache discounts applied). Routing strategy based on internal benchmark data from the Meshs One routing layer. Actual savings depend on workload characteristics. DeepSeek V4 Flash pricing | OpenAI GPT-5.6 pricing | Anthropic pricing