Stop Paying Frontier Prices for Flash-Class Work: Build a Two-Tier Model Router in an Afternoon

Cheap/fast on the left, frontier/slow on the right, one lever in the middle. The router is the whole game. Illustration: ctrlaltorion.

Look at the last seven days of AI infrastructure headlines and pretend for a second you don’t work in this industry. What’s the story?

On July 9, 2026, OpenAI shipped the GPT-5.6 family — Luna, Terra, Sol — with a launch pitch that leads not with capability but with price-per-token, and a “Luna” tier explicitly aimed at the price-performance frontier (that broke ~4 weeks ago, but a follow-on Luna price drop landed on July 30 — days old). On July 31, DeepSeek released V4-Flash-0731, an open-weight flash-class model priced to make API-only vendors sweat. Two days later, Cloudflare published “Smaller, faster, safer models,” explaining how it’s running Kimi and GLM at scale on its own edge hardware and, not subtly, why you don’t need a frontier lab’s API to get useful throughput.

Three announcements, one week, one message: inference is a buyer’s market, and the vendors are telling you so.

Meanwhile, most small operators running AI pipelines are still shopping like it’s 2024. One frontier model behind one API key, called for every task from “detect the invoice number” to “draft a legal memo,” billed as one giant undifferentiated line item every month. Both the vendors and the math have moved on. This piece is about catching your automations up in an afternoon.

We’ll cover: the case for the two-tier pattern, a working Python router you can paste into a real project, a one-hour benchmark template that gives you defensible per-workload numbers instead of vibes, and a fully worked cost example for a small-business document-processing pipeline. And a section on vendor-churn insurance, because the same week those three cheap-inference stories broke, GitHub retired GitHub Models on July 30 and deprecated Gemini 2.5 Pro and Gemini 3 Flash from Copilot on July 31. The lesson is not “pick a winner.” The lesson is “abstract behind an interface so swapping is a config change.”

Why routing beats picking

The pitch for a two-tier router is uncomfortably simple, so we’ll be honest about the discomfort.

Most pipeline steps in real small-business automation are not judgment work. They’re mechanical: classify this email as refund_request / shipping_question / other; pull the invoice total from this PDF text; rewrite this product description in 40 words; decide whether this support ticket needs human attention. There’s a right answer, there’s a wrong answer, and a well-prompted small model gets it right about as often as a frontier model — with cost and latency differences of one to two orders of magnitude.

The judgment work is real. Drafting a nuanced customer reply, reasoning about a contract clause, planning a multi-step operation — those tasks genuinely benefit from a frontier model, and trying to route them cheap will bite you.

The trap most people fall into: they’ve picked one model — usually the biggest one they’re comfortable paying for — and they run every step of the pipeline through it. That model absolutely handles the judgment work. It also handles the mechanical work, at 20x the cost per call. If 80% of your pipeline volume is mechanical and 20% is judgment (typical for document processing, ticket triage, e-commerce ops), then routing the mechanical 80% to a flash-tier model cuts your inference bill by roughly 70–90% with no measurable quality loss on the routed tier. That’s not a projection. That’s what falls out of the arithmetic below when you plug in current per-token prices.

The reason more people don’t do this isn’t that it’s hard. It’s that “just use the good model” feels safer, and the price-per-call numbers are individually small enough that nobody’s ledger screams. Until it does — usually right after a runaway loop or a bad month. See also: the piece I wrote a couple of days ago about building your own AI spend meter. This piece is the operational partner: once you can see the spend, this is how you cut it.

The router pattern in 40 lines of Python

The router is not clever. That’s the point. Clever routers become weird failure modes; boring routers just work.

Here’s a minimal implementation that decides tier per call, executes on the chosen tier, and escalates on a confidence signal from the cheap tier. It targets any OpenAI-compatible endpoint (which now includes basically every major provider and self-host tool) so swapping models is a URL and a name.

# router.py — a two-tier model router. No frameworks, no magic.
import os, json, time, sqlite3
from openai import OpenAI

# --- Two clients, two tiers. Same interface. -------------------------------
CHEAP = OpenAI(base_url=os.environ["CHEAP_BASE_URL"],
               api_key=os.environ["CHEAP_API_KEY"])
SMART = OpenAI(base_url=os.environ["SMART_BASE_URL"],
               api_key=os.environ["SMART_API_KEY"])

CHEAP_MODEL = os.environ.get("CHEAP_MODEL", "deepseek-v4-flash-0731")
SMART_MODEL = os.environ.get("SMART_MODEL", "gpt-5.6-terra")

# Per-million-token prices; update when your vendors change theirs.
PRICES = {
    CHEAP_MODEL: {"in": 0.14, "out": 0.28},   # illustrative flash-tier
    SMART_MODEL: {"in": 2.50, "out": 10.00},  # illustrative frontier-tier
}

# --- The routing decision --------------------------------------------------
MECHANICAL_TASKS = {"classify", "extract", "reformat", "summarize_short"}

def choose_tier(task_type: str, input_tokens: int) -> str:
    """Route by task shape, not by hunch. Cheap by default; escalate on need."""
    if task_type in MECHANICAL_TASKS and input_tokens < 8000:
        return "cheap"
    return "smart"

# --- Confidence-based escalation ------------------------------------------
def parsed_ok(text: str) -> bool:
    """The cheap tier must produce parseable JSON. If it can't, escalate."""
    try:
        obj = json.loads(text)
        return isinstance(obj, dict) and obj.get("confidence", 0) >= 0.7
    except Exception:
        return False

# --- The router ------------------------------------------------------------
def route(task_type: str, system: str, user: str, project: str = "default"):
    approx_in = (len(system) + len(user)) // 4  # ~4 chars/token, close enough
    tier = choose_tier(task_type, approx_in)
    resp = _call(tier, system, user, project)

    if tier == "cheap" and not parsed_ok(resp["text"]):
        # Cheap tier failed structurally — escalate. Log both calls.
        resp = _call("smart", system, user, project, escalated_from="cheap")
    return resp

def _call(tier, system, user, project, escalated_from=None):
    client = CHEAP if tier == "cheap" else SMART
    model  = CHEAP_MODEL if tier == "cheap" else SMART_MODEL
    t0 = time.time()
    r = client.chat.completions.create(
        model=model,
        messages=[{"role": "system", "content": system},
                  {"role": "user", "content": user}],
        temperature=0,
    )
    latency = time.time() - t0
    u = r.usage
    p = PRICES[model]
    cost = (u.prompt_tokens * p["in"] + u.completion_tokens * p["out"]) / 1_000_000
    _log(project, tier, model, u, cost, latency, escalated_from)
    return {"text": r.choices[0].message.content, "tier": tier,
            "cost": cost, "latency": latency}

def _log(project, tier, model, u, cost, latency, escalated_from):
    db = sqlite3.connect("router.db")
    db.execute("""CREATE TABLE IF NOT EXISTS calls
        (ts REAL, project TEXT, tier TEXT, model TEXT,
         in_tok INT, out_tok INT, cost REAL, latency REAL, escalated_from TEXT)""")
    db.execute("INSERT INTO calls VALUES (?,?,?,?,?,?,?,?,?)",
        (time.time(), project, tier, model,
         u.prompt_tokens, u.completion_tokens, cost, latency, escalated_from))
    db.commit()

Four things this router does that you should not skip:

1. Two clients, one interface. Both tiers speak OpenAI-compatible. You can point CHEAP_BASE_URL at DeepSeek, Cloudflare Workers AI, a self-hosted vLLM, or Groq without changing calling code. The whole vendor-churn insurance section below is enabled by these five lines.
2. Route by task shape, not by hunch. choose_tier is boring and inspectable. When the pipeline misbehaves, you can read the decision. Fancy learned routers hide the reason a call went where it went, which is exactly what you don’t want at 2am.
3. Escalate on a machine-checkable signal. The cheap tier’s job is to produce parseable JSON with a confidence field. If it can’t, you get the frontier model automatically. This is the pattern that lets you sleep on it: you’re not hoping the cheap model is right, you’re detecting when it isn’t.
4. Log every call, always. Same table shape as the spend meter from the Cursor piece — because until you can produce a per-project dollar total from your own database, you’re guessing about savings.

Prompt tip: for MECHANICAL_TASKS, always demand JSON with an explicit confidence field and a short reason. Small models are surprisingly good at telling you when they’re unsure if you require them to. That’s what makes escalation work.

The one-hour benchmark template

Before you switch anything in production, benchmark on your own data. Vendor benchmarks are marketing; your benchmark is the only one that matters.

The template is deliberately unglamorous — a Python script and a checklist. You can do the whole thing in about an hour if your data is already in a folder.

Step 1 — Sample 50 real tasks (10 min)

Not 5, not 500. Fifty is enough that a per-tier accuracy delta above ~6 percentage points is visible; it’s small enough that you can hand-label the ground truth over coffee. Pull the last 50 examples of the pipeline step you’re about to route — 50 support emails, 50 invoices, 50 product listings. Save inputs and true outputs to a JSONL file: {"input": "...", "expected": "..."}.

Step 2 — Run both tiers over the sample (20 min)

# bench.py — one-hour benchmark, both tiers, honest numbers.
import json, statistics
from router import _call  # reuse the metered call function

SAMPLE = [json.loads(l) for l in open("sample.jsonl")]
SYSTEM = open("prompt.system.txt").read()

def run(tier):
    results = []
    for row in SAMPLE:
        r = _call(tier, SYSTEM, row["input"], project=f"bench_{tier}")
        results.append({"expected": row["expected"], "got": r["text"],
                        "cost": r["cost"], "latency": r["latency"]})
    return results

def score(results, judge):
    """`judge` is your task-specific correctness function.
       For classification: got == expected. For extraction: field match.
       For open text: another LLM call, or human spot-check on a subset."""
    correct = sum(1 for r in results if judge(r["got"], r["expected"]))
    return {
        "accuracy": correct / len(results),
        "avg_cost":    statistics.mean(r["cost"]    for r in results),
        "avg_latency": statistics.mean(r["latency"] for r in results),
        "total_cost":  sum(r["cost"] for r in results),
    }

if __name__ == "__main__":
    cheap = score(run("cheap"), judge=lambda g, e: g.strip() == e.strip())
    smart = score(run("smart"), judge=lambda g, e: g.strip() == e.strip())
    print(json.dumps({"cheap": cheap, "smart": smart}, indent=2))

Step 3 — Score honestly (20 min)

For classification / extraction, exact match on the labeled field. For open-ended text (summaries, drafts), either:

• Cheap-and-fast: LLM-as-judge — a third model rates each pair 1–5 on faithfulness. Not perfect, but consistent enough for a decision on 50 rows.
• Correct-and-slow: spot-check 15 of the 50 by hand. Look at the failures on both tiers. You will learn more from these 15 rows than from any leaderboard.

Step 4 — Compute break-even (10 min)

You now have four numbers: cheap accuracy, smart accuracy, cheap cost/call, smart cost/call. The decision:

• If cheap accuracy is within ~3 percentage points of smart, route this task cheap. Ship it.
• If cheap accuracy is 3–10 points behind, add the confidence-based escalation. Rerun bench with escalation on. If the hybrid accuracy matches smart at a cost meaningfully below smart, ship the hybrid.
• If cheap accuracy is >10 points behind on your data, this task is not a fit for the cheap tier. Route smart. Move on to the next task — most pipelines have several steps, and cheap will win most of them even if it loses this one.

An hour of this per pipeline step is the highest-leverage hour of infra work you’ll do this quarter.

The worked example: a small-biz doc-processing pipeline

Numbers make this real. Let’s cost out a plausible small-business automation: an invoice-and-receipt intake pipeline for a 3-person bookkeeping shop, running against a shared inbox that receives ~200 documents/day (5,000/month after weekends).

The pipeline has four steps per document:

#	Step	Input	Output	Task type
1	Classify the doc	~500 tokens (extracted text)	~30 tokens (JSON label + confidence)	mechanical
2	Extract fields (date, vendor, total, line items)	~1,500 tokens	~200 tokens (structured JSON)	mechanical
3	Match to existing vendor / project	~800 tokens (candidate list + doc)	~50 tokens (match ID + confidence)	mechanical
4	Anomaly review — flag unusual items with reasoning	~2,000 tokens	~300 tokens (prose explanation)	judgment

Prices used (per million tokens, illustrative but in the current-market ballpark as of early August 2026): flash tier at $0.14 input / $0.28 output; frontier tier at $2.50 input / $10.00 output. Update these against live pricing when you run the numbers on your own workload — the whole point is that they change.

Baseline: everything on the frontier tier

Per document, using the frontier model for all four steps:

• Step 1: (500 × 2.50 + 30 × 10.00) / 1M = $0.00155
• Step 2: (1500 × 2.50 + 200 × 10.00) / 1M = $0.00575
• Step 3: (800 × 2.50 + 50 × 10.00) / 1M = $0.00250
• Step 4: (2000 × 2.50 + 300 × 10.00) / 1M = $0.00800

Per document: $0.01780. Per month at 5,000 docs: $89.00.

That’s not a bank-breaking number, and it’s exactly why nobody optimizes it. But we’re not done — that’s per pipeline. In practice, small shops run 3–5 such pipelines (invoices, email triage, support, listing generation), and the same math applied to each aggregates to $300–$500/month across the shop. Now it matters.

Routed: cheap for steps 1–3, frontier for step 4

• Step 1 (cheap): (500 × 0.14 + 30 × 0.28) / 1M = $0.0000784
• Step 2 (cheap): (1500 × 0.14 + 200 × 0.28) / 1M = $0.0002660
• Step 3 (cheap): (800 × 0.14 + 50 × 0.28) / 1M = $0.0001260
• Step 4 (frontier, unchanged): $0.00800

Per document: $0.008470. Per month at 5,000 docs: $42.35.

Savings: 52% at the pipeline level, and the visible line item drops by about $47/month per pipeline — call it $150–$200/month across the shop. Not life-changing individually; genuinely material as your automation surface grows.

Routed with escalation on steps 1–3

Now be honest about failures. Assume the cheap tier fails structurally (bad JSON or confidence < 0.7) on 8% of calls at step 1, 15% at step 2, 10% at step 3 — numbers I’ve seen for flash-tier models on unfamiliar document formats before prompt-tuning. Each failure escalates and you pay for both calls:

• Step 1 avg: 0.92 × $0.0000784 + 0.08 × ($0.0000784 + $0.00155) ≈ $0.000208
• Step 2 avg: 0.85 × $0.0002660 + 0.15 × ($0.0002660 + $0.00575) ≈ $0.001129
• Step 3 avg: 0.90 × $0.0001260 + 0.10 × ($0.0001260 + $0.00250) ≈ $0.000376
• Step 4: $0.00800

Per document: $0.009713. Per month at 5,000: $48.57.

Still ~45% cheaper than the baseline, and the escalation cost gives you an honest quality floor: any doc the cheap tier can’t handle confidently ends up on the frontier model anyway. Over time, as you prompt-tune the cheap tier and its failure rate falls (10% → 3% is typical after two rounds of fixing whatever the failure pattern actually is), that number keeps sliding down while smart-tier quality stays pinned.

The point of showing all three numbers is that “the router saves 60%” is a headline; the actual savings depend on your failure rate, and the failure rate is a knob you turn with prompt work. A ledger, a benchmark, and honesty about the escalation cost is the whole practice.

For the rest of the tooling side of small-biz automation — where these dollar figures fit alongside SaaS bills, POS costs, and the rest of the pile — the piece on running a business on spreadsheets vs. actual systems is the frame I keep coming back to. Automation is only worth it when the unit economics survive contact with the invoice.

Vendor-churn insurance

The other reason to route rather than pick: hosted model catalogs are churn-prone, and 2026 keeps proving it.

Two data points from this cycle: GitHub retired GitHub Models on July 30, and on July 31 deprecated Gemini 2.5 Pro and Gemini 3 Flash from Copilot. If your prototype or CI job depended on either of those endpoints, you had days, not months, to migrate. That’s not GitHub being unusually flaky — it’s the model layer being what it is right now. The frontier is moving fast; the vendors trailing it are pruning fast; even the winners rename tiers, deprecate versions, and change SKUs on a rolling basis.

The insurance is small and uninteresting:

• One interface, everywhere. OpenAI-compatible is the current lingua franca. The openai Python client points at any OpenAI-compatible base URL. That’s it, that’s the abstraction.
• Two vendors on each tier. Have a cheap-tier fallback (DeepSeek + Cloudflare Workers AI, say) and a frontier fallback (OpenAI + Anthropic). Store the endpoints and model names in environment variables. When one goes away, you edit env vars, not code.
• Prompts in your repo, not the vendor. No “workspace” features, no vendor-hosted prompt libraries you can’t git log. Portability is a boring skill that pays off on a two-hour timeline the week your provider changes something.
• Re-benchmark quarterly. New flash-tier models drop every few weeks. The one you picked in July may not be the price-performance winner in October. If the benchmark script exists (see above), rerunning it is a coffee’s worth of effort, not a project.

None of this is exotic. It’s the same posture that made cloud infrastructure workable a decade ago: assume any specific vendor’s specific product will change, price the abstraction accordingly, keep the switching cost artificially low.

What to do this week

If you’re a small operator with any live AI automation:

1. Pick one pipeline. The one you know best — probably the highest-volume one, or the one whose invoice line item has been growing.
2. Sample 50 real inputs. Whatever the natural unit is — emails, docs, tickets.
3. Run the benchmark template above against a flash-tier model (DeepSeek V4-Flash-0731 is a fine default; whatever’s on your existing vendor’s cheap tier works too) and your current model.
4. Ship the router with escalation on the steps where cheap accuracy is within striking distance.
5. Log costs to your own database. Not the vendor’s dashboard.
6. Set a calendar reminder for October to rerun the benchmark against whatever’s new. There will be something new.

The industry’s not going to stop shipping cheaper models. Cloudflare’s whole “Smaller, faster, safer” pitch is a bet that the interesting frontier for most workloads has moved down-market. DeepSeek’s release cadence is a bet that open weights close the gap fast. GPT-5.6’s Luna tier is OpenAI acknowledging the same trend from the top. When every serious player agrees, the honest response isn’t to keep paying for the top tier out of habit. It’s to build the two-tier switch, own the meter, and let the market’s price war show up on your P&L.

An afternoon of routing work, a lever, two tracks, one decision per call. That’s the whole trick.

Sources

• Simon Willison — “DeepSeek V4-Flash-0731” (Jul 31, 2026) — release of the flash-tier open-weight model that anchors the cheap side of the router pattern; ~3 days old at publish.
• Simon Willison — “The new GPT-5.6 family: Luna, Terra, Sol” (Jul 9, 2026) — GPT-5.6 launch overview, including OpenAI’s explicit price-performance positioning for the Luna tier; ~4 weeks old, but the July 30 Luna price drop referenced in the intro is days old.
• Cloudflare — “Smaller, faster, safer models” (Aug 2026) — Cloudflare’s write-up of running Kimi and GLM at scale on its own edge, and the industry direction toward smaller efficient inference.
• GitHub Changelog — “GitHub Models is now retired” (Jul 30, 2026) — evidence for the vendor-churn insurance section; hosted model catalogs sunset fast.
• GitHub Changelog — “Gemini 2.5 Pro and Gemini 3 Flash deprecated” (Jul 31, 2026) — same point, different vendor tier: even Copilot’s model catalog is a moving target.
• OpenAI API pricing and Anthropic pricing — canonical per-token rate references; plug live numbers into the PRICES dict in router.py.
• Prior ctrlaltorion pieces: Build your own AI spend meter and The spreadsheet is not a business system — the metering and the small-business framing this piece leans on.