Not Every Step Deserves Your Best Model

When people wire up an AI workflow, they usually pick one model — the best one — and run every step through it. It feels safe: the smart model won’t botch the easy parts. It’s also how you turn a workflow that should cost pennies into one that costs real money, because most of the steps in a typical loop don’t need reasoning at all. They need a tag, a score, a yes/no, a bit of formatting. Running your most expensive model on those is like running the deep fryer to boil an egg. The discipline that fixes it is model mixing: route each step to the cheapest model that can actually do it.

A workflow is a mix of hard steps and easy ones

Look closely at a real multi-step pipeline and the steps are wildly uneven in how much thinking they require:

• Genuinely hard: analyze this transcript, synthesize these sources, reason through this design, write this code. These need the strong model, and using it here is the point.
• Trivial and high-volume: tag this item, score it 1–5, is this relevant yes/no, does this contain fluff, reformat this into that shape. A cheap, fast model does these fine, often at a fraction of the cost — the small models are startlingly capable at narrow, well-scoped tasks.

Defaulting the whole pipeline to the expensive model means you’re paying premium rates for a pile of steps a budget model would have nailed. The skill is telling the two apart and routing accordingly, step by step, instead of picking one model for everything.

Use the powerful model where thinking matters, and the cheap one everywhere else. You don’t run the deep fryer to boil an egg.

The cheap model belongs on the loop’s hot path

Here’s the part that matters most and gets missed: in a loop, the verify or scoring step runs every single iteration, while the hard reasoning might happen once. So the highest-frequency step is usually the least demanding one — and that’s exactly where a cheap model pays off most, because you’re multiplying its per-call cost by every pass of the loop. Put the budget model on the check that runs constantly, reserve the expensive model for the occasional hard move, and the loop’s total cost collapses.

That’s also, conveniently, sound design for other reasons. The step that decides “is this done yet?” should be a hard, external, cheap-to-run check anyway — and a small fast model running a tight verification rule is a perfectly good instrument for it. The cost win and the correctness win point the same direction: keep the frequent check lightweight.

This is the lever that makes always-on affordable

The reason to care isn’t penny-pinching for its own sake — it’s that model mixing is what moves an always-on automation from “too expensive to leave running” to “costs less than the coffee I drink reading its output.” A loop re-reads its context every pass, so its cost grows with each iteration; putting the expensive model on every one of those passes is how a background job quietly runs up a bill. Mix in the cheap model on the frequent, low-thought steps and the same loop becomes cheap enough to run continuously, which is the whole reason you wanted a loop.

Cost-consciousness, made concrete

This is the constructive, per-step twin of a point I’ve made about AI assistants having a meter running: knowing the meter exists is the mindset; model mixing is the specific technique that acts on it inside a workflow. Instead of a vague “use less,” it’s a concrete design rule — for each step, ask what the smallest model that does this well is, and route there. Escalate to the strong model on purpose, for the steps that genuinely reason, not by reflex for all of them.

The exact price ratios between model tiers shift as providers reprice, so don’t over-fit to a number — the durable rule is the shape: match the model to the difficulty of the step, and put the cheap one on the parts that repeat. Do that and you get most of the quality of an all-premium pipeline at a small fraction of the cost, which is what makes ambitious automation actually sustainable to run. It’s a big part of how I keep the AI automation lab from becoming a money pit. If you’ve tuned a workflow by mixing model tiers, I’d love to hear where you drew the lines.