The cheap route has a hidden decision in it

Someone has to decide what easy means

The long message asks for opening hours. The short one contains an unusual contractual condition. If a router calls the first difficult and the second easy, cheaper model calls have acquired a new source of mistakes before either model starts its assigned job. Choosing what counts as routine is itself a decision to test, including the work created when that decision is wrong.

Anthropic’s workflow description separates routing from the specialised follow-up task: an input is classified and directed to an appropriate path, with reliable classification important to the pattern. [1] That is a useful architecture. It also gives us a specific place to investigate when a supposedly cheaper workflow produces corrections, escalations or lost information further downstream.

The hundred tasks below are invented, and the common cost units are planning assumptions rather than vendor prices or observed customer results. Their purpose is to show where the proposed saving comes from and how it can disappear. Counting only successful generations would answer a tidy question about calls. Most teams are asking a less tidy one: how much does it cost to get this work to an acceptable result?

First make the optimistic case explicit

Assume sending all 100 tasks down a larger-model path costs 8 units each, for a baseline of 800. In this teaching scenario, that path meets the required outcome standard. This is an assumption, not a claim that a larger model is always correct. A real comparison needs its failures and repairs included too; otherwise we would be pricing an imaginary perfect alternative against a fully burdened candidate.

Now add a router costing 1 unit for each task. It sends 60 tasks to a small path at 1 unit each and 40 to the large path at 8 units each. Router 100 plus small 60 plus large 320 equals 480. Against 800, that appears to save 320 units. The router itself accounts for 100 units, so omitting it would overstate the gain before we have considered a single wrong decision.

The 60/40 split tells us where the router sent the tasks, not whether those were the right destinations. That difference is easy to lose in a report: routed-small can quietly become suitable-for-small in the column heading. Keep the route as an observation and judge suitability against the required outcome. Otherwise the conclusion has been inserted into the data before anyone has inspected what the cheaper path produced.

Five errors change the arithmetic

Suppose five complex tasks were sent down the small path and produced outcomes needing repair. Assign each repair an illustrative 80 units, using the same common cost basis as the other figures. Repairs then add 400, taking the routed total from 480 to 880. The nominal 320 unit saving becomes an 80 unit increase. Nothing about the generation bill was miscalculated; it simply did not contain the whole consequence.

Why might a repair be expensive? An incorrect draft could require someone to reconstruct the source packet, remove a commitment and ask the customer for information already supplied. A misclassified integration job could enter a path that omits a required approval, leaving someone to reconcile the result. These are mechanisms to investigate, not measured 80 unit events. In a real exercise, define which correction counts, who performs it and how its cost is estimated.

Dividing the nominal 320-unit saving by five misroutes gives a break-even cost of 64 units per repair. Below that value, routing can still save under the stated assumptions; above it, repairs exhaust the saving. Some effects should instead be treated as hard limits. An unauthorised action or unacceptable disclosure cannot be justified simply because its estimated average cost leaves the overall comparison favourable.

At that break-even point, inspect what one repair includes. If the eighty units count only rewriting the draft, they may omit the work of finding the mistake or correcting a commitment already sent. If they include those tasks, adding them again elsewhere would exaggerate the penalty. The downloadable worksheet keeps the common units visible so the comparison can be challenged at that level. The arithmetic becomes useful when a disagreement concerns the actual work included, rather than whether five times eighty is four hundred.

A fallback only helps if it knows when to act

Escalation offers a different cost scenario. If the smaller path identifies all five problematic tasks before any effect, five additional large calls at 8 units add 40 rather than 400. The total becomes 520 before detection overhead. That result depends on catching every one of those cases early, so a real trial needs evidence for the detector as well as for the two model paths. If it flags ten cases instead of five, the additional large calls alone cost eighty units. Better coverage can therefore change the price of the fallback as well as its outcome.

The system needs a signal that catches those cases. Self-reported confidence may be useful evidence to test; it should not be treated as a calibrated guarantee because it looks like a percentage. A deterministic missing-field check may identify one failure while missing a false but well-formed claim. Review the signal against the kinds of errors that matter, including cases where the small path sounds confident and the result is wrong.

Detection has costs of its own: another model call, a retrieval step, a person’s attention or an added wait before the answer. Add them to the same route calculation. Also distinguish escalation before an external effect from repair afterwards. A larger model can produce a better second draft; it cannot retroactively make a message unsent or a customer’s reliance on the first answer disappear.

A useful design may route on information already known outside the message: task type, required tools, document count or a policy boundary. That can be easier to inspect than asking another model to infer an abstract difficulty score. It can also be too coarse. A routine category may contain one exceptional request. The choice is empirical: does the routing signal preserve the outcomes for the actual mix of work, at the cost claimed?

Count paths the customer experiences

Keep a trace for each trial task: chosen route, relevant reason, whether escalation occurred, final outcome, correction required and total cost under the stated accounting. Compare that trace with the all-large baseline on the same tasks. If the candidate solves a different problem or receives better source material, the result may still be useful, but it is a workflow comparison rather than a clean claim about model size.

Look at categories alongside the total, especially the important requests a route mishandles. Sending an ordinary question to the expensive path costs differently from sending an important one into a path that drops an obligation. Sample accepted small-path outputs as well as escalations. A shorter exception queue could mean fewer mistakes, or a detector that has stopped recognising them. Keep the original request, chosen route and supported outcome together so the trial can distinguish those explanations; counting only the cases sent for help cannot do it.

The right outcome may be a simple rule, a narrower small-model assignment or no router at the current volume. Complexity has maintenance costs even when a particular month’s arithmetic works. Record the assumptions that would change the decision: workload mix, repair burden, detection quality and the cost of the baseline. Revisit them when the business starts using the tool for a different kind of work.

The smaller path may be economical on a narrow assignment whose accepted outputs meet the required standard. The hundred-task comparison gives that proposition some questions to answer: what does routing cost, which errors are caught before effects, and what remains to repair afterwards? Until the detector has been checked on confidently wrong answers, 520 units is still a scenario. At this workload, a simpler rule or a direct larger-model route may be the easier decision to defend.