A passing test can share the code’s mistake

The implementation and its witness agree

The price function returns £110 for a £100 subtotal. Its generated test expects £110, so the run is green. There is just one difficulty: the business rule says to subtract a fixed £10 discount before adding a 20 per cent surcharge, which gives £108. Code and test agree with one another because they have made the same ordering mistake. The passing assertion has become a rather convincing witness for the wrong answer.

A person can make that mistake too, then write the assertion that confirms it. Generating the implementation and test together makes the agreement arrive quickly, often looking complete enough to approve. I would spend the review effort on where the expected amount came from. Something outside that shared interpretation has to bring the £108 rule back into the discussion.

The example below is a small teaching fixture, with integer-pence subtotals from £10 to £200 inclusive. It applies the stated fixed discount, then rounds the surcharged amount to whole pence. It is not a production pricing library or a statement about tax. The downloadable Node script runs a few deliberately chosen implementation variants so the discussion has observable results rather than a general instruction to write better tests.

The important question is what each assertion can distinguish. Reaching a line, returning the expected type and producing one approved example are different kinds of evidence. A green result tells us that the checks we wrote accepted this implementation. It cannot tell us whether we asked the checks to establish the right thing.

The expected amount needs an owner

Work the intended case independently. £100 minus £10 is £90; adding 20 per cent gives £108. Reversing the operations gives £120 minus £10, or £110. The fixed discount is essential. Two percentage-only operations can conceal the ordering problem because multiplication can produce the same result either way, apart from relevant rounding details. Choose numbers that expose the disagreement you care about.

Put the plain-language rule beside the examples and identify its authority. In a real application, that may be an approved pricing policy or a product decision. The generated implementation is not the authority merely because it already exists. Nor should a reviewer treat an assistant's confident calculation of the expected answer as independent confirmation of that assistant's code.

The fixture deliberately includes the mistaken £110 expectation. It accepts the wrong-order implementation and rejects the intended one, showing that a test can reliably detect changes while enforcing the wrong behaviour. Before extending this suite, a reviewer needs to resolve the expected amount against the business rule. More assertions built on the same mistaken expectation would preserve that misunderstanding more thoroughly.

If the rule is genuinely ambiguous, somebody responsible for the product needs to decide it. Asking the assistant to make the suite green would only choose the version easiest to reconcile with the current code. A deliberate policy change may legitimately replace an old expectation. Leave its reason beside the examples so the next engineer can tell a changed business rule from a test adjusted to stop objecting.

// Teaching fixture: integer pence, subtotal from 1000 through 20000.
const intended = subtotal => Math.round((subtotal - 1000) * 120 / 100);
const wrongOrder = subtotal => Math.round(subtotal * 120 / 100) - 1000;
// For 10000 pence: intended→10800; wrongOrder→11000.

Give one example something to disagree with

Correcting the first expectation to £108 fixes the ordering example, but a constant function returning 10800 pence will also pass it. The check establishes one result without distinguishing the calculation from that shortcut. Choosing another subtotal is useful because it can expose a specific implementation that the current assertion accepts, giving a concrete reason for the next test beyond increasing coverage.

Add a £200 subtotal. The intended amount becomes £190 plus 20 per cent, or £228. The constant still returns £108 and is rejected. Add the lower boundary, £10: the fixed discount leaves zero, and the surcharge leaves zero. A variant that omits the surcharge passes that boundary while failing the other two cases. A passing boundary check should therefore not be mistaken for evidence of the whole calculation.

In the executed fixture, the intended implementation passes all three intended examples. The wrong-order variant passes none. The constant and no-surcharge variants each pass one. These are actual results from the small accompanying script, not model benchmarks or a claim that three examples establish correctness for every price. Each row answers a specific question about a plausible wrong implementation.

Try a small challenge in a disposable copy: what wrong implementation could pass this assertion? A constant return value suggests a second input whose answer must differ. An assertion that a mock was called invites a different question—could the customer still receive the wrong outcome after the expected call? You can use that habit before installing a mutation tool. Each added check then has a job, rather than being another line in an increasingly impressive test file.

A surviving change is an invitation to investigate

Mutation testing automates variations in implementation and observes whether tests reject them. Stryker distinguishes outcomes including killed, survived, no coverage and timeout.[1] Those categories are useful because they say more than whether the original suite is green. A surviving variant may expose a missing distinction the tests were supposed to make.

The accompanying script runs selected manual variants, rather than Stryker's mutation engine. Each variant represents an identified misunderstanding, which makes its result easy to inspect against the stated rule. An automated mutation tool can explore more changes, including some that require further interpretation. The manually chosen cases establish how these assertions behave; they cannot supply a general mutation score for the rest of the application.

One limitation is equivalent behaviour. The fixture includes a version with an explicit zero-result branch at the lower boundary. Within the stated input domain, it produces the same outputs as the intended function. The script checks all 19,001 integer inputs from 1000 through 20000 pence and confirms equality. No test of those public outputs should reject it merely because its internal branch differs.

Stryker discusses equivalent mutants and how difficult they can be to identify automatically.[2] A surviving change therefore needs a look, not an automatic defect label. Our explicit lower-bound branch should survive because it preserves every output in the stated domain. Something that agrees only on the three examples is a weaker case. Keep that difference in view before spending hours chasing a perfect score or dismissing a survivor the business rule actually needed the tests to reject.

Spend independence where the consequence lives

A complete application has more uncertainty than this pure function. Which subtotal entered the calculation? Was the fixed discount authorised for this order? Which rounding policy applies at each stage? Unit examples can establish a local rule while leaving those boundaries untouched. The next useful test may sit around the handover, not add another nearly identical number inside the same function. The fixture also rejects fractional pence and subtotals outside £10–£200. Those rejections belong to its input contract. If the application needs a smaller basket or a credit amount, it needs an explicit additional rule; silently clamping the input would create one without agreement. The equivalence result depends on that bounded domain too. It should not be carried into a wider domain merely because the function name stays the same.

For AI-assisted changes, I would focus independent oracles on money, access, irreversible operations and customer promises. Use lighter checks where a straightforward transformation has little consequence. A small team cannot review every generated assertion with equal intensity. A deliberate allocation of attention is more credible than approving a large suite nobody has time to read.

Keep the wrong-order case, the constant and the lower boundary beside the pricing rule, together with the equivalent version that changes no output in the declared domain. If a later policy makes £110 correct, a reviewer can then explain why £108 is being replaced. The cases distinguish a changed commercial decision from an implementation shortcut without treating every internal difference as a defect.