Work Backward from the Choice

The default order is to look at the data, see what it says, and form a view. The order this method argues for is to form a view, write down what would prove it wrong, and then go looking.

Once the menu exists, the question changes from what could replace the incumbent rule to what would make a challenger lose to it. The working order runs down from there: candidate pick, claims that must hold for it to win, evidence that could kill them, analysis that produces that evidence, and only then data. Most teams run it the other way, and even when their story is true it is true by accident.

“Is that not just confirmation bias with extra steps?”

It would be, if the candidate ranking were protected. It is not. The entire point of writing it down is that it becomes a target.

An unstated hypothesis cannot be falsified, because it can quietly deform to fit whatever the data shows. Analysts who claim to approach data without a prior have a prior; they simply cannot be held to it. Writing it down converts a bias you cannot see into a claim you can attack.

The discipline that makes this work is that the claims are written before any query runs, and each one carries, in advance, the result that would falsify it. “If the cash and the misses sit in the same items, class-based stocking is dead” is a claim with a kill condition attached.

The sequence

Write the candidate ranking. What you would pick this afternoon if forced to choose among the incumbent and its challengers, and why you suspect it. It is allowed to be wrong. It is not allowed to be blank or unspoken. “Let’s see what the data says” sounds humble and gives up the only thing that makes looking at the data finite.

Write the claims that must hold for it to win. Each one, if false, would change whether the candidate beats the other alternatives. Decide now what result falsifies each.

Choose the cheapest test that could change your mind, and run it. A candidate that dies in week two is a successful week two.

Exhibit 1The work runs one way and the note runs the other, on purpose

A claim that cannot be killed is not a claim

“We should be more data-driven” cannot be killed. “The top sixth of items by missed-job cost accounted for most of that cost last year” can.

Write claims so that a reasonable analysis could embarrass you. If you are unwilling to be embarrassed, you are writing a brochure.

There is a craft failure here that looks like rigor: hedging a claim until nothing could kill it. “There may be some concentration of missed-job cost in a subset of items” cannot change a ranking. Tighten it until a query could prove it wrong, then run the query.

If you cannot imagine a result that would move an item from “reason to pick A” to “reason to pick B,” it does not belong in the claim list.

The cheap first cut is usually a query and a conversation

At Meridian Motors the candidate is a class-based replenishment rule, and its load-bearing claims are that a small set of items drives most of the missed-job cost, that a large set holds most of the idle cash, that the current rule does not distinguish those sets, that planners and dealers will actually follow a class-based rule, and that getting a class wrong costs less than the status quo. For each, ask what the least expensive thing is that could kill it. The answers are almost always smaller than the instinct.

Are the damage and the cash concentrated? A Pareto of last year’s traces and of holding dollars, plus a planner workshop on what actually stops a vehicle.

Does the current rule already see the difference? Compare the rule’s existing parameters against those two Paretos. If it already differentiates the right items, the candidate dies today.

Will they follow it? That one is not a query at all. It is a conversation with dealer operations and a look at how often the current rule is bypassed. If the bypass cannot be closed, the candidate is a brochure.

Are the costs in the same universe? A rough ratio of a miss on a water pump to a miss on a floor mat.

If the cheap tests kill the candidate, rewrite the candidate and repeat the chain rather than rescuing it with a more elaborate method: if missed-job cost is uniform, a mixed-integer program will not make class-based stocking a good idea, only an expensive one.

Keeping the incumbent is a recommendation too

Working backward sometimes ends with the incumbent winning. That is a ranking, not a shrug, provided it comes with a stated margin, stated assumptions, and the conditions that would reverse it. Agree a threshold with the owner before any test runs, work the claims backward from it, and order the tests by cost.

Data last

This is a claim about ordering, not about importance. Data gathered before the claims exist is gathered against no standard, which is why exploratory phases expand without bound: there is no criterion for when to stop. Data gathered to test a named claim has a finish line. You stop when the claim is settled.

Some of the most load-bearing claims will not live in a tidy table: whether dealers will game a new rule, whether finance will honor item-level exceptions, whether the expedite path is already eating the current rule. Those are still evidence. Interviews, expedite traces, and a week of watching planners count as much as a query.

Write the claims first, then notice which of them the schema cannot reach. Those are often the ones that actually rank the alternatives.

The ways you will fool yourself

Having a candidate makes you more efficient and worse at seeing evidence against it. The only defense is procedural.

The candidate anchors you. The 20 percent in the email becomes the benchmark, and you ask whether the class rule saves more or less than 20 percent rather than what it saves against the incumbent. Write down the quantities that matter, in the units you care about, before you look them up.

You will notice the cuts that support the candidate. The encouraging query gets a chart and the ambiguous one gets a note to follow up. The countermeasure is the refutation condition, written before the query so the result cannot be reinterpreted afterward.

Sunk cost arrives late. By week four the model works, and the evidence says it ranks no better than the current rule. Ask what you would recommend if the model had been handed to you that morning by a stranger. If the honest answer differs from the one you are drafting, the difference is the cost of the work already done, and the organization should not pay for it twice.

“But we might miss something”

You might. The goal is completeness of the ranking, not completeness of the notebook. If the candidate named the wrong alternatives, rewrite the choice sentence rather than exploring without one. And because a claim list written from intuition covers only what the analyst was already thinking about, cut the question into parts deliberately, which is the other half of the chapter.

The discipline is to work backward until the remaining uncertainty is named rather than vague. Named uncertainty becomes a condition on the recommendation, which you can carry honestly. Vague uncertainty becomes more weeks.

A test you can run in five minutes

Write one sentence naming the alternative you would pick this afternoon. Under it, write the claims that must hold for it to beat the incumbent, and next to each the result that would kill it. Any claim without a kill condition is a slogan; any claim whose cheapest test is a platform has not been thought about yet.