What Would Make the Challenger Lose?

Most uncertainty in an analysis is irrelevant, and treating all of it as equally worth reducing is how a six-week project becomes a six-month project.

The uncertainty that matters is the uncertainty that could make the challenger lose, to the incumbent or to another challenger. Everything else is scenery, however large the error bars. A recommendation that leads with a single number invites the room to treat the ranking as a fact, and when the next cycle produces a different number everyone behaves as though the mathematics betrayed them. The calculation did not fail. The presentation concealed the only uncertainty that mattered.

The worlds in which you would pick differently

The most useful single question is what would have to be true for the team to choose B instead. The answer is a short list of worlds, and writing it is most of the work, because once it exists the team knows what to analyze, what to skip, and what to carry into the write-up. If the worlds are plausible, gather evidence, hedge, stage the rule, or choose a more robust alternative.

The grouping rule applies to uncertainty as it does to claims. “The extract might have a duplicate key” is a note on data quality. “Dealers might bypass the protected class” is a condition under which the challenger loses. Listing them together under “risks” teaches the owner to treat both as background noise.

Some of what could reverse the pick is a later choice this one constrains. Finance’s blanket cut, which brings every item down 20 percent by the freeze and then replenishes under the current rule at targets 20 percent lower, liquidates protected items along with the rest and spends an option that cannot be bought back within a quarter. And whose appetite for the remaining risk the range is calibrated to is not yours to assume: Priya Shah may accept an 8 percent bypass trigger while finance will not carry the cash event across a quarter of learning. Name that as a disagreement about the tail, or it will surface as an argument about the third digit.

Ranges where a range changes someone’s stomach

Give a range when the range would change what someone is willing to run.

We recommend a class-based replenishment rule. That ranking survives if missed-job cost on the protected class is anywhere from roughly half to twice our base case. It breaks if dealers can bypass the system on the protected class, or if finance forbids item-level exceptions.

Notice what is not in that paragraph: a fabricated 87 percent confidence. Confidence belongs to claims, and even then as language rather than as jewelry. “We are willing to run this for a quarter” is a real sentence about operational confidence. A p-value is almost never the reason a vice president should change a rule.

Draw each range wider than feels comfortable. Analysts set it too close to the base case because a wide range feels like not knowing. Use the range a person who had not seen the model would defend, and if the ranking dies inside it, what remains is a point estimate wearing a sensitivity slide.

The one-way table

A one-way table is usually enough. Rows are the inputs that actually argue. Mark the value at which the recommendation changes.

If this is true	Then pick
Missed-job cost roughly uniform across items	The incumbent, or a least-damaging cut if the cash event is mandatory. Class-based stocking is theater.
Bypass on the protected class cannot be closed	Not a class-based rule, because you cannot enforce it.
Cash event mandatory and exceptions forbidden	You are no longer ranking classes against a haircut. You are designing the haircut.
Missed-job cost on the protected class anywhere from half to twice the base case	The class-based rule. Stop arguing about the third digit.

The table does something a paragraph cannot. It lets people argue about the inputs, and it lets them see, sometimes for the first time, that they are not arguing about your personality. If two inputs jointly flip the ranking in a way neither does alone, write that sentence; do not produce a twelve-dimensional tornado chart because you can.

Show the list before the pick, not after

Shown before the recommendation, the flip conditions are an invitation. Here are the worlds in which we would choose differently: missed-job cost is roughly uniform across items, dealer bypass cannot be closed, exceptions are forbidden outright. There is no pick on the page yet, so nobody is defending anything, and anyone who knows one of those worlds is real can say so while it still changes the work.

Shown after the recommendation, the identical list reads as a set of conditions attached to a position. Every threshold is now a lever, and the meeting becomes a fight about the unless rather than a conversation about the decision.

Same worlds. Different meeting. So the list goes to the people with standing, Priya and the finance partner, before it is attached to anything, while they can still answer as owners rather than as partisans.

Value of information is a delay tax

“Let us get more data first” is a proposal with a price: the freeze that passes, the quarter that runs on the incumbent, the cash event that happens anyway. If what might be learned could change the ranking and the delay is cheap, learn it. If it could and the delay is expensive, state the tradeoff, because accepting the delay is itself a choice. If it could not, stop.

The decision not to pursue evidence belongs in the evidence: “we did not rebuild the forecast this cycle because even a large accuracy gain would not reassign the items that hold the cash.” The dishonest way to stop is declining a cheap test because the candidate is attractive and the test might kill it.

Three kinds of condition

The phrase “flip condition” covers three different objects, and a write-up that does not tell them apart invites the wrong argument about each.

An analysis stopping rule governs the work before the commitment. It says when further evidence could no longer change the ranking: miss cost on the protected class can range from half to twice the base case without changing the pick, so the goodwill term is not worth another week. It belongs to the analyst and lapses at the commitment.

A deployment trigger governs the rule after the commitment, for uncertainty that resolves only once the rule is running. It names an observable quantity, a threshold, a window, the person who holds the switch, and the action: a distribution center reverts to the single rule if bypass of the protected class exceeds 8 percent for four weeks. It belongs to the operating owner, is agreed before launch, and is executed rather than debated.

A choice left open arises when the point at which the ranking reverses lies inside the range the room considers plausible, and no evidence obtainable before the deadline will narrow it. Analysis cannot settle it and a trigger cannot stand in for it, because there is not yet a ranking to protect. It goes to the owner as a judgment, stated in one sentence with the value the pick turns on, or becomes a staged trial read against a rule written in advance.

Confuse them and each fails its own way: a stopping rule mistaken for a trigger fills the write-up with conditions nobody monitors, a trigger mistaken for a stopping rule produces a launch with no agreed reversal, and a choice left open, presented as either, hides the one judgment the owner was needed for.

Robust and reversible beat optimal on a point

An alternative that stays good across the plausible worlds usually beats one that is best at the base case and poor elsewhere, because the base case is a guess. Robustness is not splitting the difference: a uniform 10 percent cut is not robust merely because it lies between zero and twenty.

Ask separately what each choice costs to undo, because the alternatives frequently differ on it more than on expected value. Reversibility lets you use information you do not have yet, so its worth depends on whether the uncertainty that matters will resolve, and how soon. Assess it per alternative, and expect surprises. The blanket cut looks reversible, since you can always buy inventory back, but liquidated parts have lead times measured in months and burned dealers change their ordering for good. The class rule looks harder to unwind and is easier, because the classification is a parameter and the fallback is the rule running today. A pilot earns its place only if it resolves the uncertainty that drives the ranking, has an end date, and has a rule written in advance saying which result leads to which decision.

“The room wants a number”

Give them a number, and then immediately give them the flip: we expect about this much less cash in the cheap class, and about this many fewer vehicle-stopping misses, unless the bypass cannot be closed. Put the condition in the top third of the note, beside the reasons. If the room will not hear the condition at all, you are no longer in a recommending job. You are in an informing job or a power fight, and more decimal places will not help you in either.

The test

Write the sentence “we would pick something else if” and finish it three times. If you cannot finish it once, you do not yet understand the recommendation. If you can, label each ending as a stopping rule, a trigger, or a choice left open, and put a name next to every trigger.