The Analysis Is Not the Product

A vice president reports that dealer inventory is too high and fill rate is still deteriorating, and asks for a dashboard in time for Friday’s meeting. A competent team does what it was taught to do. It extracts data, explores it, fits a model, charts the exceptions, and assembles a presentation. Forty-seven slides later, someone senior asks the only question that mattered at the beginning: what, exactly, are we supposed to do?

The meeting ends in agreement that the work was thorough. On Monday the replenishment system places orders under the same rule it used the week before.

The work was not empty. It was aimed at the wrong product, in a contest it did not know it had entered.

Every decision already has a rule

The replenishment system did not wait for Friday. Every decision an analyst is asked to inform already has a rule firing underneath it, and that rule is the incumbent. Sometimes it is documented: a min/max, a staffing standard, last quarter’s pricing policy. More often its working parts are not: an unofficial spreadsheet, a pad a planner adds after the system’s number lands, a promise between two supervisors, the value a field takes when nobody touches it, the judgment of the person who has done the job for twenty years.

The analysis competes with that rule. Its product is a challenger: a rule that beats the incumbent, gets installed where the incumbent ran, and carries a built-in way to tell whether it was wrong.

Three consequences follow. The baseline is the incumbent as it actually runs, not the documented rule; a challenger measured against the documentation gets credit for gains the pad already captures and escapes blame for failures the pad prevents. The incumbent starts ahead, because it is trusted, familiar, the default on someone’s screen, and needs nobody’s signature, so a challenger that is merely better on average usually loses. And the product is not finished when the ranking is: a challenger not installed has replaced nothing, and one installed without a stop condition can only be defended or abandoned on opinion.

Read the artifacts and the substitution is visible

The default week leaves recognizable objects behind. Key Insights is a set of findings, which can be true and commit nobody. Root Causes groups a process issue, a metric, a complaint, and a data-quality note under a heading that cannot be summarized as one thought. Next Steps, where a decision might have appeared, lists further work. That is a plan to keep working while the incumbent keeps running.

Set that against a finished product for the same request. By the April freeze, Priya will choose among keeping the current replenishment rule, taking finance’s cut (every item’s stock down 20 percent by the freeze, then the current rule at targets 20 percent lower), or replacing it with a class-based stocking rule that protects the parts that stop vehicles and cuts the cheap, slow ones. We recommend the third, because the excess cash and the missed jobs sit in different items and the current rule cannot tell them apart. It would lose if dealers can bypass the protected class at will. Parts Planning owns it.

The slide pack can only be admired. The paragraph can be disagreed with, which is what makes it useful.

Why labor gets mistaken for rigor

The substitution is not laziness. Labor feels like rigor, but rigor in this job is whether the work could have changed the ranking of alternatives a named owner can choose among. Three forces keep the default week in place, each reasonable where it operates. Technical training rewards the quality of the analysis, and a person can be promoted for a model that never changed an order. Organizations ask for visibility or a dashboard because those requests are socially safe: nobody has to name alternatives or admit who owns the call. And teams present work in the order they performed it, extract first and recommendation last, which tells the room that inspecting the work is the business of the meeting when the business is whether to replace the rule.

A dashboard also lets the room stay uncommitted, and an uncommitted room has not deferred the decision. It has chosen the incumbent for another cycle, with nobody’s name on the choice.

The same failure wears other costumes

A forecast team spends a quarter shaving error and reports a two-point improvement. Purchase orders do not move, because planners pad the forecast by a margin learned from years of being short, and the pad absorbs the improvement. The missing product was a change to the rule that consumes the forecast, pad included, or an honest finding that the rule is insensitive to the error.

A machine-learning team delivers a respectable churn model. Retention still runs on a quarterly campaign list assembled in a spreadsheet, because nobody named the choice the score was meant to change. The spreadsheet was the incumbent, and the model was never specified as a rule that could replace it.

In both cases the technical work can be excellent and the product unfinished. A finding cannot be installed.

The unit of work is the decision, and it comes in two kinds

The unit is a commitment to one alternative over others, by somebody who can make it, under uncertainty, that will enter an operating process. “Inventory is too high” is a problem and can leave nothing to install; “fill will improve” is a hope about a scoreboard. If you cannot say what somebody will do differently if you are right, there is not yet decision work to do. (Discovery, monitoring, and research have purposes of their own; this method concerns work commissioned to change what an organization does.)

A one-shot happens once: close a warehouse, sign a contract. A repeating rule fires whenever the situation recurs: how you replenish, allocate, expedite, staff, and set thresholds. Most analytics, data science, and operations research inside companies is the second kind, even when requested as the first, and a one-time recommendation delivered into a repeating process is absorbed by the incumbent within a cycle.

Five questions define the product

If the product is a challenger that beats the incumbent, gets installed, and can be stopped, five questions say what has to be known before the work is finished. They are also the best questions to ask before it starts.

1. What rule runs today? The incumbent as it actually operates, including the parts nobody wrote down, and why it has survived.
2. Who can change it, and by when? An owner with standing to replace the rule, and the date after which the choice goes stale.
3. What could replace it? A menu of challengers, including the incumbent itself and at least one alternative the request had no words for.
4. What would make the challenger lose? The claims and conditions that would change the ranking, and the cheapest evidence that tests them.
5. How will it run, and how will we know to stop it? The rule specified so a process can run it, with an owner, a revisit date, and a condition for switching it off.

The first three produce the choice: the alternatives, including the status quo, who can commit, by when, and whether the decision fires once or repeats. The fourth produces the evidence: what could change the ranking, as distinct from what would be interesting to know. The fifth produces the rule: what the process will do, who runs it, and when it is revisited. Choice, evidence, and rule are compact vocabulary for the answers, not a separate framework, and the order matters. Without a named choice, evidence has no stopping rule; without evidence that could change a ranking, a proposed rule is a preference.

The answers fit on one page, and the page is the decision product. For Priya it names the rule today (one min/max at one service target, padded by hand, with dealers buying elsewhere when the shelf is empty), the owner and date (Priya, with finance owning this cycle’s cash target, by the April freeze), the three challengers, the recommendation (classes at the two central distribution centers), what would make it lose (dealer bypass of the protected class, or a missed job costing about the same for every part), and how it runs and stops (Parts Planning owns the classes, the class quantity is what the planning screen proposes, and a center returns to the single rule if bypass passes a stated threshold). Every line can be refused, which is the point. Appendix C writes the decision out in full.

“But we have to look at the data first”

The objection contains a limited truth: a few days may show that the live alternatives are not the ones in the request, because the rule is not being followed or a freeze nobody mentioned is holding stock. That argues for a cheap first pass at the choice, not an unbounded tour of the warehouse. Write the sentence, see whether it is about the wrong thing, and rewrite it if so. Exploration aimed at a sentence stops when further cuts cannot change the ranking; exploration without one never stops. And the first look should usually be at the incumbent: compare what the system recommended with what was actually done, find the person who makes the difference, and read what they built.

The diagnostic

Take the last substantial study your team produced. What rule was it trying to replace? Which alternative was chosen because of it, by whom? What does an ordinary Tuesday do differently now? If the answers are nothing, that is a verdict on the brief rather than on the analyst.

Then count across a year: how many of your function’s analyses replaced an operating rule? Most organizations have never counted. It is the most useful number a technical leader can have, because it cannot be met by working harder.