A number that looks authoritative

We spent a while on a regime classifier: something to label the market as trending, mean-reverting or choppy, and attach a confidence score to the label. The obvious next step was to put that score on the dashboard.

We didn't, and the reasoning is worth writing down, because the same argument applies far beyond regime classification.

Computing and displaying are different decisions

A number that exists in a research notebook is a hypothesis. The same number rendered on a dashboard, in a clean typeface, next to real positions, is an instrument. Nobody makes a conscious decision to start trusting it. It is simply there every morning, and after a few weeks of being there it acquires the authority of everything else on the screen.

The classifier might well have been fine. That was never the question. The question was whether a number we had not validated should be given a place that implies we had.

Precision reads as accuracy

Confidence scores are especially bad for this, because they carry a decimal point. A label that says choppy invites a judgement. A label that says choppy, 0.78 invites arithmetic. The second one looks like it came from somewhere, and the somewhere is doing a lot of unearned work.

This is not a claim that confidence estimates are meaningless. It is a claim that the visual grammar of a dashboard — aligned figures, consistent formatting, everything sitting at the same level of apparent rigour — strips the caveats off anything you put in it. The caveat lives in the notebook. It does not survive the journey to the screen.

The bar we settled on

Anything displayed alongside live positions has to have survived the same tests as a rule that trades:

• Out-of-sample on data the idea was not developed on.
• Replicated across more than one market, without retuning between them.
• Compared against the simplest thing that already works, not against nothing.

That last one is the one people skip. There is almost always an existing, duller mechanism doing the job adequately. A new method has to beat it, not merely function.

The regime classifier failed on the third point rather than the first two. Something simpler was already routing between models using a feature that had already been validated for a different purpose, and which added no new parameters. A composite score with several new parameters would have had to beat that, out of sample, before it earned any screen space. It hadn't been asked to.

Dashboards accrete

The reason to be strict about this is that dashboards grow one addition at a time. Each individual number seems harmless — it's only a diagnostic, you'll remember it's provisional, it's just there for context. None of those sentences survive six months and a stressful morning.

We would rather have a sparse dashboard where everything on it has been tested, than a rich one where we have to remember which half to ignore. The discipline is not "don't compute speculative things." Compute as many as you like. The discipline is that the screen is not the notebook, and moving a number from one to the other is a decision with its own burden of proof.

What this costs

Honesty about the trade: we probably lose some real information this way. A provisional signal that would have been mildly useful sits unused because it hasn't cleared the bar. That is a genuine cost and we accept it, because the alternative failure mode — acting on a number because it looked official — is both more likely and harder to detect after the fact.

A wrong number you ignored costs nothing. A wrong number you trusted costs whatever you did about it.