Losing families: how we bench some signals without quietly deleting them

Every signal engine has its losing families: asset × direction combinations that historically get it wrong more often than right. The temptation to quietly delete them from the backtest is immense — the hit rate climbs, nobody asks questions. That's exactly the kind of cherry-picking we forbid ourselves.

The problem with quiet "cleanup"

If we remove the families that lost after the fact, we prove nothing anymore: we're describing the past, not predicting the future. The only honest way to remove a family is to decide in advance which one, on what criteria, then measure — and publish the result even if it contradicts us.

So before any calculation, we froze the list of suspect families: VIX up, oil momentum, S&P 500 down, and the very short (≤ 7 days) legs of multi-horizon signals.

The measured result

Across two independent backtest runs, removing these families from the traded flow lifts accuracy from 66.7% to 70.8% (and from 68.0% to 71.2% on the second run). The criterion we'd set — that the entire confidence interval shifts upward — is met on both runs. The removed families, taken alone, run at 40-44%: they were indeed losing.

But we also publish the gaps with our starting intuition. Oil momentum, which we thought was catastrophic (25-36%), actually runs at 50-57% on these runs: it's a candidate for rehabilitation, not a convict. S&P 500 down had no resolved decisions over the period. We write it as it is.

The way back isn't decided by gut feeling

A family put "under observation" isn't deleted: it keeps being evaluated. Every time one of its signals is blocked, we measure what the price did next — the counterfactual. If, on a sufficient sample, the blocks turned out mostly winning, the family would be automatically reopened. Rehabilitation is data-driven, never a mood call. No one, not even us, can re-authorise a family "because it feels right."

That's the principle behind all of GeoPulse: the scope is frozen in advance, the numbers decide, and the full report is public — including where it contradicts our starting assumptions.