The only backtest we trust: parity with the live engine

The dirty secret of backtests is that you can make almost any strategy shine in the past. You just have to let a tiny bit of the future leak into the calculation. Most of the "+200% over 5 years" charts you're shown don't measure an ability to predict: they measure an ability to fit once you already know the answer. For us, a backtest is only worth something if it replays exactly what the engine would have done live, never knowing anything about the future.

The three ways to cheat — often without meaning to

Look-ahead bias sneaks into a backtest through three doors, and almost always in good faith:

• Weights optimized after the fact. You calibrate the parameters over the whole period, then "test" on that same period. Of course it works: you learned the answers before sitting the exam.
• Future data leaking in. A news item dated today used to "explain" a move from six months ago. Or worse: re-scoring an old dispatch with today's NLP model, which has since seen how things played out.
• Cherry-picking rules afterward. Keep the signal families that won, erase the rest from the ledger. The hit rate climbs, but you're no longer predicting anything.

Replaying the live pipeline, dated to the day

Our parity backtest fixes all three by construction. It doesn't "simulate" an idealized strategy: it replays 365 days through the exact production pipeline, and for each date T it only has access to what we knew at T.

• The market context is reconstructed as of date T: prices, volatility, macro indicators, as they were on that day, never after.
• The rule weights evolve walk-forward from their initial value — not the final calibrated weights, but the ones the engine would actually have held, learned progressively over simulated time.
• The news corpus is filtered on scored_at ≤ T: only dispatches already published and already scored by that date enter the calculation. Nothing from the future.
• And above all, they're the same emission gates as the live engine — confidence floor, fundamental veto, abstentions on contradiction. The decision code is literally shared between live and backtest; there is no more lenient "backtest version."

What it costs, and why we accept it

This rigor has a price. Replaying the entire pipeline day after day is slow — the full run executes autonomously once a week, on Sundays, and can resume if interrupted. And the result is less flattering than a classic backtest: with no leaking future, no perfect weights, no cherry-picked winners, the numbers are necessarily more modest. That's exactly what we're after.

This isn't one demonstration among many: the latest finished parity run is the source of the numbers on display. The hit rate, the calibration, the rule health you read on the Truth Table all come out of that same corpus — which is why they shift slightly from one week to the next, in step with each new run.

A backtest that faithfully replays the past produces numbers we don't always like. But it's the only version that has anything to do with what will happen to you tomorrow. The scope is fixed in advance, the pipeline is shared, and the numbers — flattering or not — fall out on their own.