Take any strategy, fit it to twenty years of history, and report the result: you have just measured how well your model memorised the past, not how it handles the future. The model chose its parameters after seeing the answers. Add survivorship in the assets chosen, ignore transaction costs, let the researcher quietly discard the forty variants that didn't work, and you get the backtests that fill marketing decks — precise, impressive, and close to meaningless.
None of this requires dishonesty. It's the default outcome of doing the easy thing. Honesty in backtesting is a protocol, not a personality trait.
Walk-forward validation simulates the only thing that matters: what a model would have said at the time. Ours works the way a skeptical auditor would demand:
Numbers that look worse and mean more. Out of sample, our regime strategy earns less than buy-and-hold in most markets. It cuts the maximum drawdown in fifteen of twenty and volatility in all twenty. On Sharpe ratio, it beats buy-and-hold in just four markets of twenty — a statistic we display in large type on our own strategies page.
If a result would look better with a detail hidden, the detail is the result.
Why publish that? Because the alternative is a website whose model has never been wrong, and every professional reader knows exactly what that means. Publishing the losses does something subtle: it makes the wins legible. When the same audit that shows four Sharpe wins in twenty also shows drawdowns shrinking almost everywhere, a reader can finally see what the tool is for — it's a brake, not an engine — and decide with open eyes whether that trade suits them.
We built the validation page so those questions have public answers here. Hold us to it.
Read the full walk-forward audit →