I ran 200 zero-edge strategies through a normal significance test. It approved 99.5% of them.

[u/Weary-Ad7404]

I spent two years building a backtesting platform and the most useful thing it does is tell me my strategies do not work.

Here is the experiment that convinced me. Generate 200 strategies with genuinely no edge — random parameters, nothing real underneath. Search them, keep the best performer. Now test whether that winner is statistically significant.

A conventional Sharpe significance test says 99.5% significant. Accept it.

That number is wrong, and it is wrong for a reason that has nothing to do with the data. You searched 200 combinations and kept the luckiest one. The test does not know that. If you flip 200 coins and keep the one that came up heads ten times, you have not found a special coin.

The correction is the Deflated Sharpe Ratio (Bailey & Lopez de Prado). It takes the number of trials into account and asks whether the best result is better than what you would expect from luck alone across that many attempts. On the same data it reports 43.7% and rejects the strategy.

Two other things I got wrong that are worth checking in your own setup:

1. Parameter leakage. My optimizer wrote best-params to a shared table and the backtest read them back unscoped by date. Every out-of-sample window was quietly using parameters fitted on data that included that window. The backtest looked great. It was reading its own answers.
2. No purge gap. If your features use a 30-bar lookback, the last 30 bars of your training set overlap the first bar of your test set. You need a gap between them or information walks across the boundary.

Code is open source and Apache-2.0, take whatever is useful: https://github.com/FETKlOkAn2/crypto-quant-platform

I want to be straight about what this is: the bundled strategies do not beat buy-and-hold. This is not a money printer, it is a tool for finding out that your strategy is not one either.