What Backtesting Proves, and the Data It Needs
    Backtesting Mastery
    52 of 57

    Module 17 · Lesson 1 of 3

    What Backtesting Proves, and the Data It Needs

    16 min readUpdated

    Understand the purpose, limitations, and correct interpretation of backtesting results before running a single test.

    Goal

    Develop an accurate mental model of what a backtest proves and what it cannot prove, so you neither over-trust nor dismiss backtesting results.

    Mistake This Prevents

    Running a backtest, seeing 80% win rate, declaring the strategy proven, and going live with full capital, before realising the test was run on only 12 trades over 3 months.

    What Backtesting Proves (and Doesn't)DOES Prove✓Historical expectationStrategy performed this way on past data✓Strategy logic worked on past dataRules were consistently applied and tested✓Approximate edge exists historicallyPositive expectancy over sufficient sampleMin 30 trades; 100+ for high confidenceDOES NOT Prove✗Future profitabilityMarkets change, past edge can disappear✗Will work in next market phaseTest across bull AND bear periods both✗Overcomes human executionSlippage, emotion, errors not in backtestCannot guarantee future performanceBacktesting proves historical expectation only, forward testing bridges the gap to live performance
    Backtesting proves historical expectation, it cannot guarantee future performance. A strategy with strong backtest results still requires forward testing to validate execution realities. Test across both bull and bear market periods for meaningful results.

    Core Concepts

    What Backtesting Proves

    1. Whether the strategy had a positive expectation in the historical data period tested
    2. The approximate drawdown and win rate to expect if conditions repeat
    3. Whether the rules are specific enough to be applied consistently

    What Backtesting Does NOT Prove

    1. That the strategy will work going forward (markets change)
    2. That your live execution will match the backtest (slippage, emotions, timing)
    3. That a strategy is "found", because every strategy needs forward validation too
    4. That 80% win rate in 2021 means 80% win rate in 2024 (market regime dependence)

    The Three Backtesting Errors

    1. Look-ahead bias: Using future data (e.g., daily close) to generate a signal that would only be known after the candle closes. This gives artificially good results
    2. Overfitting: Testing 50 parameter combinations and selecting the one with the best results. The "best" parameters are tailored to historical data and will not perform the same going forward
    3. Insufficient sample size: A 10-trade backtest is statistically meaningless. Minimum 30 trades for any conclusion; 100+ for high confidence.

    The Correct Interpretation Framework A backtest is not a guarantee. It is evidence that the strategy's rules, applied to historical data, produced a certain result. This evidence must be:

    • Verified across multiple time periods (not just one bull market)
    • Verified across multiple assets (not just BTC in 2020)
    • Followed by forward testing (paper trading) before live deployment

    Key Metrics to Report

    • Total trades
    • Win rate (%)
    • Average R:R on winners
    • Profit factor (gross profit / gross loss)
    • Maximum consecutive losses
    • Maximum drawdown (%)
    • Sharpe ratio (if implementing)
    3 Critical Backtesting ErrorsERROR 1Look-Ahead BiasUsing future data to makepast decisions. Results arecompletely invalid.Move candle-by-candle onlyERROR 2Overfitting15+ paramsToo many parametersoptimised to past noise.Fails on new data.Test parameter stability ±20%ERROR 3Tiny Sample Size<30tradesFewer than 30 trades isstatistically meaningless.Need 100+ for confidence.Min 30 trades; 100+ preferredAny one of these three errors invalidates your backtest results entirely
    Three critical errors destroy backtest validity: look-ahead bias (using future data), overfitting (curve-fitting parameters to past noise), and insufficient sample size (fewer than 30 trades). Avoid all three before drawing any conclusions from backtest results.

    Checklist

    • Minimum 30 trades required before drawing conclusions
    • Test across at least 2 different market conditions (bull and bear)
    • Report all seven key metrics, not just win rate

    Common Mistakes

    • Running backtest only on a bull market period ("2020–2021 only")
    • Selecting the best parameter set from 50 combinations without out-of-sample testing
    • Reporting only win rate without drawdown or profit factor

    Fedha Tool Tie-in

    FedhaAcademy's Backtesting Engine requires a minimum of 30 trades before displaying results and automatically tests across at least 2 years of data. It reports all seven key metrics on a standardised results card.

    Key takeaways

    What to remember

    1. Backtesting proves historical expectation; it cannot guarantee future performance.
    2. Three critical errors: look-ahead bias, overfitting, and insufficient sample size.
    3. Minimum 30 trades required; 100+ for high confidence. Test across bull and bear market periods.
    4. Report all seven metrics, because win rate alone is insufficient for strategy evaluation.
    Lesson quiz

    Quiz

    Question 1 of 6

    Pass: 5 of 6 correct

    Question 1 of 6

    What does a backtest with 80% win rate on 10 trades prove?

    Keep your place

    Save this lesson and continue tomorrow.

    Create an account to mark lessons complete, build a daily practice streak, and use paper trading with $10,000 in virtual capital.

    Create free account I already have an account
    Free forever No card required Continue from any device

    Up Next

    Running It: Manual, Coded and Walk-Forward

    Continue