From Setup to System: Rules, Backtesting and Avoiding Overfitting

Recognising a pattern is not the same as having a method. A method turns observations into rules that can be written down, repeated and tested. Once rules exist, you can study how they would have behaved on past data. This lesson covers how to write clear rules, how to backtest them honestly, and how to avoid overfitting, the most common way traders fool themselves with historical data.

Writing rules that can be tested

A testable rule leaves no room for interpretation. "Buy when the trend looks strong" cannot be tested. "Buy when the daily close is above the 50-period moving average and the RSI(14) crosses back above 40" can be. A complete system answers at least these questions:

1. Market and timeframe: for example, EUR/USD on the 4-hour chart.
2. Entry condition: exactly what must happen, and on which candle's close.
3. Stop: where the trade is wrong, defined by a rule rather than a feeling.
4. Exit: a target, a trailing stop, or a time limit.
5. Position size: how much is risked per trade (covered in the next lesson).
6. Filters: conditions under which you do not trade, such as major scheduled news.

Backtesting honestly

Backtesting means applying the rules to historical data and recording every trade. It can be done by hand, scrolling bar by bar through a chart, or with software. Manual testing is slow but teaches you how the setup looks in real time. Whichever way you choose, a few principles keep results meaningful:

• Include costs. Spreads, commissions, and in crypto, funding fees on perpetual futures can turn a slightly positive system negative.
• No look-ahead. Only use information that was available at the moment of the decision. A signal based on a candle's close cannot be entered before that close.
• Enough trades. A handful of trades says almost nothing. Aim for a sample large enough to include different market conditions.
• Different regimes. Test across trending, ranging, calm and volatile periods. BTC, gold and EUR/USD have each gone through very different phases.

Record the number of trades, win rate, average win and average loss, and the largest peak-to-trough drop in equity (maximum drawdown). A low win rate is acceptable if average wins are clearly larger than average losses, and the reverse is also true.

Avoiding overfitting

Overfitting happens when you keep adjusting parameters until the rules fit the past perfectly. The result looks impressive on historical data and then fails on new data, because it learned the noise rather than a real behaviour. Warning signs include many conditions, very specific parameter values, and results that collapse when a parameter changes slightly.

Practical defences:

• Keep rules few and simple.
• Split the data: develop on one period and confirm on a later period you have not looked at (out-of-sample).
• Check that nearby parameter values give similar results.
• Forward-test with very small size, or on a demo account, before committing more.

A backtest is a description of the past under your assumptions. It is evidence, not a promise.

Key takeaways

• A system is a set of written, unambiguous rules for entry, stop, exit, size and filters.
• Honest backtests include costs, avoid look-ahead and cover different market conditions.
• Look at drawdown and the size of wins versus losses, not only the win rate.
• Fewer rules and out-of-sample checks are the best defence against overfitting.
• Forward-test with small size before trusting any historical result.