Imagine that you have dedicated several weeks to the development of a profitable trading system. Your simulations are showing you brilliant results, so you decide to test your system on a live account for a few months. Then you discover that performance is nowhere near what you expected. What is wrong ? Well, the most probable explanation is that you have curve-fitted your system to past data and therefore your system is unable to behave similarly on a different data set. On the following paragraphs I will talk to you about what curve-fitting is, how it affects system development and how we can develop systems to avoid this seemingly unavoidable curse.
So what is curve-fitting anyway ? Simply explained, the term is derived from the fact that any given "curve" or data set can be accounted for by a given mathemtical function of arbitrary complexity. That is, you can always find a mathematical function which can predict with absolute accuracy all the items of a data set. However, the function may have absolutely no predictive power. For example, take into account the data set 1,2,3,4. Can you predict the next number ? You can immediately think of a function which can predict all the items of the data set but can it predict the next one ? Well, what if the next one was 12 ? then your function has no predictive capability even though it was able to predict all previous items successfully.
The problem applies to trading because the fact that a given system was able to exploit a market inefficiency in the past does not guarantee that the inefficiency will be present in the future. Usually optimization is great to curve-fit trading systems because what an optimization does is merely to "adjust function parameters" to find a mathematically sound answer to the problem. The better and tighter the optimization, the more curve-fitted the system will become, this is a reason why neural networks - which are excellent at optimization- tend to fail in successful trading systems as they always curve-fit their data excessively.
So how can we make a trading systems with the slightest possible chance of ending up with a useless curve-fitted expert advisor ? The first important measure is to do simulations on periods which are as long as possible. The longer the trading period, the more statistically significant the data set is and the less likely it is to allow the curve fitting of your system. Long time periods introduce a wide variety of market conditions which make curve fitting very difficult. However the number of trades is also very important. Having more trades for a given long trading period is better (against curve-fitting) since it implies a larger number of market conditions in which the EA was able to trade and succeed.
Another important aspect of curve-fitting is to avoid excessively correlated and exhaustive optimization of trading systems. Trading systems should be optimized one variable at a time or using two cross-related variables at a maximum without going into excessive detail. For example if an EA has a parameter which can go from 20 to 50 it is better to optimize it in 2 steps (20,22,24, etc) rather than running a full optimization of all values (20,21,22, etc).
Another great idea is to have very robust profitability, this means that results in an optimization should form "thick clusters" of profitability. For example, if in the above example 24 gives very profitable results but 23 and 25 are unprofitable then it means that in the future the system is bound to be unprofitable if there are slight changes in the optimum criteria (something which will happen). When running an optimization as many results as possible should be profitable and the area around the most profitable result should also be close to that result.
Then finally we also have the matter of adaptability. A system should be able to adapt to changes in market conditions therefore having some flexibility in its entry and exit criteria. If you optimize a system which gives very good results with a TP of 100 pips the system is likely going to be unprofitable in the future as market volatility and conditions fluctuate, invalidating the 100 pip TP. The best thing would be to optimize a dynamic criteria, for example, what percentage of the standard deviation is the ideal TP ? This ensures that the EA can change its exit and entry values in a dynamic way therefore adding another layer of protection against curve fitting.
However, despite all our efforts it is still a mathematical possibility to curve fit a trading system to the past given the fact that the future may have absolutely no relationship with it (like in 1,2,3,4,12). Nonetheless, following the above criteria in the development of a trading system aiming for adaptability, broad optimizations, robust profitability and large periods of testing data guarantees a better chance of suceeding even if there are significant changes in future market behavior. If you would like to learn more about these concepts and how I use them for the development of likely long term profitable systems please consider buying my ebook on automated trading or joining Asirikuy to receive all ebook purchase benefits, weekly updates, check the live accounts I am running with several expert advisors and get in the road towards long term success in the forex market using automated trading systems. I hope you enjoyed the article !
So what is curve-fitting anyway ? Simply explained, the term is derived from the fact that any given "curve" or data set can be accounted for by a given mathemtical function of arbitrary complexity. That is, you can always find a mathematical function which can predict with absolute accuracy all the items of a data set. However, the function may have absolutely no predictive power. For example, take into account the data set 1,2,3,4. Can you predict the next number ? You can immediately think of a function which can predict all the items of the data set but can it predict the next one ? Well, what if the next one was 12 ? then your function has no predictive capability even though it was able to predict all previous items successfully.
The problem applies to trading because the fact that a given system was able to exploit a market inefficiency in the past does not guarantee that the inefficiency will be present in the future. Usually optimization is great to curve-fit trading systems because what an optimization does is merely to "adjust function parameters" to find a mathematically sound answer to the problem. The better and tighter the optimization, the more curve-fitted the system will become, this is a reason why neural networks - which are excellent at optimization- tend to fail in successful trading systems as they always curve-fit their data excessively.
So how can we make a trading systems with the slightest possible chance of ending up with a useless curve-fitted expert advisor ? The first important measure is to do simulations on periods which are as long as possible. The longer the trading period, the more statistically significant the data set is and the less likely it is to allow the curve fitting of your system. Long time periods introduce a wide variety of market conditions which make curve fitting very difficult. However the number of trades is also very important. Having more trades for a given long trading period is better (against curve-fitting) since it implies a larger number of market conditions in which the EA was able to trade and succeed.
Another important aspect of curve-fitting is to avoid excessively correlated and exhaustive optimization of trading systems. Trading systems should be optimized one variable at a time or using two cross-related variables at a maximum without going into excessive detail. For example if an EA has a parameter which can go from 20 to 50 it is better to optimize it in 2 steps (20,22,24, etc) rather than running a full optimization of all values (20,21,22, etc).
Another great idea is to have very robust profitability, this means that results in an optimization should form "thick clusters" of profitability. For example, if in the above example 24 gives very profitable results but 23 and 25 are unprofitable then it means that in the future the system is bound to be unprofitable if there are slight changes in the optimum criteria (something which will happen). When running an optimization as many results as possible should be profitable and the area around the most profitable result should also be close to that result.
Then finally we also have the matter of adaptability. A system should be able to adapt to changes in market conditions therefore having some flexibility in its entry and exit criteria. If you optimize a system which gives very good results with a TP of 100 pips the system is likely going to be unprofitable in the future as market volatility and conditions fluctuate, invalidating the 100 pip TP. The best thing would be to optimize a dynamic criteria, for example, what percentage of the standard deviation is the ideal TP ? This ensures that the EA can change its exit and entry values in a dynamic way therefore adding another layer of protection against curve fitting.
However, despite all our efforts it is still a mathematical possibility to curve fit a trading system to the past given the fact that the future may have absolutely no relationship with it (like in 1,2,3,4,12). Nonetheless, following the above criteria in the development of a trading system aiming for adaptability, broad optimizations, robust profitability and large periods of testing data guarantees a better chance of suceeding even if there are significant changes in future market behavior. If you would like to learn more about these concepts and how I use them for the development of likely long term profitable systems please consider buying my ebook on automated trading or joining Asirikuy to receive all ebook purchase benefits, weekly updates, check the live accounts I am running with several expert advisors and get in the road towards long term success in the forex market using automated trading systems. I hope you enjoyed the article !
Posts
No comments:
Post a Comment