As many of you may already know, the Metatrader 4 trading platform has a strategy tester whose objective is to evaluate a strategy based on historical data, rather than executing an expert advisor for an extended period of time. But the question for many remains, is it reliable ?
This question is of the highest importance as many decisions could be based on this feature of the program. Someone who believes in the backtester being accurate will have no problem in using it as a filter for his expert advisors and also using it as a measure of an unknown strategy.
First of all, we have to be very clear. There is only one way to test the accuracy of the strategy tester in Metratrader 4. First, we need to run an expert advisor in a demo account for a given period of time, second, we must backtest this exact same expert in that exact same period of time. The comparison between trades taken in both settings will let us know if the backtester is or is NOT accurate.
Many people say backtesting is not accurate because the expert was profitable for two months of backtesting, then wiped out the accound in real life. This is actually NOT an indication of a flawed tester (the expert could have behaved this way naturally because of changing market conditions). To be clearer, an expert being profitable in backtesting and not in forward testing is a necessary but not sufficient condition to declare the backtester unreliable. To be sure, you must perform the test suggested on the third paragraph.
To do this, you must first set the backtester so that you can get the best possible results. For this I have set my metatrader platform with alpari historical one minute data dating back to 2004. You also want to use the every tick mode and use every less time frame in your analysis. I have done small tests for different expert advisors and I have arrived to the following conclusions.
- Expert advisors that open and close trades in a 1-4 minutes timeframe CANNOT be accurately or even approximately tested with the Metatrader 4 backtester, this, because the tester uses fractal interpolation. It guesses what happens under the one minute candlesticks.
- Expert advisor behavior is not predicted in any case when a sudden move occurs in a one minute candlestick. When an expert trades a 100 pip move which occured in under a minute (like after some NFP releases). In many cases the results in reality and with the tester are fairly different.
- Experts that trade 1-3 times a day and have wide (50 or more) stops and take profit levels are usually predicted far more accurately than experts with tighter levels.
- Experts that trade on the closing price of bars sometimes exhibit abnormal profits because of "divination" of the strategy tester. It uses future knowledge by mistake.
- To deem an expert advisor profitable by backtesting, at least a 3 year analysis are necessary. This supplies a wide range of market conditions, which would at least justify a demo testing of the expert advisor.
So is backtesting useless ? This is actually NOT the case. Would I buy an expert advisor based on backtesting ? Absolutely NOT ! The reliability of the results is very dependable upon the nature of the expert advisor and only access to the source code and understanding of the trading method can guarantee a critical grasp of the tester's results.
So personally I will continue to use metatrader's backesting as long as I know that the expert does comply with the requirements necessary for reliable testing. I will, however, never buy any commercial experts based solely on backtesting as it is - quiet frankly - easily fooled by programming manipulation.
This question is of the highest importance as many decisions could be based on this feature of the program. Someone who believes in the backtester being accurate will have no problem in using it as a filter for his expert advisors and also using it as a measure of an unknown strategy.
First of all, we have to be very clear. There is only one way to test the accuracy of the strategy tester in Metratrader 4. First, we need to run an expert advisor in a demo account for a given period of time, second, we must backtest this exact same expert in that exact same period of time. The comparison between trades taken in both settings will let us know if the backtester is or is NOT accurate.
Many people say backtesting is not accurate because the expert was profitable for two months of backtesting, then wiped out the accound in real life. This is actually NOT an indication of a flawed tester (the expert could have behaved this way naturally because of changing market conditions). To be clearer, an expert being profitable in backtesting and not in forward testing is a necessary but not sufficient condition to declare the backtester unreliable. To be sure, you must perform the test suggested on the third paragraph.
To do this, you must first set the backtester so that you can get the best possible results. For this I have set my metatrader platform with alpari historical one minute data dating back to 2004. You also want to use the every tick mode and use every less time frame in your analysis. I have done small tests for different expert advisors and I have arrived to the following conclusions.
- Expert advisors that open and close trades in a 1-4 minutes timeframe CANNOT be accurately or even approximately tested with the Metatrader 4 backtester, this, because the tester uses fractal interpolation. It guesses what happens under the one minute candlesticks.
- Expert advisor behavior is not predicted in any case when a sudden move occurs in a one minute candlestick. When an expert trades a 100 pip move which occured in under a minute (like after some NFP releases). In many cases the results in reality and with the tester are fairly different.
- Experts that trade 1-3 times a day and have wide (50 or more) stops and take profit levels are usually predicted far more accurately than experts with tighter levels.
- Experts that trade on the closing price of bars sometimes exhibit abnormal profits because of "divination" of the strategy tester. It uses future knowledge by mistake.
- To deem an expert advisor profitable by backtesting, at least a 3 year analysis are necessary. This supplies a wide range of market conditions, which would at least justify a demo testing of the expert advisor.
So is backtesting useless ? This is actually NOT the case. Would I buy an expert advisor based on backtesting ? Absolutely NOT ! The reliability of the results is very dependable upon the nature of the expert advisor and only access to the source code and understanding of the trading method can guarantee a critical grasp of the tester's results.
So personally I will continue to use metatrader's backesting as long as I know that the expert does comply with the requirements necessary for reliable testing. I will, however, never buy any commercial experts based solely on backtesting as it is - quiet frankly - easily fooled by programming manipulation.
Posts
5 comments:
Could you explain how to set up a portfolio of EAs in the Strategy Tester for backtesting with historical data?
Also, how to set the same portfolio of EAs in a live account?
Could you also make available the free EAs that you used in your recent tests of portfolios of free EAs?
Thank you so much!
Unfortunately Metatrader only allows to test one expert at a time, so right now it is not possible to test them at the same time. Testing them individually is not the same thing, as some of the experts have money management and are thus affected by the profits or losses taken by other experts.
To set them in a live account you would have to open several charts and attach each ea to the chart of your choosing. Also remember to set each expert's magic number differently.
I believe the experts have been made available in my ftp at ftp://75.127.87.121. Let me know if there is an expert missing.
Is there a way to write a "wrapper EA" that embeds several EAs, allowing the user to give values to the parameters of each of the EAs and also passing the info from MT4 to each EA, as well as accepting info from each EA and passing it to MT4?
If so, that would be a way to package a portfolio of EAs as one EA that can then be backtested, thus backtesting the portfolio.
Yes, I am agree with you. Backtest is not reliable enough to test an EA. But demo test is not enough too, you know why? because tick data in demo that is sent to client is different from data in live account. and one more, there is no server problem i.e. requote,trade context is busy, etc in demo account.
So, the best way to test an EA is using live account with small quantity of money.
Keep writing, my friend!
Btw, if there any chance to exchange link with mine?
http://myforexstrategy.blogspot.com
thanks
free of charge ex4 decompiling
Hi there folks ... if anybody needs a decompiling of ex4 file just drop me an email at ex4source@gmail.com . It will take 1-2 days until you get the source code.
Post a Comment