A few days ago, a customer sent me an email requesting the review of several expert advisors available at iticsoftware.com. These expert advisors which are named Stomper, Eldorado, Chinchilla, Upstream Portfolio, LimitsOffensive, neuralscalping, smart vision and multitrade are available for purchase on their website. These automated trading systems range from scalping to trend following and are supposed to give a very comprehensive and profitable portfolio for anyone wanting to trade the forex market using automated trading systems. The objective of these reviews is to evaluate the expert advisors in the light of the evidence provided by the authors.
The first thing I have to say is that I actually like the website designed by the iticsoftware.com team. There is almost no hype and no claims made about the experts' profitabilities. It is quiet refreshing to see a website that goes straight to demo and live testing instead of showing endless screenshots of useless backtesting results and trade examples. However, when I looked deeper into the results I saw several things I did not like.
If you notice the demo and live testing results of many of the expert advisors, like eldorado and forexer, you will notice that there are very prominent open draw down periods. That is, the experts were losing a very signficant amount of equity, sometimes even close to wipeout levels, on open trades. In fact, if these experts were put to trade just before these draw downs happened, they would have wiped the accounts. This points out to the lack of adequate closing mechanisms and actually a lack of market exposure limitation. In fact, I consider all experts showing this behavior to be extremely risky because of this.
Certainly they are experts that do not show this happening. The most clear and prominent example is the Stomper expert advisor which shows backtesting from 1999 and almost 2 year long live testing. This is certainly something I cannot ignore as I always look objectively at all the evidence presented and it certainly calls my attention that they have traded the expert for so long. Now, they are certain things that we must consider in order to look at these tests and evaluate them accurately.
The backtests sadly cannot be taken into account because of the very small amount of the takeprofit which makes one minute interpolation errors completely prominent. As a matter of fact, a comparison between live and backtesting shows almost no correlation since you see almost no loses in the strategy tester report while live testing does show a lot of losing trades. Since backtesting is not consistent, we are just left with the live tests for the evaluation of long term profitability.
The live tests are also quiet suspicious since there seems to be a void in the trading period from 10/2008 to 03/2009 which is also suspiciously aligned with the largest movements suffered during the beginning of the current economic crisis. I never like it when these periods appear in live testing. Why was trading interrupted ? was it because of the expert's logic ? or because it was stopped ? These are a few questions the people at iticsoftware would need to answer in order for me to understand this odd behavior on the live tests.
Now, the EA is also traded on one of the brokers with the most favorable EUR/GBP spread, which is Alpari UK. As a matter of fact, trading on any other broker with spreads higher than 3 may affect trading in a very unfavorable manner since it would reduce either the probability of winning trades happening. Since the nature of the expert advisor is scalping, I am almost certain that the EA will stop being profitable as soon as market condition shift towards a different behavior on the EUR/GBP (something which is happening recently), reason why I personally would need at least another year of live testing to even think about long term profitability (since backtests cannot be trusted for this EA).
Another interesting point is the fact that the draw down seen for the Stomper EA is also huge and suffering from that draw down at the start would have wiped the account. It seems that in order to achieve the currently highly unrealistic profit returns, the people at iticsoftware grossly overtraded this expert advisor. I estimated that with adequate risk, the EA would have produced about 30 to 100% during the past two years, something which is certainly not bad at all.
As I have said, the fact that there is a void in the live trading, the fact that the EA has a very small take profit (therefore very broker limited) and the fact that scalping makes the EA very vulnerable to changing market conditions (it also has no logic to adapt to changing markets) makes this EA not worth testing or buying. In my opinion, the creators should provide an explanation for the void in testing as well as another year of live testing and possibly testing on a different broker in order for me to reconsider this stomper review.
As far as the other experts go, the fact that all of them (except stomper) have huge open draw downs at times and also have backtests which are inconsistent with live and forward testing results makes me conclude that non of them are worth buying, since non of them seem to adequately limit market exposure. However, I do appreacite the effort of the people at iticsoftware.com to provide readily available live and demo testing results. In the future, lower risk settings which give more realistic perspectives should be used in testing (in my opinion). If you would like to have a deeper explanation of the reasons why I consider any of the indiviual experts not worth buying or testing please leave a comment and I'll elaborate.
Also, if you would like to learn more about automated trading systems, how to evaluate them and how to finally start trading profitably with conservative, long term profitable trading systems please consider buying my ebook on automated trading or subscribing to my weekly newsletter to receive updates and check the live and demo accounts I am running with several expert advisors. I hope you enjoyed the article !
The first thing I have to say is that I actually like the website designed by the iticsoftware.com team. There is almost no hype and no claims made about the experts' profitabilities. It is quiet refreshing to see a website that goes straight to demo and live testing instead of showing endless screenshots of useless backtesting results and trade examples. However, when I looked deeper into the results I saw several things I did not like.
If you notice the demo and live testing results of many of the expert advisors, like eldorado and forexer, you will notice that there are very prominent open draw down periods. That is, the experts were losing a very signficant amount of equity, sometimes even close to wipeout levels, on open trades. In fact, if these experts were put to trade just before these draw downs happened, they would have wiped the accounts. This points out to the lack of adequate closing mechanisms and actually a lack of market exposure limitation. In fact, I consider all experts showing this behavior to be extremely risky because of this.
Certainly they are experts that do not show this happening. The most clear and prominent example is the Stomper expert advisor which shows backtesting from 1999 and almost 2 year long live testing. This is certainly something I cannot ignore as I always look objectively at all the evidence presented and it certainly calls my attention that they have traded the expert for so long. Now, they are certain things that we must consider in order to look at these tests and evaluate them accurately.
The backtests sadly cannot be taken into account because of the very small amount of the takeprofit which makes one minute interpolation errors completely prominent. As a matter of fact, a comparison between live and backtesting shows almost no correlation since you see almost no loses in the strategy tester report while live testing does show a lot of losing trades. Since backtesting is not consistent, we are just left with the live tests for the evaluation of long term profitability.
The live tests are also quiet suspicious since there seems to be a void in the trading period from 10/2008 to 03/2009 which is also suspiciously aligned with the largest movements suffered during the beginning of the current economic crisis. I never like it when these periods appear in live testing. Why was trading interrupted ? was it because of the expert's logic ? or because it was stopped ? These are a few questions the people at iticsoftware would need to answer in order for me to understand this odd behavior on the live tests.
Now, the EA is also traded on one of the brokers with the most favorable EUR/GBP spread, which is Alpari UK. As a matter of fact, trading on any other broker with spreads higher than 3 may affect trading in a very unfavorable manner since it would reduce either the probability of winning trades happening. Since the nature of the expert advisor is scalping, I am almost certain that the EA will stop being profitable as soon as market condition shift towards a different behavior on the EUR/GBP (something which is happening recently), reason why I personally would need at least another year of live testing to even think about long term profitability (since backtests cannot be trusted for this EA).
Another interesting point is the fact that the draw down seen for the Stomper EA is also huge and suffering from that draw down at the start would have wiped the account. It seems that in order to achieve the currently highly unrealistic profit returns, the people at iticsoftware grossly overtraded this expert advisor. I estimated that with adequate risk, the EA would have produced about 30 to 100% during the past two years, something which is certainly not bad at all.
As I have said, the fact that there is a void in the live trading, the fact that the EA has a very small take profit (therefore very broker limited) and the fact that scalping makes the EA very vulnerable to changing market conditions (it also has no logic to adapt to changing markets) makes this EA not worth testing or buying. In my opinion, the creators should provide an explanation for the void in testing as well as another year of live testing and possibly testing on a different broker in order for me to reconsider this stomper review.
As far as the other experts go, the fact that all of them (except stomper) have huge open draw downs at times and also have backtests which are inconsistent with live and forward testing results makes me conclude that non of them are worth buying, since non of them seem to adequately limit market exposure. However, I do appreacite the effort of the people at iticsoftware.com to provide readily available live and demo testing results. In the future, lower risk settings which give more realistic perspectives should be used in testing (in my opinion). If you would like to have a deeper explanation of the reasons why I consider any of the indiviual experts not worth buying or testing please leave a comment and I'll elaborate.
Also, if you would like to learn more about automated trading systems, how to evaluate them and how to finally start trading profitably with conservative, long term profitable trading systems please consider buying my ebook on automated trading or subscribing to my weekly newsletter to receive updates and check the live and demo accounts I am running with several expert advisors. I hope you enjoyed the article !
Posts
3 comments:
Interesting post, Daniel. I'm quite amazed how much effort you've put into reviewing 'everything forex'.
The take away from much of your work for me is that scalpers and grid traders, which are common in the forex EA world, aren't really worth much in live trading for the reasons you elaborate...broker spreads (and costs), 'soft' backtrading history, elimination of profitability for scalpers that do work (get too popular), and secular changes or shifts in currency relationships for grid traders.
With all that in mind, I would like to take you up on your offer in this posting and ask your opinion of the V-Trader at iticsoftware. That's the one Boris gave me additional, updated backtesting on, which basically showed an extension of the prior backtesting results.
Many thanks.
Hello Redford,
Thank you very much again for your interest and comments. I do give my best to place as much insight into forex products as I can :-).
Regarding the V-trader expert advisor this is what I have to say.
First of all, the expert trades the EUR/CAD and AUD/CAD pairs, it is very important to take into account spread variations within these pairs, specially in times on low volatility. I would therefore say that backtesting is not a measure of the system's profitability. A reason why you get so much profits in back testing. I have never seen a backtest like the one shown giving the same results as a live test. Another problem is encountered when we realize that the EA uses a tick volume indicator which varies significantly between brokers (because of the lack of an fx central exchange to represent "true" volume) and may make your results increasingly more variable depending on your broker. More over, backtests are VERY limited, with only an evaluation of the past year. Backtesting should be done for at least the past 9 years which are the years for which enough data is available. It is naive to believe the backtests are a depiction of reality when there are no live tests to compare them to the backtest itself and the experts trading does tilt the balance towards the other side. This EA, as the others, is NOT worth buying or testing. It lacks back/live testing consistency proof (therefore the backtests CANNOT be trusted), its profitability may be affected significantly by spread widening and it is likely to give different results between different brokers due to the fact that it uses tick volume data.
I am glad I could fulfill your request. I hope you continue to access my website frequently !
Best Regards,
Daniel
Thanks Daniel, as always you make too much sense, dashing the dreams of those of us who want to make easy money in our sleep. So you've identified another category of EA's that are not likely to do well - those trading pairs with wide spreads, particularly those with relative small T/P. What's left are those EA's trading less frequently than scalpers, with much larger T/P and smaller S/L than scalpers, trading currencies with spreads that aren't too wide.
Here's another you may want to investigate sometime, if you haven't already:
http://www.forexsprinter.com/
The ongoing demo results are here:
http://forexsprinter.mt4live.com/
I've found some serious problems with it already, for example a series of trades done yesterday that couldn't possibly have occurred because the range for the pair (EUR/GBP) at the times of the trades (around 13:06-13:08 on 20091208) never came close to the result (5 to 7 pips). More like 0.5 to 0.7 pips. Other than a couple of sets of these bogus trades, the other trades look reasonable. Imagine an EA starting out with $2000, trading one lot, making 25,000 + within six or seven trading days!
Post a Comment