One of the things I get most criticized about is my evaluation of trading systems which have small TP and SL targets. People often do not understand the reasons why I do not recommend products that seem to be generating "a lot" of money for traders using them. I have been sent from time to time statements from several EA vendors and from third party users showing the excellent results of some commercial and freely available scalpers and they all look surprised when my replies all say the same thing : "your results are not statistically significant", "your results are not reliable", etc. Today I want to explain a little bit more the problem with scalper profitability and why there is currently no scalper available which has statistically significant and reliable evidence of profitability. For the purpose of this article I will define a scalper as any system that has an average profit target of less than 10 times the spread or which closes positions - in average - in less than 5 minutes.
What is the deal with these scalpers and their profitability ? The problems are several and may not be evident to people who are new or inexperienced in the field of automated trading. As a matter of fact, even people who have been trading automated systems for years may not see them clearly if they have never applied a strong mathematical analysis to what they are doing. However these problems are real and they affect all scalpers - as defined above - equally.
What I see as the two most important problems faced by scalpers are the unreliability of their simulations and their unfavorable risk to reward ratio. In metatrader 4, scalpers cannot have reliable simulations mainly because of one minute interpolation errors. These errors which are generated because of the lack of real tick data in metatrader 4 cause both signals derived from price action and position profit targets to be faulted by a significant percentage. This problem wouldn't be particularly bad if the SL value was of the same magnitude as the TP - as both would be affected equally - but the fact that the SL is usually 4-10 times the size of the TP makes these errors absolutely important since a small overestimation in the number of profitable trades equals a great underestimation of draw down.
For example, if you have a trading system that has a 90% winning ratio with a 5:1 risk to reward ratio, a simple overestimation of profitability of 5% in backtesting would equal a 25% increase in the actual draw down suffered by the system. Through the analysis I have done of several systems with small profit targets it is notable that this overestimation is even more important, usually around 15-20% or more depending on the actual closing mechanisms and shortness of the TP.
The most important problem why profit and draw targets cannot be estimated accurately for scalpers - which arises from the unreliability of simulations - is the fact that live execution problems usually greatly diminish profitability in the long term. Variables such as slippage and spread widening - which are not expressed in backtesting - play a decisive role when dealing with automated trading scalping systems.
It is important to note that I am not making these problems up, they are real and ignoring them will have terrible consequences in the long term. In the beginning these problems may not seem apparent as these systems have very high winning ratios and the incoherence with simulations and the appearence of bad losing periods will usually appear later on, after a large number of trades has already been taken.
Does this mean that there are no scalpers that work ? No, that is not what I am saying. What I am saying here is that currently simulations are not valid - because of metatrader limitations - and extensive live evaluation of a statistically significant period - which in the forex market cannot be less than 5 years - is necessary to prove that the systems have a high like hood of being long term profitable. This criteria is derived from extensive statistical analysis of market cycles, a criteria meant to evaluate the performance of a system under very varied conditions. Otherwise, you may just be looking at temporary profit or draw down cycles that may not let you see the global character of the system. For regular systems - with reliable simulations - six months of live/back testing consistency are enough to validate simulations and grant these performance results but for scalpers this simply cannot be done because simulations have all the above mentioned problems - which again - are very real and well documented.
So in the end - yes - there can be a lot of scalping systems showing positive results in live testing results or simulations. However, the fact remains that none of these systems have tests long enough to be considered statistically significant. These systems simply cannot be considered long term profitable because they haven't shown adaptation to varied market conditions and the ability to survive in the long term. So even though they may have positive results, I will never recommend - nor use - any system which does not have enough evidence to consider that I will be able to trade it safely in the long term. Small, statistically insignificant trading results are simply not enough to consider any system likely long term profitable (which means that the system has a high probability to work for the next few decades).
If you would like to learn more about automated trading systems and how you too can code your own reliable systems based on sound trading tactics please consider buying my ebook on automated trading or joining Asirikuy to receive all ebook purchase benefits, weekly updates, check the live accounts I am running with several expert advisors and get in the road towards long term success in the forex market using automated trading systems. I hope you enjoyed the article !
2 comments:
Hi Daniel-
You had to know I was going to comment on this post ;-)
I think its safe to say that live trading is the only indisputable way to prove the profitability of scalping robots.
On the drawdown issue, if the scalper uses a 1 to 4 TP to SL ratio, and performance in the live account shows 93% winners, does that address your concerns that back tests underestimate the drawdown of actual trading?
In terms of broker re-quotes and spread widening I think that falls under the category of broker dependency. Granted some systems are more broker dependent than others. And systems that are trying to take a 10-pip profit may be specifically designed not to trade when there's a 5-pip spread because you need a 20-pip move to cover the spread plus your take-profit.
I'm not 100% convinced than an ECN or pay-per-trade broker might not address the spread and re-quote issues. I haven't traded one live yet, but i'm sure they are out there with some other caveats like minimal position size for example.
Finally, is there any amount of time (or # of trades) of live trading evidence in which you would reconsider your conclusions?
Thanks,
Chris
Hi Chris,
Thank you for your message :o) You were right, I was expecting your comment !
Actually you are right about broker dependency and about spread filters. However you need to take into account that these filters are not active in backtesting so the actual extent to which they affect trading is unknown. Obviously they tend to diminish statistical significance in the long term but their effect on profitability is unknown until live testing is shown.
You should take into account that trade number is not a very important issue here as what matters is the ability to adapt to varying market conditions, something which requires at least 5 years of trading. This is not something I have made up out of the blue but it comes from the analysis of volatility changes within the market. If you want to know if a system can truly handle change, then 5 years are necessary, less time is actually not very statistically significant.(systems that can be simulated accurately may get this validation through a live/back testing consistency analysis)
Regarding ECNs and other ways to diminish dependency, they do come at higher costs per trades, something which would also hinder the profitability of scalpers in the long term.
Of course, if a system has an unfavorable risk to reward ratio but reaches a 93% profit, then it is a profitable system but the importance here is the STATISTICAL significance of those results. If you can show just a few months or a year the results are simply not significant enough since it doesn't account towards a significant amount of volatility change in long term market conditions.
In the end, these scalpers are dangerous because they are very difficult to evaluate and they may give very profitable results with differences that may be apparent - although ignored - by people in the beginning.
For example - and this is something I think you have noticed - if a system shows significant differences between demo and live results that diminish profitability by 20-30%, then this may not seem important in the short term but in the long term it will mean that draw downs were quite underestimated. On these systems usually the bad periods take time to show, but when they come, you will see if there is any underestimation and how important it actually is.
Again, I am not saying that long profitable scalpers are not possible and I will be happy if within 5 years you will be able to show that there actually are some out there.
Yet again, this concept is not arbitrary or made out with no criteria, it is not my opinion but merely the time it takes to achieve a statistically significant sampling of long term volatility changes to address performance under varied market conditions.
I hope this answers your questions ! thank you very much for your comment Chris :o)
Best Regards,
Daniel
Post a Comment