The Metatrader 5 Series : When Huge Backtesting Differences Appear

If you look at my last few weeks of posts I have been very excited about the new qualities of the Metatrader 5 trading platform and the benefits it brings to system development regarding faster execution, faster optimization and added flexibility. Using my prototype implementation of Watukushay FE I analyzed trading over several different instruments and I finally came up with the "starting point" of intra-currency trading for this well-known freely available trading machine. This week I decided to do a small experiment and compare my results of Watukushay FE for the AUD/USD on Metatrader 5 with those found out with Metatrader 4 and the results found were extremely surprising. On today's post I want to talk about my findings, the possible causes of the issues found and what I will be doing to investigate the nature of this problem and what solutions can be implemented to deal with it.

During the first post comparing Metatrader 4 with 5 and the backtesting results of Watukushay FE we already saw a small yet noticeable difference between the testing results obtained on both trading platforms. I talked about the possibility of these errors being feed-related and the fact that the Metatrader 5 history feed might be more reliable since it has been "remastered and fixed" for this new platform. However the difference was small and therefore there was no substantial issue besides an anecdotical note pointing out this curious fact.

However when I decided to run the initial AUD/USD tests on Metatrader 4 to compare the results I obtained with Metatrader 5 the difference changed from "noticeable" to "abismal". The pictures below show you the results for MT5 and MT4 using the exact same settings on the AUD/USD currency pair backtest from 2000 to 2010. The overall equity curve is very different and the results do point out that something is substantially very changed between the historical feeds of MT4 and MT5. I first thought that the difference would be due to the presence of Sunday candles but this turned out to be false since the MT5 feed doesn't have any of them, so regarding this aspect it is the same as MT4. I then thought about the possibility that the whole difference is caused by important changes in data prior to 2006 (before metatrader 4 was launched) and the fact is that data differences are NOT limited to pre 2006 periods, the whole historical feed is different between both trading stations and meaningful differences are present. If you analyze the results you'll notice that almost all candles have different - if only very slightly - high/low/open/close values pointing out that RSI and ATR values will be very different. The change in one minute interpolation mechanisms is also not likely a factor here as Watukushay FE strictly controls bar opening on both its MQL4 and MQL5 implementations.
--
What causes such a dramatic change in profitability ? To get to the bottom of this problem I decided to strip down the logic to its simplest form and eliminate the closing logic of the EA, leaving only the entry rules. This shows us that there is still some difference between backtesting results (shown below). This means that differences in results are caused by differences in the RSI and ATR indicator calculations which are dependent on each backtest's particular historical feed. Stripping down the logic does reveal that most dependency is located before 2002 with results beyond this date being in better agreement. However there is still some dependency which is caused by differences in data between both historical sets beyond this period.
--
Since we simply cannot know for sure which of the two historical data selections is better - and they are probably both valid within normal broker differences (with the 2000-2002 data being very different probably due to differences between feeds in this period) - it becomes a wise decision to run backtests on both and trust the less profitable results to calculate profit and draw down targets. In some cases like the EUR/USD backtests this proves to be trivial but on others like the one I showed you today doing this mixed analysis proves to be extremely important. I will email the people at metaquotes to get some information about the different nature of the feeds and I will let you know once I have more information about their origin. However up until now all backtests of Watukushay FE seem to be more profitable on MT5 (meaning that our MT4 simulations are in fact the worst case scenario). Investigating other issues which may be related with the closing mechanism of orders in MT5 is also something I am currenlty doing since I have seen that the differences when the closing logic is enabled seem to have other strong causes besides simple feed dependency (more on this on a later post !).

If you would like to learn more about automated trading, the evaluation of expert advisors and the programming of your own strategies please consider buying my ebook on automated trading or joining Asirikuy to receive all ebook purchase benefits, weekly updates, check the live accounts I am running with several expert advisors and get in the road towards long term success in the forex market using automated trading systems. I hope you enjoyed the article !

The Metatrader 5 Series...Trying Metatrader 5 : Porting Watukushay FE

I have to confess that I was pretty excited last week when I learned about the release of the full metatrader 5 strategy tester. Being a fan of simulations to accurately model past trading I was dreaming about all the great enhancements that were now available and how all current Asirikuy systems would benefit from these implementations. This week I have been playing extensively with this new platform, mainly getting to know the MQL5 new programming language and all the "new ways" of doing things we must now relearn. I started by analyzing and understanding the sample experts released with the platform and later coded a few very simple systems to get to know the way in which things are done, indicators called, etc within this new trading program. On today's post I want to talk to you about my first "real exercise" which involved the porting of a preliminary version of Watukushay FE to metatrader 5. I will describe the process I went through, what I achieved and some of the things that surprised me when comparing results obtained with the Metatrader 4 backtester.

First of all, I have to say that coding experts for MQL4 has almost nothing to do with coding in MQL5. Almost everything is done differently and even though previous knowledge of MQL4 is useful, the truth is that several things like order placement, reading of indicator values, reading of bar values (high/low/open/close) and order selection are done differently. I found the approach implemented in MQL5 to be a little bit harder but yet a ton more flexible. This is particularly true of the new implementation of indicator and order handling which will now lend itself to an incredible coding flexibility. There are also some very useful functions included within the Trade.mqh file (downloaded with the platform) that will make EA programming a ton easier for those of you looking towards an "easier transition" from MQL4 to MQL5.

After familiarizing myself with the platform I decided to code the system that would be most easily available to all of you. Since Watukushay FE is the most widely used Asirikuy system -due to its free character- I decided to give the porting of Watukushay FE first priority. It is however important to say here that I did this porting merely as an exercise and it won't be the final release version of this EA for metatrader 5 which will probably take a few more months of polishing to develop (since we are currently developing a common MQL5 framework for Asirikuy systems taking advantage of the new classes implementation in this language). However the preliminary EA version I coded contains all the logic of Watukushay FE and follows the exact same logic as the MQL4 version in backtesting.

I have to say that I am amazed with the capabilities of Metatrader 5 regarding speed of execution. Simulations are not only fast but about an order of magnitude faster than in Metatrader 4. Watukushay FE runs a 10 year simulation in MQL4 in about 5 minutes while the Metatrader 5 implementation takes a little bit more than 30 seconds. Added to this absolutely great feature is the fact that multithreading is available for optimizations meaning that I can run a full 200 run optimization in less than half an hour when it previously took about half a day. Being able to run optimizations in parallel is a blessing and it will most likely open up the doors towards the finding of many profitable instruments and setups not currently used with Asirikuy systems. Below you can see an image of an optimization running on the 4 cores of one of my beloved quad processors.
--
You are probably also wondering how the Metatrader 5 backtesting results compare with the Metatrader 4 results and I would have to tell you that the results are a little bit puzzling to me. Watukushay FE is definitely long term profitable on both platforms but results are overall much better in Metatrader 5 when compared with the MQL4 version. The nature of this difference seems to be the difference between data sets. Since this version of Metatrader is new, we could assume that the data quality and interpolation mechanisms are better but such an assumption would have to be proved. Right now I am pleased to tell you that the EA - with the exact same settings - performs better on MQL5 and the Metatrader 5 data than on the Metatrader 4 version, meaning that -if anything- Watukushay FE is as good or better than what we think.
-
-
There is also a ton more of things to discuss regarding the optimizations of Watukushay FE and the new features and results available within the Strategy tester but for today I think that all the above information is enough :o). I have now started this series of posts called "The Metatrader 5 Series" which will deal with topics related to simulations, programming and overall features of this new Metatrader 5 platform which is bound to become the standard within the next few years in automated trading. During the next few weeks I will release several new posts on Metatrader 5 and probably Asirikuy members will also be able to enjoy a few videos on the matter :o)

If you would like to learn more about automated trading and how you can code your own likely long term profitable systems based on sound trading tactics please consider buying my ebook on automated trading or joining Asirikuy to receive all ebook purchase benefits, weekly updates, check the live accounts I am running with several expert advisors and get in the road towards long term success in the forex market using automated trading systems. I hope you enjoyed the article !