1 (edited by sergiel 2016-06-13 01:25:50)

Topic: Different results btw FSB backtest and MT4 backtest?

Hi there, I seem to be repeatedly facing this problem of having extremely different back test result in FSB versus the MT4 back test using the same data and same time period.  The MT4 back test having much worse results.

I always ensure that my Darwinex data saved to FSB data is up to date.  I am not using Metaquotes but rather saving from each Darwinex time frame directly to FSB data / Darwinex folder.

I have attached the image for the FSB strategy settings and back test result.  Net balance $1,855.17 starting with an account of $1,000 over the period of April 4, 2014 to June 11, 2016.

When I run the EA back test on my MT4 Darwinex account for the same period the resulting net balance is $1,057.

I have re-saved the new EA based on this strategy uploaded it to my Darwinex account and ran a back test in Darwinex for a 1 week period only from February 1, 2016 to February 8, 2016.

I have posted the following charts:

1) FSB Strategy screen shot;
2) Feb 1 - Feb 7 2016 FSB Journal;
3) Feb 1 - Feb 7 2016 MT4 back test results;
4) Feb 1 - Feb 4 2016 FSB chart;
5) Feb 1 - Feb 4 2016 MT4 back test chart;
6) Feb 2 2016 FSB chart magnified;
7) Feb 2 2016 MT4 back test chart magnified.

To me the data appears to be identical.  I have triple checked the EA settings in MT4 to ensure they are identical to the setting of the strategy back test in FSB.

I cannot understand why the results are drastically different?  I need to get to the bottom of this because I feel I am always second guessing the FSB back test results and wasting a great deal of time.

Thanks in advance as any assistance would be greatly appreciated.

http://s33.postimg.org/agdyo3erf/2016_06_12_FSB_Strategy_Image.jpg



http://s33.postimg.org/m5s26aid7/2016_06_12_FSB_Journal.jpg


http://s33.postimg.org/tq8451ugr/Feb_1_Feb7_MT4_Backtest.jpg

http://s33.postimg.org/8hh41rssb/Feb_1_Feb_4_GBPUSD15_M.jpg

http://s33.postimg.org/5dpa7bmgr/Feb_1_Feb_4_MT4_Back_Test_Chart.jpg

http://s33.postimg.org/w0axns5ff/Feb_2_2016_FSB_Chart.jpg

http://s33.postimg.org/u7d4k4323/Feb_2_MT4_Chart.jpg

Re: Different results btw FSB backtest and MT4 backtest?

What we do know is..... there is something not set up quite right as time after time, Popov sets up and shows that he can get the results the same.

So............ please get your strategy to Popov so he can examine and then do a test. without your strategy he is unable to help you much.

As you have probably seen in several other instances of differences. there is probably some simple thing causing the difficulty, however, without the strategy, Popov is unable to diagnose and fix.

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....

Re: Different results btw FSB backtest and MT4 backtest?

Thanks for your response.  Bet "get your strategy to Popov" I assume you mean send him the FSB .xml file?

Would I just private message that to him?

Thanks in advance.

Serge

Re: Different results btw FSB backtest and MT4 backtest?

sure. private message, or email or attach to post......

Keep in mind that he is really busy. so reply may be slow..

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....

Re: Different results btw FSB backtest and MT4 backtest?

Ok thanks so much smile

Serge

Re: Different results btw FSB backtest and MT4 backtest?

Popov doesn't seem to have an email address listed in his contact page.  The PM function does'nt appear to allow me to attach a file?

Thanks
Serge

Re: Different results btw FSB backtest and MT4 backtest?

Just upload your file via google drive and send him the link, once he has downloaded the file, you can delete it.

I’m working on improving my English and trading skill

Re: Different results btw FSB backtest and MT4 backtest?

It took me 5 minutes to check your strategy. Probably it took you longer to write your post.
It would be much better to do your homework instead throwing it to us.

How I checked the strategy:

1. Save data from MT and load the same data in FSB.
2. Run the EA on MT
3. Run the strategy on FSB
4. Compare the charts' indicators

I found perfect match

http://s33.postimg.org/6rr18rmgr/indictaor_values_match.jpg

5. See where in the FSB chart has a period with deals starting from a Flat positions
6. Set the starting period in MT and run the backtester
7. Stop the backtester after 5-6 deals and compare the results

I found perfect match

http://s33.postimg.org/v2skr39mj/backtest_perfect_match.jpg


Please do that at home.

Please note that we will not consider reports without:

1. a screenshot that shows the data files are same
2. a screenshot that shows compared indicators values
3. a screenshot that compares individual deals
4. attached strategy


This is the last time I do that work for you for free

Re: Different results btw FSB backtest and MT4 backtest?

Thank you for your reply and the time taken to respond.   My original post was not necessarily directed to you as I understand you are very busy, as am I.  However I have to say I am am somewhat annoyed by the tone of your response.

I had taken the time to do my "homework" as you put it, and I did provide everything you listed in my original post aside from the comparative indicator screen shots.  I didn't just "throw up" some information wanting to be spoon fed an answer as you have suggested in your response.

Like I said the data appears to be identical between my MT4 test and FSB as it should be, but the trades that my mt4 strategy tester is taking are completely different for some unknown reason.

That is great that you were able to have the results match on your mt4 platform for a 1 week period but I was not and that was the reason for my post in the first place.  Your response does not help me at all try to get to the bottom of why I am getting vastly different results.

I understand and you are very busy and I do appreciate your time. However the other license users and I have paid a fair price for this software license.  This was a great deal of money for some of us and a bit of patience on your end as we are trouble shooting and learning how to use the software properly goes a long way.

If this issue I keep having with vastly different results is an issue with my Darwinex MT4 platform somehow then that is fine but I am trying to determine that.

Again I do appreciate your time and I realize that you are very busy but on the other hand I did put a great deal of time into trying to determine the problem before turning to the forum for some assistance. 

Is anyone else having the same issue of vastly different FSB and MT4 back test results with the same data or am I the only one?

Serge

Re: Different results btw FSB backtest and MT4 backtest?

Like I said the data appears to be identical between my MT4 test and FSB as it should be, but the trades that my mt4 strategy tester is taking are completely different for some unknown reason.

Well.

Why not start from here. Take one trade that differs and try to find why does it differ.

One single trade will give you the response and the solution.

Find such trade, post zoomed in charts from MT and FSB and we will see what it happens.

Re: Different results btw FSB backtest and MT4 backtest?

Many people have errors and after some exploration we locate a small error in their setup.

I also get errors from time to time and I have to find the cause myself.

When we go to Popov to find our errors...... it takes away his time from developing the software.

What all of us have to understand is ....  Popov is a programmer, he works on a fee basis, he has an hourly rate. so when he tales his time to find our mistakes we are very fortunate that he does not charge for his time.

He also has a considerable business to run, several projects.

The software is fine, as it is, the help files are pretty well current, we can get help by fully documenting our situation  and posting the documentation and the forum members will attempt to contribute their assistance.

All that being said, as a trader, you have to do your homework and examine things thoroughly before making a submission.

Keep in mind that many of the users are fully consumed with their own situations, dealing with their own difficulties... and. also trading....

Personally, I try to find my own solutions, I spend a lot of time looking through the posts on the forum to glean whatever information I can find - because - I do not want to be dependent on someone else for my income.

If you spend the time looking at how others have solved their difficulties by scanning the threads in the forum you will save yourself a lot of time as you build your knowledge.

Traders can not be dependent. or they will not make any profits......

This software is complex, the indicators are complex........ we each have a lot of study to do, experimenting, examining.... There is no way we can rely on another to make the good things happen if we are going to succeed.

I use one broker, and only one........ because - I learned years ago that I make mistakes when switching the data back and forth. I use their MT5 and their MT4. Sure, I have been tempted to try other brokers but I recall the mess I had when I did that in the past.

Keep in mind that your errors may result from the latency between you and the different brokers..... Have you checked that....... is your internet functioning the same for each?

My broker has a huge lag time....... but I generally use the higher time frames so I am not all that concerned.

Unless you have a very fast internet and low latency, errors can be caused in your execution, and that is something that you will have to examine for yourself.

As far as tone of response, please ignore supposed 'tone', not all people have perfect English, nor do they have time to look up exactly how to say something, again, we are most fortunate that Popov reads all of the posts and offers his time and help.

Always keep in mind, that this project was started to provide for traders a free method of developing strategies, that it eventually evolved to the point where there had to be a fee charged due to the time required....... which is 7 days a week, most weeks.

And now we have a top notch program with a very moderate price, there is a generous affiliate program and now there is the Binary Option Tester which offers more huge opportunities.

Personally, I am really thankful for all of these.

I am also thankful for the contributions made by Footon and Ahmed for all of the indicators, and a few other people who have been contributing to the project for many years and continue to contribute on a regular basis.

Trading is a self taught business, we are so very fortunate to have this tool, I would not be concerned about tone whatsoever.

daveM

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....

Re: Different results btw FSB backtest and MT4 backtest?

Thank you for your post as well.  I am very grateful for this amazing tool and the time that Popov and others have contributed to helping myself and others in this community.

I understand that trading is very much an individual endeavor as I have been at this for about 10 years now.

I guess in my defense I had spent a great deal of time double, triple checking everything in my MT4 back test settings and to ensure I had uploaded the Darwinex data to FSB properly.  Your eventually going to hit a point where you have checked everything you can think of. 

If the data is identical and the MT4 EA settings are set perfect there isn't much more I can check to determine what the issue may be.

As far as multiple brokers that isn't an issue because I was only testing on 1, Darwinex.

As far as latency, internet connection, etc. I don't see how that is relevant.  Remember I am not testing live yet, I am simply back testing in my MT4 account with the same data that was uploaded to FSB.

I will go back and closely examine several days to see if I can spot what may be going on.

Thanks
Serge

Re: Different results btw FSB backtest and MT4 backtest?

Serge, I need to ask you - do you plan to trade or only backtest? It doesn't take much to confirm via demo if your strat trades the same as in FSB, additional backtest with MT is a waste in every respect and doesn't add anything to the table. I'm happy to be proven wrong, though.

Re: Different results btw FSB backtest and MT4 backtest?

I do not use MT4 except as a bridge to the market. I guess that is why I do not run in to these problems.

I try to get my strategies to look ok in FSBPro and then export them and start to use them live.

I spend most of my time trying to improve results and forward testing in FSBPro.

When Popov was developing this software, I trust that he did all the MT4 work to make sure that FSBPro did the job. And looking through the old posts you can see some of the testing that he did, I am sure he has done a lot more.

If, in fact, there are discrepancies.... I am not too bothered.... I do not want to spend the time looking for MT4 stuff. I have simple strategies on higher time frames...... and I have a lot of them.

Now i have so many strategies trading that I would not like to have to check.

One thing I do keep track of is the trading history, to ensure that the ratio of wins to losses is satisfactory.

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....

Re: Different results btw FSB backtest and MT4 backtest?

I might suggest that you work on that strategy to get rid of ambiguous bars......  That may well be the cause of your situation.

I really encourage you to have zero ambiguous bars if you are going to be trading real time. MT4 has a problem with data anyways, might as well have as good a strategy as you can get.

Also. did you get this strategy to run properly without the compound orders..... and then do the additions and reductions.

Those compound orders may be affecting the execution, you might try to get the thing running with out those and then when that is ok, add the position adjustments.

Use the simplest and test it first, the start to add the bells and whistles.... that way is easier to locate errors.

Also........ turn on intrabar scanning to help reduce errors.

None of these is specifically the cause of error....... but all together may be a cause of concern.

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....

16 (edited by jgp_altus 2016-06-14 18:51:06)

Re: Different results btw FSB backtest and MT4 backtest?

Thanks. will try to comply.  Most of my EAs already have 0 Ambiguous bars and Auto Scan is always "switched on" as is trade until margin call.  What are compound orders?
My understanding is FXCM MT4 [which I'm having trouble with] is a basically unmodified MetaQuotes product, but that Oanda [which works much much better] has a quasi-Metaquotes set up.

Re: Different results btw FSB backtest and MT4 backtest?

Sergiel,

Did you find the problem? If not, we can try fixing the problem on your machine. If you agree I will be happy to record a video how I usually debug such issues. We can do it via team viewer and skype. It will probably take 30-40 minutes.

I want to save the session because it will demonstrate working on a real world problem and will help to many users. (If they are interested in watching live records with tries and errors).

18 (edited by sergiel 2016-06-15 00:32:02)

Re: Different results btw FSB backtest and MT4 backtest?

footon wrote:

Serge, I need to ask you - do you plan to trade or only backtest? It doesn't take much to confirm via demo if your strat trades the same as in FSB, additional backtest with MT is a waste in every respect and doesn't add anything to the table. I'm happy to be proven wrong, though.

Hi footon, to answer your question I do trade live and I currently have 4 different EAs running on my live account.  However, I do take a fairly cautious approach before running an EA on a live account.

At minimum I like to see the following criteria met before I will even consider running an EA on a live account with real money at stake:

1) A profitable strategy on FSB.

2) Passes the Montecarlo test for robustness.

3) Change the data feed in FSB to another broker data (FXDD) and still have profitable results.  (If vastly different results here a good indicator the strategy is rubbish and is just curve fit.  However comparison will not work if strategy is broker time dependent.)

4) Run the same back test in MT4 to double check the results.

If all of these tests pass then I will consider running the EA on a live account.  To simply run every EA that looks profitable in FSB without some more confirmation is account suicide in my opinion.

Sometimes I will also forward test on a demo account for several weeks as well.

Thanks
Serge
)

19 (edited by sergiel 2016-06-15 00:28:46)

Re: Different results btw FSB backtest and MT4 backtest?

Popov wrote:

Sergiel,

Did you find the problem? If not, we can try fixing the problem on your machine. If you agree I will be happy to record a video how I usually debug such issues. We can do it via team viewer and skype. It will probably take 30-40 minutes.

I want to save the session because it will demonstrate working on a real world problem and will help to many users. (If they are interested in watching live records with tries and errors).

Hi Popov thank you so much for the offer of assistance, that would be fantastic!  In all honesty I still need to have a closer look at what is going on and post my results.  I should be able to do this on the next day or so.

Our 1 year old daughter keeping me pretty busy these days lol smile

Re: Different results btw FSB backtest and MT4 backtest?

Blaiserboy wrote:

I might suggest that you work on that strategy to get rid of ambiguous bars......  That may well be the cause of your situation.

I really encourage you to have zero ambiguous bars if you are going to be trading real time. MT4 has a problem with data anyways, might as well have as good a strategy as you can get.

Also. did you get this strategy to run properly without the compound orders..... and then do the additions and reductions.

Those compound orders may be affecting the execution, you might try to get the thing running with out those and then when that is ok, add the position adjustments.

Use the simplest and test it first, the start to add the bells and whistles.... that way is easier to locate errors.

Also........ turn on intrabar scanning to help reduce errors.

None of these is specifically the cause of error....... but all together may be a cause of concern.

Thanks Dave I will try this out as well.

Serge

Re: Different results btw FSB backtest and MT4 backtest?

I feel that trying to switch the ea from one broker to another may cause you grief........ because they do not have a uniform feed, they all operate differently.

You can forward test by changing the dates in the 'Market' tab... ie you can have large periods or smaller periods to make a rigorous test in addition to Monte Carlo.

At one time, Popov was considering adding forward testing to FSBPro, probably that is well down on the 'to do' list.

I wonder, when you are forward testing in demo mode...., do you keep track of the variances..... I know that would be a lot of work to maintain a record keeping system, however, it would give you, over time, an expectation for results.

That being said, I suspect broker demo accounts as there are so many ways that they can cheat you when you switch your strategy to real account.

The broker I use has small lot sizes ie 10,000 instead of 100,000, so I can test using a small lot size on a real account, and as long as I am at least a bit prudent, I am losing only 50 cents or a dollar per trade.

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....

Re: Different results btw FSB backtest and MT4 backtest?

Popov wrote:

Sergiel,

Did you find the problem? If not, we can try fixing the problem on your machine. If you agree I will be happy to record a video how I usually debug such issues. We can do it via team viewer and skype. It will probably take 30-40 minutes.

I want to save the session because it will demonstrate working on a real world problem and will help to many users. (If they are interested in watching live records with tries and errors).

Perhaps this video, if you do make it.... will have an introductory part that details all of the steps to get everything turned on so that data is scanned properly, instruments are set up properly.. all of this to cut down on the number of errors that seem to be cropping up on a regular basis.

Many of the errors seem to stem from improper initial setup of the instruments and the data... and forgetting to use the tools that are already set up in the software.

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....

Re: Different results btw FSB backtest and MT4 backtest?

Sorry for the delay.  Ok I randomly picked a spot where the positions were flat to begin the comparative analysis between the FSB Indicator chart and the MT4 back test chart.  This system is using the GBPUSD 15M chart.

On March 24, 2016 six short entries were taken.  Three reductions occured on March 25th, and the other three reductions occurred on March 30, 2016.

After comparing the MT4 chart and FSB charts along with the journal for each, the entries and exits appear to be almost identical in terms of price and time.

I tried posting my charts but there appears to be a problem with the java script now.  Big ads for Porter Airlines show up and it wont let me post my charts as before.

Anyhow this particular spot in the trading cycle appears to give me identical results between FSB  and MT4, so still not sure why the results over a two year back test differ so drastically.

Thanks
Serge

Re: Different results btw FSB backtest and MT4 backtest?

I have no answer re differences in a 2 year backtest, puzzled....... unless somehow data was a bit different.......

Maybe the Porter ad was so you would take the ferry to the island and hop a flight to Hawaii.

OR  maybe Premier Wynne wants you for her Cabinet to help her blow our money and wants you to fly all over the province.....  LOL

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....

Re: Different results btw FSB backtest and MT4 backtest?

Blaiserboy wrote:

I have no answer re differences in a 2 year backtest, puzzled....... unless somehow data was a bit different.......

Maybe the Porter ad was so you would take the ferry to the island and hop a flight to Hawaii.

OR  maybe Premier Wynne wants you for her Cabinet to help her blow our money and wants you to fly all over the province.....  LOL


Lol those are both possibilities.  I do love Porter though, the only airline left to give you a complimentary beer smile

I will try and post my charts again to see if I have any luck.

Serge