Topic: Differences in results of strategies developed with FSB and FSB Pro

When I will develop a strategy (in FSB Pro or FSB) I will test it with the program I didn’t developed it with, i.e. if I developed a program with FSB Pro I will tested with FSB, and vice versa). Of course, I am testing with same historical period  and all other parameters equal in both cases. What is interesting, I will often (not always) get very different results under both tests.

As the FSB have been under development under a longer period then FSB Pro, I probably should trust the results of FSB more. But the discrepancy, if it will appear, tells something and should be followed up. There is of course one thing which differs in both those cases – indicators. As I understand, indicators for FSB Pro has to be rewritten in some cases. But the question is still there – which version shows the correct results (but, of course, both results can be also erroneous).

Following up the different results can be helpful to find bugs in both programs. This strategy gives such results for the period from 2013-01-01 to (approximately) few days ago – other periods will give different results.

Another explanation is a possibility of “butterfly effect” – small changes in any parametric start condition will give large changes in end result. But this sensitivity is also an quality indicator – a strategy will only survive “out there” if it is robust, and not too sensitive to initial conditions.

Here follows an example of such strategy. This strategy gives different results for the period from 2013-01-01 to (approximately) few days ago – other periods will not behave in same way. I also used FSB Pro data downloaded from FSB Pro History Center. The strategy gives positive results under FSB (which I used to develop it with) and bad results in FSB Pro. I check controlling parameters in both programs (FSB and FSB Pro) throughout to be sure that they are exactly same.

Strategy name: EurUsd_1d_PivotPoints-SteadyBands
Forex Strategy Builder v3.8.0.0
Exported on: 2014-07-07 22:39:58

Description:
Automatically generated on 2014-05-29 16:22.
Out of sample testing, percent of OOS bars: 30%
Balance: 1934,98 EUR (2014-01-08 00:00  Bar: 215)

Market: EURUSD 1 Day
Spread in points: 20,00
Swap Long in Points: 2,00
Swap Short in Points: -2,00
Commission per lot at opening and closing in Points: 0,00
Slippage in points: 0

Use account % for margin round to whole lots
Maximum open lots: 20,00
Entry lots: 3,00% of the account for margin
Martingale money management multiplier: 2,00

Intrabar scanning: Accomplished
Interpolation method: Pessimistic scenario
Ambiguous bars: 0
Tested bars: 350
Balance: 39230 points (3193,98 USD)
Minimum account: -1426 points (191,21 USD)
Maximum drawdown: 1426 points (108,79 USD)
Time in position: 97 %

[Strategy Properties]
     A same direction signal - Does nothing
     An opposite direction signal - Does nothing
     Permanent Stop Loss - None
     Permanent Take Profit - None
     Break Even - None

[Opening Point of the Position]
     Pivot Points
     Enter the market at the Pivot Point
     Base price  -  One day
     Vertical shift  -  -512
     Use previous bar value  -  Yes

[Closing Point of the Position]
     Steady Bands
     Exit long at Upper Band
     Smoothing method  -  Simple
     Base price  -  Open
     MA period  -  30
     Margin in points  -  1539
     Use previous bar value  -  No

Re: Differences in results of strategies developed with FSB and FSB Pro

What is interesting, I will often (not always) get very different results under both tests.

Such discrepancy is unacceptable.  Both programs should give 100% equal results (except of the cases you are using some of the unique FSB Pro features or HTF indicators in FSB).

As I understand, indicators for FSB Pro has to be rewritten in some cases.

This is not true. Both programs use absolutely equal indicators.

Another explanation is a possibility of “butterfly effect” – small changes in any parametric start condition will give large changes in end result.

This is a good shot. A difference in the initial parameters is the most probable reason for the discrepancy.

...


Can you attach the strategy file to test it.

Re: Differences in results of strategies developed with FSB and FSB Pro

Please use a fixed lot for testing, no increase and no Martingale. These both can skew results.

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....

Re: Differences in results of strategies developed with FSB and FSB Pro

I'm receiving completely equal results in both programs.


http://s7.postimg.org/s2dxuqdfb/screenshot_457.jpg

Re: Differences in results of strategies developed with FSB and FSB Pro

And here the same strategy with reduced entry units to 1% in order to test the whole data series.

Also completely equal results:

http://s7.postimg.org/5rq2urg53/screenshot_458.jpg

Re: Differences in results of strategies developed with FSB and FSB Pro

Thank you for you prompt response and clarification of some issues. Your answer makes me understand better inner working of FSB. I will go back to my charts and numbers and, once again, recheck why there is difference between your results and my.

Re: Differences in results of strategies developed with FSB and FSB Pro

I will go back to my charts and numbers and, once again, recheck why there is difference between your results and my.

What I did was to point FSB Data Directory to my FSB Pro data source. This guarantee that we are using same data.

http://s7.postimg.org/6iit0jiif/screenshot_459.jpg

http://s7.postimg.org/cxhtx7p87/screenshot_460.jpg

I also checked Data Horizon for both programs. FSB Pro can load up to 100 000 intrabar data, but the Free FSB up to 50 000. Set 50 000 intrabar limit in the FSB Pro to eliminate a possible difference.

Also I compared the account settings and the symbol settings.

Finally I saved the strategy with FSB Pro and opened it with FSB (You can use Copy / Paste also).

Re: Differences in results of strategies developed with FSB and FSB Pro

I also used same data for both FSB and FSB Pro (I didn’t point to FSB Pro data, but copied values from FSB Pro Data directory to FSB Data directory). Also intrabar data was checked and is the same in both cases – 50 000. But here comes my additional observations:

When working with a strategy in FSB, I can chose to change Data Horizon values for loaded strategy, by, for example, moving the start date one day forward (in this case I used 2013-01-01 and moved it to 2013-01-02). This will immediately change both the Balance/Equity chart (in this case by shorting it with one day) but also will show the new value for “Date of beginning” in “Market Statistics” list (and the Intrabar Scanner opens a small window and reloads data values – strategy is recalculated).

It is no so in my working with FSB Pro. If I will load same strategy into FSB Pro and try to change the Time of beginning value in “MARKET/Data Horizon/Use start date limit” (by, for example, once again moving start date one day forward), the program will allow me to change the value in the “Data Horizon/Use start date limit” window, but after returning to the strategy I can read the old value shown in “Input Parameters” list – the “Time of beginning” date is not changed (and the Intrabar Scanner is not reloading data values as it did in FSB – strategy is NOT recalculated). I have yet to find how to change this behavior to similar in FSB.  Forcing recalculation of the strategy with F5 key will not help. 

Is it a correct behavior of FSB Pro? The behavior of FSB seems for me be more logical. Or I am missing something here.

Re: Differences in results of strategies developed with FSB and FSB Pro

FSB Pro loads the data files in a different way.

You are right that Data Horizon tool doesn't reload the data files automatically.

You have to use the Reload Corresponding Files command. It will force FSB Pro to reload the files for the selected Data Source except the files used currently in the Generator.

http://s7.postimg.org/s7hp4ekqf/screenshot_462.jpg