• Registered: 2012-08-21
  • Posts: 3

Topic: Different results with exact same system & data

I'm new to the strategy builder, but I've encountered something odd. I built a strategy using the data provided by FSB and saved it. I then built the exact same strategy with all the same parameters using the same currency pair and time frame and again using the exact same data provided by FSB. In other words, everything was exactly the same. However, the back test results were off by a good 20%.

I understand differences in results when using data from different providers, and understand getting different results when using a strategy live. But how can the results be different if I back test identical systems on the exact same data?

  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 8,024

Re: Different results with exact same system & data

Probably there is a difference in some strategy parameter. Trading sizes for example.
It's possible also some double value parameter to differ with a small value not shown on the data fields.

Make a zip with both strategies and we'll find the difference.

Popov's Website

  • Registered: 2012-08-21
  • Posts: 3

Re: Different results with exact same system & data

Hi, thanks, yes, I did find discrepencies, my mistake. They match now. But I have an additional question:

When I build a strategy for a currency pair in a particular time frame, can I just change the currency pair and time frame to see how the strategy would work for the other pair or time frame? Or do I need to build the strategy from the ground up for each particular pair or time frame to see how it performs?

  • From: Germany
  • Registered: 2012-02-17
  • Posts: 122

Re: Different results with exact same system & data

Hi bfleming,
if a strategy is good she works one more timeframes and currency pairs. You don´t need to build a new strategy for every timeframe and curency pairs.

For example:
Step1:
You build a strategy for EURUSD H1.

Step2:
Check if strategy is profitabel on M30 and H4

Step3:
Check if strategy is profitable on GBPUSD and USDCHF....

Step4:
Make a WalkForwardTest for this strategy

But most of the strategies I have seen don´t fullfill all 4 steps.
thomas

tnickel's Website

  • Registered: 2012-08-21
  • Posts: 3

Re: Different results with exact same system & data

Thanks for explaining. So a robust strategy should be profitable across time frames/pairs? Would it be considered less robust if the strategy uses the same indicators and entry/exit signals but uses different parameters?

And I'm going to sound dumb here, but how do I perform a Walk Forward test?

Thanks again!

  • From: Australia
  • Registered: 2011-07-01
  • Posts: 431

Re: Different results with exact same system & data

tnickel wrote:

Hi bfleming,
if a strategy is good she works one more timeframes and currency pairs. You don´t need to build a new strategy for every timeframe and curency pairs.

For example:
Step1:
You build a strategy for EURUSD H1.

Step2:
Check if strategy is profitabel on M30 and H4

Step3:
Check if strategy is profitable on GBPUSD and USDCHF....

Step4:
Make a WalkForwardTest for this strategy

But most of the strategies I have seen don´t fullfill all 4 steps.
thomas


If you go to a different timeframe you will find that even if the indicators work the parameters such as stop loss, offsets etc. will need to be changed because generally the higher the timeframe the bigger the operating range. I can guarantee that if you had a strategy that works on a 15 min timeframe and then you just run it on a 1 day timeframe  the mismatch in parameters will be so great that it will never work. An example of this the other way is Mr Popovs daily fractal strategy which for the life of me I cannot get to work on lower time frames.

Conventional wisdom also says that if it works on one currency pair it should work on others. My experience tells me this is also incorrect. All currency pairs have their own life. Whilst there may be similarities between say the GBPUSD & CHFUSD I have yet to find a strategy that works on both without any changes.

  • From: Australia
  • Registered: 2011-07-01
  • Posts: 431

Re: Different results with exact same system & data

bfleming212 wrote:

Thanks for explaining. So a robust strategy should be profitable across time frames/pairs? Would it be considered less robust if the strategy uses the same indicators and entry/exit signals but uses different parameters?

And I'm going to sound dumb here, but how do I perform a Walk Forward test?

Thanks again!


You can do a walkforward test in two ways.

1. Change the date range for the data to block out say the last 20% of the data when building the model. Then revert back to the full data set after your strategy is built to see what happens with the equity curve.

or

2.  Set up another directory with the walk forward data and test that after you have built your strategy.

  • From: Germany
  • Registered: 2012-02-17
  • Posts: 122

Re: Different results with exact same system & data

Hi SpiderMan,
thanks for the information.

One thinks about this.
If I use a different timeframe I changed the spread a little. The orginal spread was for example 3 and the spread in the new timeframe 1.

I forgot something.

If I develop from brocker1   I backtest the found strategie witch data from different other brockers.

For example:
a) I found strategie with data from fsb
b) Make backtest with data from Dukascopy
c) Make backtest with data from forextester

you can see the data on the brockers differs a little.

It is very difficult to find a strategie that fullfill all citerias.
thomas

tnickel's Website

  • From: Australia
  • Registered: 2011-07-01
  • Posts: 431

Re: Different results with exact same system & data

Data from all brokers is different but similar.

The main thing I have spent a lot of time working on is the different offsets from GMT by the different brokers. This is important if you are using Day Close since this is different for different brokers and can vary by 3 hours between a broker that is based on GMT and a broker that is GMT+3.  The other issue this gives rise to is Sunday bars if the broker is on GMT as the market opens at about 10.00pm GMT on Sunday. This throws all your indicators out, especially if they are using low values in the parameters (i.e. 3 day period RSI).

I would personally use data exclusively from the same broker when building a strategy and use them for trading as well.

  • From: Germany
  • Registered: 2012-02-17
  • Posts: 122

Re: Different results with exact same system & data

Hi Spiderman,
yes the GMT is a problem.
If the strategie has the rule "open trade at 8am" as example the strategy doesn´t work with different GMT.

We neet an auto calculate GMT-feature in the FST.
It will be good that the FST calculate the GMT offset automaticaly and show us the offset.
If the data of the different brockers are similar it must be possible to calculate the GMT?

To get the data exclusively from the same brocker it not so easy to get this data.
My broker is ActiveTrades. ActiveTrades don´t have tickdata for users.

If you ask 1M data for the different currencys you get 1M data of the last 3 years. But this data is not the actual data. three months later is normal.

If you get the data of metatrader, this data is from metaquoute.

thomas

tnickel's Website