The best way to take advantage of OOS is to begin with a strategy that *fails* OOS.  In this situation you at least know the IS and OOS sections are different.  You then treat the entire data set as IS and attempt to optimize your strategy until you get a balance chart that is nearly linear throughout the IS and OOS sections.  If you can do that, then you know that strategy is more robust -- since it has been trained against two chunks of different data.

OOS does no harm -- except it does cause people to think they have a strategy that can see into the future.

it is not a guarantee of good strategies but they work better than others

Thanks for attaching a picture -- it helps a lot.

When I look at the picture then I would not necessarily interpret this as a good strategy.  For example, in the "First OOS" section there are only 3 winning trades over the course of 2-3 months (20% OOS  1.1.2016.- 1.5.2017).  That means winning trades are a very *rare event*.  And that is confirmed in the "Second OOS" -- a winning trade is a very rare event.  If you were to place this in a live account with new incoming data then what is the probability you'll have a winning trade within the next month, or 6 months or next year?  It's hard to say.

I don't know the optimal number of trades when performing back testing, but I've read elsewhere that it should be at least 100 - 200 trades per year.

it is not a guarantee of good strategies but they work better than others

Hello, everyone!
I do not know English well, but google is powerful
I use double OOS

for example:
1)search strategy 1.1.2016.- 1.5.2017 -20% OOS
2)finding a strategy with a good  first OOS
3)remove data horizon 1.1.2016-present
4)second OOS is good
5)I continue on monte carlo, etc.

it's simple but it works for me

Interest Idea, have you in live experience with this ea´s then?

How long live?

for example:
1)search strategy 1.1.2016.- 1.5.2017 -20% OOS
2)finding a strategy with a good  first OOS
3)remove data horizon 1.1.2016-present
4)second OOS is good
5)I continue on monte carlo, etc.

it's simple but it works for me

Maybe there is some indicators pointing out when it is time or there is edge to make these on/offs..

Yes -- an indicator or strategy that is not used for trading, but only to provide a clue as to the trend or pattern of the incoming data.  And this could then be used to turn on / off different EA portfolios -- or something like that.

This would be most interesting...

Robust means working consistently on different pairs and timeframes. It is subjective in my opinion.

rantampla -- thanks for the good word.  It is appreciated.

rantampla wrote:

After few months test live EAStudio and FSB, the FSB Strategies are little better then EAStudio...

I also spent a lot of time with EA Studio and recently came to the same conclusion.  It did create some nice strategies that I continue to use, but I've now returned to FSB.  The main thing I miss is the ability to create an EA that contains a portfolio of strategies.  That is a very powerful feature. 

I'm currently working on a tool that will allow one to merge multiple FSB strategies into a single EA -- like EA Studio.  Popov's code is very clean and organized, so I think it is possible.  I hope he doesn't mind...

No Problem i feel it, and then i must say thanks, it is much big helping for me

I have read, that Mr. Popov from 01.10.17 EA-Studio make Pause and he will work on FSB, and i hope he can make Portfolio like EA-Studio.

But when you earlier, i hope you give it free, your work is very good

I have no experince with coding

I also spent a lot of time with EA Studio and recently came to the same conclusion.  It did create some nice strategies that I continue to use, but I've now returned to FSB.  The main thing I miss is the ability to create an EA that contains a portfolio of strategies.  That is a very powerful feature. 

I'm currently working on a tool that will allow one to merge multiple FSB strategies into a single EA -- like EA Studio.  Popov's code is very clean and organized, so I think it is possible.  I hope he doesn't mind...

Consider OOS.  Many people feel that a strategy that performs well for both IS and OOS is more "robust" or less curve-fitted and will, therefore, perform better in a live account on future data.  But isn't there another explanation?  If the OOS portion is similar to the IS portion (i.e. the IS portion did a good job training the strategy for the OOS portion), then it is not surprising the strategy does well in both.  That is, the OOS portion really didn't exercise the strategy in a way that would reveal whether or not it truly is more robust.

In a recent post you mentioned something along the lines we have all the data, we just need to figure-out what the right question is to ask -- which is great insight.  So, in this particular experiment, what exactly was the question?  The experiment yielded interesting results that can help going forward.  I'm curious to hear what people think the right questions are.

For example -- here are a few questions:
 
Does it make a difference which data set is used when generating, optimizing and back testing a strategy?

Should you always use the latest (and greatest data set) when back testing, or is there a particular chunk of historical data that routinely does a good job of training strategies even though it might be "old"?

What is the definition of robust?  And if it is a good thing, then does a metric currently exist that measures "robustness"?  Or could a new metric be computed?

My Strategie is only using MT4-Tracker from Sleytus, a big big big tool, i am very gladly with that tool.

Thanks again Sleytus