I guess we need to make sure if that thing has positive effect. Will make couple experiments "by hand" with this idea few variations on dozens strategies. Will let you know if results will be better than if chosen best variables results.

By the way, I am planning to upgrade my system to AMD Ryzen 7, 8 cores, 16 threads. I hope there is no any limit with FSB about using max cores/threads?

Hi,
Just finished experiment. Maybe someone will find results useful
Planned to do it with at least 20 strategies, but stopped with 5, because it is very time consuming and I can see that strategies results getting worse, so for me it is enough proof not to use it. Used different strategies, pairs and time frames.

Checked 4 variants for selecting parameters. Variants:
1. optimized (FSB full optimization box checked, getting best one from about 200-600 optimization tests)
2. 4th from the top 20 random generated variable sets (SQN sorting, 80% percentile)
3. 10th from the top 20 random generated variable sets (SQN sorting, 50% percentile middle)
4. average indicator values from all 20 random variable sets

Question was if I choose not the best backtest parameters will it trade better than optimized results on unseen data chunk? Does choosing not the best parameter set makes sense? Answer what I found is no. Best sum of unseen data profits is fully optimized strategies. Also it made profits in all 5 strategies, when with other methods some strategies  was in loss, for me it is indication for more robust results.

However it can be that choosing not the best parameter set is effective in other conditions. I mean that FSB chose best from ~300-600 optimization runs, but if I chose best from thousand runs from other backtesting platform, probably there is a lot bigger chance to get overfitted strategies, and that "Anti optimizer" would make sense and better end results Also I am aware that using bigger sample size can change things, but I am OK with what I did to stay away from this idea.

Used =RANDBETWEEN() in excel to generate random variable value sets to mimic FSB montecarlo test.

Some pics...

I am very low with expectations in trading, so got no sad about this... What's next? Try other things till success maybe? Still lots of ideas to test.

What is your craziest idea what you put in a test? For me is "remote viewing" for future market movement after 6 losses and 5 wins decided to stop and concluded that it is not working.

I like this topic -- and have a different perspective...

Consider OOS.  Many people feel that a strategy that performs well for both IS and OOS is more "robust" or less curve-fitted and will, therefore, perform better in a live account on future data.  But isn't there another explanation?  If the OOS portion is similar to the IS portion (i.e. the IS portion did a good job training the strategy for the OOS portion), then it is not surprising the strategy does well in both.  That is, the OOS portion really didn't exercise the strategy in a way that would reveal whether or not it truly is more robust.

In a recent post you mentioned something along the lines we have all the data, we just need to figure-out what the right question is to ask -- which is great insight.  So, in this particular experiment, what exactly was the question?  The experiment yielded interesting results that can help going forward.  I'm curious to hear what people think the right questions are.

For example -- here are a few questions:
 
Does it make a difference which data set is used when generating, optimizing and back testing a strategy?

Should you always use the latest (and greatest data set) when back testing, or is there a particular chunk of historical data that routinely does a good job of training strategies even though it might be "old"?

What is the definition of robust?  And if it is a good thing, then does a metric currently exist that measures "robustness"?  Or could a new metric be computed?

No Problem i feel it, and then i must say thanks, it is much big helping for me

I have read, that Mr. Popov from 01.10.17 EA-Studio make Pause and he will work on FSB, and i hope he can make Portfolio like EA-Studio.

But when you earlier, i hope you give it free, your work is very good

I have no experince with coding

Yes -- an indicator or strategy that is not used for trading, but only to provide a clue as to the trend or pattern of the incoming data.  And this could then be used to turn on / off different EA portfolios -- or something like that.

This would be most interesting...

Interest Idea, have you in live experience with this ea´s then?

How long live?

Thanks for replays guys! I see a need for experiment which will use 3 OOS to test out if double OOS would make money on third OOS Good thing that trading is not boring, because you will never run out of ideas to test. Until you become profitable then you will need to stick to what works...

it's just an example. This test will tell me whether to leave the strategy or not