Just finished experiment. Maybe someone will find results useful
Planned to do it with at least 20 strategies, but stopped with 5, because it is very time consuming and I can see that strategies results getting worse, so for me it is enough proof not to use it. Used different strategies, pairs and time frames.
Checked 4 variants for selecting parameters. Variants:
1. optimized (FSB full optimization box checked, getting best one from about 200-600 optimization tests)
2. 4th from the top 20 random generated variable sets (SQN sorting, 80% percentile)
3. 10th from the top 20 random generated variable sets (SQN sorting, 50% percentile middle)
4. average indicator values from all 20 random variable sets
Question was if I choose not the best backtest parameters will it trade better than optimized results on unseen data chunk? Does choosing not the best parameter set makes sense? Answer what I found is no. Best sum of unseen data profits is fully optimized strategies. Also it made profits in all 5 strategies, when with other methods some strategies was in loss, for me it is indication for more robust results.
However it can be that choosing not the best parameter set is effective in other conditions. I mean that FSB chose best from ~300-600 optimization runs, but if I chose best from thousand runs from other backtesting platform, probably there is a lot bigger chance to get overfitted strategies, and that "Anti optimizer" would make sense and better end results Also I am aware that using bigger sample size can change things, but I am OK with what I did to stay away from this idea.
Used =RANDBETWEEN() in excel to generate random variable value sets to mimic FSB montecarlo test.