If we see a great rate of return in the back-test it is not a guarantee of future profits, maybe the strat has a neat position bias in perfect harmony with overall net trend for a given data set. Later, when it's put out live the trend changes and we have a hot potato in our hands and a live account in a steep decline, even though it all looked so wonderful. Eventually, at some point we have to realize and accept the fact that our development has no predictive power.
One main gripe I've with Generator/Optimizer is that the combination as you said sometimes just happened to be position bias in perfect harmony...for a given data set/time.
But the real reason for such "failure" is that the generator/Optimizer often give us parameters that are theoretically unsound. That's my main reason why I don't use generator/optimizer at all for my strategies....99% of my strategies are "manually, self made". Once a while I venture off to give generator/optimizer a try but they are not able to produce consistent results because it is not based on a strong theoretical structure. Whereas, if you looked at my MT4 indicator chart, they are consistent and therefore stable, higher predictive power that translate to true results and the added advantage is that I can use it also in other currencies/instruments/commodities/multi markets.
That's why I propose for a Mini Optimiser, whereby instead of plucking random figures, I rather work on the same parameters but need the optimizer to choose which smoothing method or base price to uses.
One possible suggestion to improve the Optimizer or Generator....have a general rule to restrict the Generator or Optimizer from churning out "theoretically unsound" probabilities.
What do I mean by unsound probabilities?
Let's take MA as an eg. If we have fast MA 14 cross Slower MA 50, we would logically expect that it is a Bullish direction and vice versa.
However, what I noticed from the strategies generated or optimized, I sometimes see Bearish direction in the Opening conditions...it's against logic and unsound...unless such indicators is use as a divergent.
Hence, I propose that if we were to put certain "restrictions/ Trading rules" on the Generator or Optimizer to filter out these "unsound proposition"...we have a better chance of seeing more stable and profitable EA generated/optimized. We need to restrict the generator/optimizer from using a bigger parameter over a smaller parameter for certain indicators such as the MA and other similar indicators. Although this is a broad brush stroke that may eliminate certain strategies but if we were to compare the qualities of Strategies with and without this rule, I believe we would have better results from strategies that have such rules imposed.
Popov, focused and put a lot attention to ensure data/statistic soundness in FSB...but I think we also need to add in theoretical soundness in the way Generator/Optimizer function and the way indicators are used in Opening and Closing conditions.
In all fairness, user can have the option to enable/disable this rule. I'm rather excite to find out the results of having some Trading rules applied to the Generator/Optimizer. If this work, honestly, churning out profitable EA would be easier...imagine using Trailing Limit and we would have endless combination of EA generated that are theoretically sound. When that day comes...we all are laughing our way to the bank...I do see it possible one day as we fine tune FSB along the way. FSB 5 years ago and now is such a big improvement...I believe in the next few years or less, with each fine tuning...FSB one day will be THE Trading Tool, all traders must have