bru1 wrote:...So I only trust OOS checked EA...
What you say is true. I've had the same experience -- so it is interesting to also hear from others. And I used to always use OOS, but no more.
Yes -- all strategies include curve fitting. Since that is the case, then I think it would be more valuable if OOS was reversed -- i.e. for 33% OOS the oldest 33% of the data was OOS and the most recent 67% was "in sample". Consider this example -- suppose you backtest for 3 years using 33% OOS. That means the first two years are "in sample" and the last year is OOS. Since curve fitting is part of the game, why would I want to use a strategy whose settings haven't been optimized for the most recent 1 year worth of data. Yes -- you would select only those strategies that performed well over the entire 3-year period. But you would be missing out on those strategies that would have performed well over the most recent year if you had included their most recent data when optimizing.
Another example -- suppose I have two strategies that I back test for 1 year. One uses 33% OOS and the other just "in sample" data. And in the end their statistics look very similar -- let's say they look identical. Which is the better strategy and why? Would you trust only one in a live account, or both or neither?
Prior to Popov's software I would be inclined to spend a lot of time creating one EA with the hope of using it for a long time. So, any test -- like OOS -- that improved "robustness" was valuable. But using Popov's tools I now have more EAs than I know what to do with. So I no longer consider a strategy a bad one if it performs well for a few months and then I have to replace it. I think that is one of the major paradigm shifts that Popov has introduced -- either intentionally or unintentionally.
I know this is getting long, so I'll wrap it up. I've noticed that settings make a big, big difference. That is, a strategy is composed of 1 or more indicators. With the proper settings, it works great. And if the settings are off, then it looks terrible. My point is that small changes to settings can make a big difference. This is why people advocate to "refresh" your strategies from time-to-time. If you are one who is willing to refresh your strategies, then I think that is also consistent with not using OOS.
Again -- just my two cents. Since I'm relatively new to forex then I want to better understand what really is behind the different trading strategies that people use and how those now fit in with FSB Pro and EA Studio.