Topic: OOS Analysis and Over Curve-Fitting
I'm a big fan of OOS Analysis, and next to fast strategy generation and backtesting it is one of my favorite features. Over curve-fitting is the biggest enemy of algorithmic traders and OOS analysis provides a mechanism to mitigate the harm. However, by including OOS analysis in your work flow can you guarantee your strategies won't be over curve-fitted? That is the question...
ANSWER: It's a rhetorical question -- and the answer is: It depends. The curious might ask "What does it depend on?" It depends on the market conditions in the IS and OOS sections of your data horizon. If the market conditions are the same in the IS and OOS sections then the strategy will likely be over curve-fitted and OOS analysis offers little to no benefit. Think about it -- the strategy is optimized against the IS data. If the market conditions are the same then the OOS section will perform the same as the IS section and does not provide a filter to detect over curve-fitted settings. On the other hand, if the market conditions are different then the OOS section can provide a filter to detect over curve-fitted settings.
Question #2: How do you know whether market conditions are the same or different in the IS and OOS sections?