I will start with my views on this subject. Please correct me and contribute with your opinions. Thanks in advance.
I recently read an interview to Robert Pardo, the "inventor" of Walk Forward Analysis in trading, and I got some ideas from that interview:
- About the window size (determining the optimum window size is one of the keys in the method) it is very important to have a significative number of trades in each window (to be statistically significant). So window sizes must be that to have (enough) a number of trades that could be considered significative.
- Window size depends then on how the strategy trades: if it trades quite often, the window sizes could be smaller. Smaller window sizes mean more adaptative strategies, but they must be re-optimized more frequently. On the contrary, if your EA trades less often, then you need bigger window sizes. In both cases, the number of trades both in the IS and the OOS must be significative.
- The number of windows (iterations in WFA) is also important. Concretely, the number of validations must be significative too. The robustness of the EA comes from the EA being able to survive many rolling windows and not only a few that may happen by chance. If the EA passes 10 out of 12 rolling windows it gives more confidence in the reoptimization method than passing 1 out of 3. In the last case, this succesful OOS (1 of 3) might have come by pure luck.
- The windows must be rolling. No anchored windows. The oldest data is less important, so rolling windows are prefered.
- A possible scenario could be 2 years IS - 6 months OOS --> (25% OOS). So, if you want 10 IS/OOS validations of 6 months you will need data from the last 7-8 years. If we don't have those data, we'll get smaller OOS sizes (with the same ratio IS/OOS) and we have to be careful to have enough number of trades on the window. An issue here is we usually don't have so much broker data (we have tickstory/dukascopy data, but don't usually have 8 years of H1 data from our broker, for example).
With these points in mind, I made my first attempts to WFA in EA Studio, so:
- Number of iterations must be high, but at the same time I must have enough OOS to have a number of trades.
What are your window sizes? I can have 10 of 6 months OOS in H4 timeframe, but cannot have that window sizes in H1 for example. I'm testing 2 months OOS right now. Maybe not the best balance. What do you think?
- When optimizing each IIS right now in EA Studio I must use Net balance/profit because otherwise, if I optimize using profit factor, sharpe or SQN for example, the number of trades in IS and OOS are very few. I think there is a problem there or a feature missing because I feel I shouldn't optimize each segment to have the better absolute balance, but the better profit factor or the better sharpe, etc.. However, it should be not the absolute profit factors or sharpes in each segments (because they usually come from taking only a very small number of trades), but these best conditions (best pf,sharpe,SQN) should be met with at least a number of trades. Best profit factor means nothing if there aren't at least a number of trades. On the contrary, best net profit isn't the best parameter to optimize, as the profit might come from a couple of lucky trades only. I think there's a problem in WFA implementation in EA Studio there.
- When using WFA in the reactor right now (and as I use a number of rolling windows) most of the process (processor time) is now consumed in the WFA. ES Studio must carry out n optimizations-validations for each strategy in the WFA stage. I'm not sure but I guess this process could be optimized. For example if I ask the EA to have 8 out of 10 validations, as soon as 3 windows fail the validation it is not needed to continue and finish the WFA. This way it will reduce the time the Reactor is in the WFA section. Another idea could be to continue searching other strategies while one EA is in the WFA section. Why stop generating strategies ehwn one is in the WFA stage? Couldn't we use a different processor / thread to continue generating EAs?
- Expiration dates. Once/If we have built our EAs based on a WFA it is supposed they must be re-optimized on a timely basis. Somehow we should remember that they must be reoptimized (for example) in six monhts from now. Sometimes I think there must be an expiration date in the EA to stop trading/remember us that the EA must be reoptimized before continuing trading.
Please contribute to this topic. I have a lot to learn yet and I'm here to learn from all of you to build better strategies. So far, I'm not taking advantage of using WFA and although I think it is a great improvement in EA Studio I haven't found a reliable way of using it yet.
So what do you think? How are you using the WFA analysis in EA Studio? what do you think I'm not doing well? Is there any idea I must take into account when using WFA?
Thanks a lot in advance!