1 (edited by jordytr971 2021-04-26 12:29:45)

Topic: How Walk Forward work ?

Hello, Im having a bad time trying to figuring out how Walk Forward Validation can help me testing robustness of my EA.
First of all why changing validation setting : "OOS Minimum net profit"  affect my result ? (Screenshot attached)

https://i.ibb.co/TggJRkH/Walk-forward-test-OOS-minimum-net-profit-0.png
https://i.ibb.co/41ZvF5k/Walk-forward-test-OOS-minimum-net-profit-10.png


Then how Walk forward validation is supposed to help me with robustness ?

Lets say my validation setting is  : OOS minimum net profit to "10"

I generate strategies then 1 passed AC and get to WF Validation
All segment is validated so that mean they have more than 10 of profit .
How that supposed to show me the robustness of the EA ? I don't get it

Thank you

Re: How Walk Forward work ?

> First of all, why changing the validation settings: "OOS Minimum net profit"  affect my result`?

The Optimizer (internally used in Walk Forward) validates each produced strategy variant against the validation criteria from the Walk Forward's Validation section.
It means that WF searches for the best result within the limitation of the Validation criteria. If you change the criteria, WF may accept or reject particular indicator values and ends with a different parameters set (strategy). Even if you see equal Profit for a segment, it doesn't mean that the indicator parameters are the same. You may confirm that in the Parameters section. You can also see the difference in the Statistics section segment by segment.

> How that supposed to show me the robustness of the EA? I don't get it.

I also don't see any particular benefit of that.

The WF advocates may state that the benefits are:
- The strategy is "optimizable". It means that optimization doesn't break it or even it can improve it. So, if you decide to optimize it after a period of trading, it may act well.
- Also it shows that the strategy works well for all the segment. (nothing that cannot be detected by DD and R-squared)
...

Anyway, in the way Walk Forward is publicly defined, it doesn't work well, in my opinion. We apply WF on an already tested strategy for the full data series. I mean that every strategy with a smoothly rising balance curve will pass WF by definition when we optimize each segment with +- 0 steps. The WF only proves that the Optimization doesn't destroy the performance. However, we set the Optimizer with a particular limitation.

If someone asks for my humble opinion, the Walk forward will serve its validation purpose if we initially generate the strategy on the first segment data only. Then we can walk forward on unknown and untested data.

Re: How Walk Forward work ?

Thank you for your clarification,
I understand better now.

Re: How Walk Forward work ?

Popov wrote:

If someone asks for my humble opinion, the Walk forward will serve its validation purpose if we initially generate the strategy on the first segment data only. Then we can walk forward on unknown and untested data.

When Pardo introduced WFA, then that was the whole point that one dataset would be for generating/optimizing the strategy, and then another one for validation. I haven't delved into Studio's WFA yet, but if the same dataset is used for generating and then it gets sliced into segments for WFA, I can't understand the time and effort, which was wasted on its development. If truly one and the same dataset is used, the results it offers are absolutely statistically invalid. The whole strategy collection is biased and WFA only adds to that bias.

Re: How Walk Forward work ?

It looks like we have to make a special Walk Forward Generator (possibly with Monte Carlo and Multi Market validator).

It can be an additional variant under the Generator tab. "Walk Forward Reactor"?

Re: How Walk Forward work ?

Popov wrote:

It looks like we have to make a special Walk Forward Generator (possibly with Monte Carlo and Multi Market validator).

It can be an additional variant under the Generator tab. "Walk Forward Reactor"?

I think so, yes! One way or another WFA is conceptually quite different and resource-consuming, so keeping it sort-of separate is the way to go. One thing though - to have the possibility to use WFA on already generated, old collections (for inexperienced users it needs to have remainders about using unseen dataset for actual effect).