Topic: Walk Forward

Hello Traders,

We have Walk Forward in EA Studio.

It work for a single strategy and also in the Reactor as a validation tool.

https://s15.postimg.cc/cr0dk839n/screenshot_241.png


https://s15.postimg.cc/pv5xx287v/screenshot_242.png

I'll provide documentation within two days.

Please take at the Walk Forward tool, play with the options and post your opinion.

Trade Safe!

Re: Walk Forward

Good job!!! big_smile big_smile big_smile

Re: Walk Forward

Very nice, going to test this...

Re: Walk Forward

Hi,Popov

And the next Walk-Forward tool will be in FSB Pro also,Right?

5 (edited by moreirajbn 2018-06-17 17:29:51)

Re: Walk Forward

This is awesome.

Can you give some quick explanation on how it works in Reactor? Does the engine expose the strategy to the entire period first, and THEN run walk forward on the same period?

Re: Walk Forward

Can you give some quick explanation on how it works in Reactor? Does the engine expose the strategy to the entire period first, and THEN run walk forward on the same period?

The reactor gives the strategy and the complete Data Set to the Walk Forward engine. It calculates the segments according to the settings and cuts the Data Set in parts. It starts optimizing form the first segment's In Sample part. Than the Walk Forward calculates the OOS backtest and stats. It also validates the OOS parts according to the validation criteria set in the Walk Forward page. When the engine finishes with the first segment, it continues with the rest one by on at the same way.
When all the segments are calculated and validated, the Reactor compares the count of the validated segments with the set threshold. If there are enough validated segments, the Walk Forward passes the strategy to the next selected tool (if any).

7 (edited by GD 2018-06-17 18:42:13)

Re: Walk Forward

Hi Popov
Nice job
Could you check please the scaling of graph in X-axis. It looks to exist some delay in last green space.
GD

Re: Walk Forward

Yes, I also noticed that. I'll check it more thoroughly.

Re: Walk Forward

Well done Popov - excellent feature to be implemented!

Re: Walk Forward

A new version of EA Studio is uploaded.

It has the Walk Forward segments calculated correctly and also some other minor fixes.

Please reload the app to use the newest version.

Re: Walk Forward

Actually I will spend the next two days in making the Walk Forward more powerful.

I'll add chart line of the initial equity before the optimization of each segment.

I'll add also tables for all stats params like that:

Net Balance
Segment #, Original , Before Optimization, After Optimization, Gain

Later I'll think of implementing Matrix calculations. My concern about it is that the output may become blotted with stats data.

12 (edited by moreirajbn 2018-06-18 23:33:54)

Re: Walk Forward

Popov wrote:

Actually I will spend the next two days in making the Walk Forward more powerful.

Since you're working on improving WFA, I have a suggestion:

As it is now, it's running the walk forward process on market data that the strategy development process has already been exposed to (the same period from where the strategy was discovered). So, technically the WFA is suffering from data snooping, because it's "walking" on the same data used to develop the strategy. For example: if 2005-2010 is used in the "Data Horizon", the Generator is looking for strategies that work on 2005-2010 period, and then the Reactor runs WF validation on the same period.

To simulate the "true" walk forward concept, the in-sample period used to discover and optimize the strategy should be completely independent of the out-of-sample, since this is what will happen when we are developing a strategy to run in the future (we have data from XXX to June-2018, and the key challenge is to be profitable after that).

So, my suggestion is: if the user selects walk forward validation in Reactor, and 2005-2010 is selected in "Data Horizon", only the first in-sample period should be available to Generator. Let's say that in-sample period is 1 year, then the data mining process should only run from Jan-2005 to Dec-2005, and the remaining period (2006-2010) used for walk forward validation.

Awesome work you're doing here!!!

Re: Walk Forward

Looks very good Mr. Popov, good work!

@moreirajbn: I understand what you say, but different users = different approaches. I use it for parameter stability testing, so data snooping (using the same data the strategy was generated on) is not an issue in that case and I do not like to be forced data that has "not been used before", especially since I create my strategies on 32 years of data without any OOS. So if at all, this should become and OPTION, not something that is forced upon the user.

Thanks smile

14 (edited by moreirajbn 2018-06-19 00:14:58)

Re: Walk Forward

geektrader wrote:

Looks very good Mr. Popov, good work!

So if at all, this should become and OPTION, not something that is forced upon the user.

Thanks smile


Excellent point @geektrader... Totally agree that "unseen data for walk forward" should be an option, not a static feature :-)

Re: Walk Forward

@moreirajbn your approach is reasonable.

Fortunately, you can solve this problem with EA Studio:
- Decide what time span you will use for creating your strategy and for testing on unseen data ( OOS ).
- Set the first time span in Data Horizon and built and test the strategy.
- Set the second time stamp in the Data Horizon (You may overlap the data span). EA Studio will show you the full performance of the strategy on the OOS data.   
- Use Walk Forward to see if the strategy will perform better, if you optimize it on segments.

Re: Walk Forward

I've been looking for a viable solution for years, I'm going to use it right away! Congratulations to those involved smile

thanks in advanced! cheers from Brazil

17 (edited by GD 2018-06-19 14:46:53)

Re: Walk Forward

Dear Popov,

it looks very promising. It gives another way of optimization.

https://s22.postimg.cc/o1x0wwmod/2018-06-19_22-40-14.jpg

I wonder if to include percent at gain can be helpful

#    Before Optimization    After Optimization    Gain
1    2769                                   2120                     -649   (23%)
2    2270                                  1952                     -318   (14%)
3    2219                               2007                      -212 (9.5%)

Re: Walk Forward

I'm making a new stats table inside the Statistics tab. It will show the full info.

Re: Walk Forward

Added full OOS stats by segments. Please reload the app.

https://s15.postimg.cc/ase55cey3/screenshot_245.png