1 (edited by hannahis 2018-07-21 08:39:06)

Topic: OOS filter?

Hi is there a filter whereby you can filter out strategies that didn't improve in the OOS?

For example, the Green section shows the equity curve of the strategy's performance under OOS data.

Some strategies improve in it's equity curve under OOS data, i.e. the equity curve goes up (instead of down), whereas there are some strategies that performed worse during OOS data, ie. the equity curve goes down.

The problem is - both these 2 groups of strategies still passed the overall performance test, ie. they both show positive equity curve.

I want to prune those Strategies that didn't make profit during the OOS data even if their overall performance show positive equity.  These strategies are likely be curve fitted and hence didn't make profit in the OOS data.

So is there such a filter currently available (if yes, what is it named as?)  I try searching but didn't seem to find one that fit the abovementioned function.

PS: Not all strategies that showed negative growth under OOS data means it's curve fitted.  It could be that the OOS data doesn't fit the strategy's market conditions (eg. A trend strategy and the OOS data happened to be a ranging market).  However if the OOS data is long enough to cover various market situations, then there is a high chance that Strategies that didn't show improvement in it's equity curve during OOS testing are likely curve fitted.

Re: OOS filter?

Yes, you can do it via the Acceptance Criteria - OOS part.

Set Minimum net profit to some value and the Generator will produce only strategies that comply with that.

https://image-holder.forexsb.com/store/acceptance-criteria-oos-part-thumb.png

You need to have the option "Strategies fulfill the Acceptance Criteria" option on in the Generator / Reactor.

You may add as many criteria as you want.

3 (edited by hannahis 2018-07-21 09:18:09)

Re: OOS filter?

Popov,

It's not about the min Net Profit because different strategies have different NP level.

I'm looking for the EA Studio to automatically check the equity curve before and after OOS, whether after the OOS testing, the strategy continue to make money during the OOS data? or it loses money during the OOS data.  It's not the same as how much is the minimum Net Profit, cos it doesn't matter how much the strategy made, I'm interested whether the Strategy's rule continue to stay robust and profitable during it's OOS testing.

A strategy can make lots of Net Profit during IS testing but if during OOS testing, it didn't make any profit (i.e OOS's Equity Curve is lower than the IS's equity curve), that could also mean the Strategy's rule isn't robust.  Hence, it doesn't matter even if it made lots of money during IS data cos it's performance may be curve fitted according to the IS data for it to do well.

Re: OOS filter?

I'm looking for the EA Studio to automatically check the equity curve before and after OOS,

Yes, it can do it. EA Studio has separate Acceptance Criteria for the in-sample part, for the out-sample part and for the complete backtest. By choosing proper criteria, you can be sure that the Generator will  produce only strategies that satisfy your expectations. For example, the default OOS Criteria for the OOS part is min net profit = 0 (shown to my previous screenshot). It means that the OOS part will make profit. You may set other criteria for the OOS part, if you want.

5 (edited by hannahis 2018-07-21 13:14:24)

Re: OOS filter?

Oh I see what you mean now. 

So the Acceptance criteria for the OOS would only mean to calculate the Strategy's Net Profit ONLY during OOS, not the overall Net Profit (not the sum of IS & OOS overall Net Profit)

That's what I need.  Thanks for taking your time to clarify further.  Appreciate your guidance.