Forex Software — OOS Acceptance Criteria: Interesting Phemonima

Re: OOS Acceptance Criteria: Interesting Phemonima

2017-08-21T22:07:09Z

The generation was over the same number of bars (all generation occurs on the IS, whether or not you have an OOS element) and over exactly the same data. "Generated Strategies" is the first step in the process...Every time the generator finds a strategy with positive profit (I assume), it is checked against Acceptance Criteria. I am speaking of the number of strategies generated even before checking against Acceptance Criteria.

The "Generated Strategies" of the IS/OOS generation numbered more than ten times the number of those generated by the exclusively IS generation; therefor, it had a lot more strategies to check against the Acceptance Criteria.

Re: OOS Acceptance Criteria: Interesting Phemonima

2017-08-21T18:46:11Z

My bet would be that on less data bars it is easier to find strategies what pass your acceptance criteria. Maybe during longer data period market had changed from ranging/trending, and it is a lot harder to get good looking strategies

OOS Acceptance Criteria: Interesting Phemonima

2017-08-21T15:54:04Z

I woke up this morning and compared generations of IS vs. IS & OOS generation and validation (optimization + 100 MC tests).

I'm using identical IS periods, with the addition of a 30% OOS period in one of them (added on after the IS period of the first, making the total data 22500 bars instead of 15750 bars).

Acceptance Criteria and validation methods are identical for both IS and OOS.

Upon examination of multiple instances, I noted that the exclusively IS generations had generated about 60,000 strategies each.

I was surprised to see that the combination IS/OOS generations had generated in excess of 600,000 strategies during the same generation period.

For a few minutes, I thought that maybe my settings were wrong somewhere or that this was a bug in the new backtesting engine...but no, I realized that these results are correct and why they occurred.

Quiz: Can anyone explain the reason this happened?