Popov, thank you, but it would be VERY important (to me at least) that the extra acceptance criteria for the Optimizer also has 3 parts: Complete Backtest, in sample, out of sample, like the normal acceptance criteria for the generator (and how it was before for the optimizer too). So that if selecting "10% OOS" for the optimizer in the reactor or directly within the optimizer, we can specify for example "at least 95 R-Squared for the COMPLETE backtest, but also at least 80 R-Squared for the 10% OOS part", then begin the optimization. I have always used this to find strategies that are overall stable, but ESPECIALLY stable in the last years (the 10% OOS). This option is not possible anymore now, having just 1 set of acceptance criteria for the whole backtest (or IS, as the optimizer will ignore the OOS part now and we can set now minimum values for the OOS part anymore during an optimization), which is a pitty :-(

I´ve awlays used the optimizer this way: set it to 10% OOS, set the acceptance critera for the 10% OOS to at least a R-Squared of 80, then optimize the IS part for maximum R-Squared value, this isn´t possible anymore now and basically breaks my complete workflow of how I´ve created my good strategies :-(

I think it would be better if you´d simply add a second set of criterias in the settings page and then, in the Reactor / Generator / Optimizer, we can select "criteria set 1" for the generator, and "criteria set 2" for the optimizer, or use "criteria set 1" for both if we don´t need different acceptance criteria. Both "criteria set 1" and "criteria set 2" would have acceptance criteria for the complete backtest, for the IS and for the OOS part. If you can´t do that, can you please bring in the 3 split-parts into the Optimizer directly as well again (complete backtest, IS, OOS). That would be higly important to me as I am stuck now with only having 1 set of criteria for the Optimizer.

On large data-sets that is a real problem, because optimizing for a high R-Squared for the complete backtest can sometimes result in great first 28 years, but bad last 4 years (as I use 32 years of data). When we had the option to optimize the IS part for maximum R-Squared (or whatever) but FORCE at least a R-Squared of 80 in the 10% OOS part, the optimizer found great strategies that would be very stable overall and not just 28 years and then go down sligthly like it often is the case after 2015.

If using the new optimizer now, I have no way to set which minimum criteria to have in the 10% OOS. If I set it to 10% OOS now, the optimizer will only care about the IS and will completely ignore what is going on the in 10% OOS. Previously, the optimizer would also only optimize the IS part, but we could set that in the 10% OOS, it should at least have 100 trades, or at least return/drawdown of 10, or a R-Squared of 80, or whatever. Now it does absolutely not care anymore what happens in the last 10% of the backtest and just optimizes the IS part, regardless of what happens in the 10% OOS part. And that is a real problem at least for me, as I can´t optimize the way I´ve optimized anymore.

So summarized:
- Please bring back the option to use separate critera for the complete backtest, IS and OOS in the Optimizer instead of just 1 set of acceptance criteria.

Thank you so much!!!

Thank you Popov.
Thanks.
Great.

Hi Popov

Very nice work

1. Do the Acceptance Criteria of Generator AND Optimizer are common and they are selectively in Optimizer can supersede the validation criteria of Optimizer by changing its settings? Am I correct?

2. I used Reactor with optimization 00S 10% but it did not used Validation criteria from optimizer. Is it Bug or it is expected?

Optimizer alone with validation criteria looks to work.

An explanation of how things work now is needed.

+1 and as I´ve suggested earlier already. It´s confusing now that some acceptance critera are in the settings page (for the generator only from here on) and others (for the optimizer) are within the Optimizer. Better just make 2 acceptance criteria sets in the Settings page and then let us assign them to the generator / optimizer, etc.

Roughey

Thanks  a lot. hehehe. Popov has fixed already...

Thanks for the quick change Mr. Popov, that is amazing fast in the middle of the night - thank you so much!!! :-)

@GD: Yes, it seems like they are ignored if used through the Reactor and work only in the Optimizer directly, but I have to do more tests to confirm. I am sure Mr. Popov will quickly fix it though as always - what a great guy!

Popov

It looks fine. I remind you also to check OOS report  to be compatible
AND if possible create and 5% OOS.

Thanks in Advance.

To have different criteria for Generator and Reactor creates a new era in the development of EAs.

Hi Popov

I use as first example Reactor and " in sample" in Generator and in Optimizer and in Editor. I used only Generator and Optimizer clicked.
I made simple acceptance criteria in Generator (clicked) and I accept same acceptance critiria (clicked) in Optimizer (inside editor). In Optimizer I added only minimum r-squared 85% in sample in validation criteria.

Then I run Reactor.   Strategies added to collection. I checked them with editor and I found in OOS monitor and in Stats info r-squared  lower than 85. I expected to see in the collection only the ones with r-sqaured higher than 85. I did not.

Do I have mistake or there is a bug;

Relative to this oppinion

"We may add options in the Reactor optimizations for whether to use the optimizer criteria or the generic ones."

It sounds good. There is a button in editor/ optimization which permits to user to use Acceptance criteria if clicked.
It is simple if it is not cliccked the validation criteria to be used automatically. But it has to be mentiond in that point somehow.

Another way is to keep acceptance criteria in optimizer which do not change in Optimization criteria together with optimisation criteria.

The same manipulation can be done if used In sample and out sample.

This way looks simple for new user

it looks great.

What about if common selections in criteria Optimization and Acceptance Criteria exist ? Optimizer`s should have priority. Right?

I´d still think the easiest solution would be to have:

"Tools" -> "Acceptance Criteria Set 1"
"Tools" -> "Acceptance Criteria Set 2"

In the Generator / Reactor / Optimizer we´d have:

"The strategies fulfill the Acceptance Criteria" as it is right now and to the right side is a dropdown menu to select "Acceptance Criteria Set 1" or "Acceptance Criteria Set 2".

And it would default to "Acceptance Criteria Set 1" for the Generator / Optimizer, so it´s easy for new users and they don´t have to worry about it at all and setup different acceptance criteria in the Tools and additional in the Optimizer. This just complicates things too much for them. And us experienced users can use different acceptance criteria sets for generator / optimizer if needed. This also keeps the whole Acceptance Critera setup in the "Tools" section, not cluttering things up to different places like it is now.

Just my 2 cents :-)

I will check. Congratulations.

I also think all the different naming is confusing.

It´s called "Tools" -> "Acceptance Critera". But under the "Optimizer" it is called "Validation" instead of "Acceptance Criteria", and says "Add validation criteria" instead of "Add acceptance criteria" (which is the same problem under Tools -> Acceptance Criteria too).

I think you should stick to a continuous naming:

Tools -> Acceptance Critera -> Add acceptance criteria
Optimizer -> Acceptance Critera -> Add acceptance criteria

It´s very confusing for new users I´d think, as they do not understand why it is sometimes called this way and other times another way.