26 (edited by GD 2019-01-21 08:48:50)

Re: Acceptance Criteria for the Optimizer - Development

Hi Popov

It is a good solution and fast!

There is still something which should be clarified to the user.

If Reactor run and a user selects a strategy from the collection created to treat with editor (reactor still is running).

Suppoose user changes Optimisation criteria in Editor and/or during that, he does or he does not want to use optimization criteria, what will change with reactor which runs, with the Optimization criteria and/or acceptance criteria allready clicked?

Re: Acceptance Criteria for the Optimizer - Development

That would be interesting to know, wondered about that too.

Re: Acceptance Criteria for the Optimizer - Development

Actually a synchronization between the Reactor's Optimization options and the one in the Optimizer is not a good idea. the problem is that we will not be able to optimize individual strategies during the Reactor work.

Re: Acceptance Criteria for the Optimizer - Development

Suppoose user changes Optimisation criteria in Editor and/or during that, he does or he does not want to use optimization criteria, what will change with reactor which runs, with the Optimization criteria and/or acceptance criteria allready clicked?

The Reactor must copy the Optimizer's criteria at the moment of starting and to use its own copy. It is not the case right now, but I'll do it.

30 (edited by geektrader 2019-01-21 10:06:53)

Re: Acceptance Criteria for the Optimizer - Development

I agree with both Popov.

31 (edited by geektrader 2019-01-22 12:05:05)

Re: Acceptance Criteria for the Optimizer - Development

Hi Mr. Popov,

I think there is still something wrong if using the Optimizer within the Reactor and just using the "Common Acceptance Criteria". If the "Command Acceptance Critera" is use for the Generator AND the Optimizer, both with 10% OOS, no strategy should fail the validation test after the Optimizer, because the Optimizer would take into account the rules for the 10% OOS too (like ret/dd ratio minimum of 2). The Optimizer would then get the strategy from the generator and the strategy hence already fullfills the acceptance criteria (because both Generator and Optimizer are using the Common Acceptance Criteria with the exact same values and 10% OOS). During the optimization in the Optimizer, the Optimizer and if for example optimizing for best R-Squared value, the Optimizer would only find new best results if the OOS acceptance criteria have been fulfilled too - at least that´s how it always has been before you gave the Optimizer it´s own criteria. But now, I often see that the Optimizer optimizes a strategy but then it does not pass the validation of the Optimizer. Which means that the optimizer must ignore the acceptance criteria during optimization for best R-Squared (or whatever value) and only tests it AFTERWARD. But that makes no sense, previously the Optimizer always only found a new best result if all acceptance criteria have been met already during the optimization - it always worked like that until you´ve made the new change, now it doesn´t work like that anymore :-( Now ~50% of strategies that have been forwarded from the Generator are optimized, but don´t pass the validaiton of the Optimizer. Before (with the exact same settings like now), 100% have always passed the second step of the Optimizer validation (which is correct behavior), as the Optimizer must already obey the acceptance criteria during the actual optimization, not just afterward.

Example:

1) Generator and Optimizer both use the Common Acceptance Criteria (complete backtest: ret/dd of at least 4, 10% OOS at least a ret/dd of 1).
2) Both are set to 10% OOS
3) Optimizers goal is to optimize for best R-Squared, with 10% OOS
4) The Optimizer only gets strategies that have already passed the Common Acceptance Criteria from the Generator, so the strategies ALL must pass the validation-step of the Optimizer too and the Optimizer should ONLY find a new best R-Squared value during it´s optimization IF it can keep the ret/dd of 2 in the 10% OOS too - and that´s how it always was the case before that change (it has worked like that for the whole last year). But exactly that does not seem to be the case anymore, it seems to ignore the 10% OOS min ret/dd of 2 during the optimization and just optimizes the IS for best R-Squared, accepting any results that have a better R-Squared in the IS, regardless if the OOS ret/dd falls under 2. Hence it can happen that the strategy does not pass the validation-step of the Optimizer anymore - which never was the case before (always 100% passed, which is logical and it worked like that for the last year).

Re: Acceptance Criteria for the Optimizer - Development

Here is a picture that might make it more clear what I mean:

https://i.imgur.com/3liEBRn.png

Re: Acceptance Criteria for the Optimizer - Development

Thank you for the report!

It is fixed.

Please reload the app.

34 (edited by GD 2019-01-22 20:18:28)

Re: Acceptance Criteria for the Optimizer - Development

Thanks Popov

I wlll check it an report.

And what about this situation?

There is still something which should be clarified to the user.

If Reactor run and a user selects a strategy from the collection created to treat with editor (reactor still is running).

Suppoose user changes Optimisation criteria in Editor and/or during that, he does or he does not want to use optimization criteria, what will change with reactor which runs, with the Optimization criteria and/or acceptance criteria allready clicked?

Re: Acceptance Criteria for the Optimizer - Development

Suppoose user changes Optimisation criteria in Editor and/or during that, he does or he does not want to use optimization criteria, what will change with reactor which runs, with the Optimization criteria and/or acceptance criteria allready clicked?

This is already fixed. The Reactor makes own copy of the Optimizer Acceptance Criteria at the moment of starting.

36 (edited by GD 2019-01-22 20:29:11)

Re: Acceptance Criteria for the Optimizer - Development

Ok. Everything is clear.

We can save all criteria by using  Settings export. Right?
And then we can run Optimizer for any strategy from collection... changing the criteria of Optimizer.
Maybe a button to export settings from inside each tool can be useful.

Please think if possible to create graphics for portfolio EA as you did for each EA.
So we can make combinations better.

Re: Acceptance Criteria for the Optimizer - Development

Popov wrote:

Thank you for the report!

It is fixed.

Please reload the app.

Super, thank you - will check.