* Would it be possible to add validation statistics of IS & OOS separately?
We have exactly that. Hmm, if you speak for Monte Carlo, it doesn't have OOS. If we decide to add it, the IS and OS validation will be possible.
Sorry, I was not at all clear with this question. What I mean by "validation statistics" is the statistics section on the main screen of the Reactor, showing separately Generated/Calculated strategies #s, and the number/% that have passed validation.
In this way, we could see (with our current settings) what percentage of strategies are being rejected due to poor OOS performance. This could indicate potential problems with our Acceptance Criteria.
I wasn't referring to OOS Monte Carlo testing. Actually, MC testing should probably be done over OOS data as well, but I've always found it far too brutal for even ME to use. On the other hand, if a strategy holds up with OOS MC testing, it would be almost assured to have great potential.
Because we are generating so many strategies, it may very well be feasible to do this. At the very least, the strategies that pass OOS MC tests could be given a little extra leeway when deployed live. I think this would be a good addition, but I would guess that not many people would use this feature until it starts to prove itself. The reason for that is that all of us here are dreamers and being told that only 2% of our otherwise validated strategies don't pass the stupid OOS MC test won't validate our dreams very well.