Create and Test Forex Strategies
You are not logged in. Please login or register.
Forex Software → Expert Advisor Studio → Acceptance Criteria
Hello Traders,
A new version of EA Studio is uploaded.
Now it has three types of Acceptance Criteria:
- Complete backtest - it covers the complete data series statistics
- In Sample (training) part - covers the In Sample part of the data: from the first bar up to the OOS Bar (the white zone)
- Out of Sample (training) part - covers the Out of Sample part of the data: from the OOS Bar up to the end of the data series (the green zone)
Fantastic!
I'm super-excited about this. A couple of questions.
* Is Monte Carlo run on the entire dataset (IS & OOS)?
* Would it be possible to add validation statistics of IS & OOS separately?
* My initial trial run is 30% OOS, with Acceptance criteria of:
> Complete BT: Max amb Bars: 10; Min net profit: 10; Min count of trades: 100
> IS: Min net profit 100; min trades: 100
> OOS: Min net profit: 100; min trades: 50
Attached is one strategy push to the Collection. The OOS is clearly Net Profit < 0. This is the only such result out of about 40. Perhaps it's a case of the OOS colored area not correctly lining up with the actual OOS portion (combined with the super-small minimum value)? (I found another similar result after increasing the OOS Net Profit to 1000...still a small value)
Overall successful trial runs!
* Is Monte Carlo run on the entire data set (IS & OOS)?
Yes, it is.
* Would it be possible to add validation statistics of IS & OOS separately?
We have exactly that. Hmm, if you speak for Monte Carlo, it doesn't have OOS. If we decide to add it, the IS and OS validation will be possible.
I found adding 50%, 40% or 30% of Max stagnation useful also.
> The OOS is clearly Net Profit < 0.
This is a huuge rewrite of the backtest engine and some issues are possible. Please report all bugs and I'll try to fix them ASAP.
All three types of the Acceptance Criteria are validated only if the option is checked.
*
* Would it be possible to add validation statistics of IS & OOS separately?
We have exactly that. Hmm, if you speak for Monte Carlo, it doesn't have OOS. If we decide to add it, the IS and OS validation will be possible.
Sorry, I was not at all clear with this question. What I mean by "validation statistics" is the statistics section on the main screen of the Reactor, showing separately Generated/Calculated strategies #s, and the number/% that have passed validation.
In this way, we could see (with our current settings) what percentage of strategies are being rejected due to poor OOS performance. This could indicate potential problems with our Acceptance Criteria.
I wasn't referring to OOS Monte Carlo testing. Actually, MC testing should probably be done over OOS data as well, but I've always found it far too brutal for even ME to use. On the other hand, if a strategy holds up with OOS MC testing, it would be almost assured to have great potential.
Because we are generating so many strategies, it may very well be feasible to do this. At the very least, the strategies that pass OOS MC tests could be given a little extra leeway when deployed live. I think this would be a good addition, but I would guess that not many people would use this feature until it starts to prove itself. The reason for that is that all of us here are dreamers and being told that only 2% of our otherwise validated strategies don't pass the stupid OOS MC test won't validate our dreams very well.
It's good to see that the project has never been so active.
We can see that you are really committed to delivering a product of value and listen to the community, deserve every success in the world.
This is a excellent news to every eastudio user, my congratulations to everyone involved on it.
It's good to see that the project has never been so active.
We can see that you are really committed to delivering a product of value and listen to the community, deserve every success in the world.
This is a excellent news to every eastudio user, my congratulations to everyone involved on it.
That was one of two major reason I bought EA Studio/FSB. I was very impressed by Popov's engagement and responsiveness. It seems that this is more of a passion for him than it is "work" and that's reflected in this forum.
ThanX for your hard work and constant involvement and communication, Popov!
My Generation using very minimal OOS settings is proving very enlightening for the very short time I've used it. I'm not surprised by the results at all, but I'm absolutely positive that 90% of the members here will be. I'll post further details in five hours or so.
Popov, can you reduce the minimum possible value for R/DD to "0"?
for very small OOS sample sizes, "2", the current minimum is probably far too large. I'm certainly getting results with that number, but there's the possibility that many good strategies are being filtered out at this level.
My current OOS is ~7 months, and I can certainly envision smaller OOS sizes.
Last night I left EAS spanking the data with only 2 criteria to find a good slope trading system
All Data: Maximum equity drawdown: 5%
In Sample: Minimum profit per day > 10
OOS: Minimum profit per day > 10
Very happy to see the output, until now it's Working like a charm
You've still got to be very careful even with the nice-looking OOS data. We sort through so many strategies that you are bound to find a few that are still "false-positives" , depending on the length of your data sample.
But, yes, this OOS Acceptance Criteria makes many more things possible.
Forex Software → Expert Advisor Studio → Acceptance Criteria
Powered by PunBB, supported by Informer Technologies, Inc.