Topic: Collections compilations results using different "Sort by" settings

I accumulated 16 Strategy Collections of 100 strategies for GBPUSD over roughly 3 days, each having been built with the “sort collection by” = Return/drawdown, so that might be said to be a bias in their make-up. 

I saved them, then uploaded all 1600 strategies to EA Studio using the Validator with all of the visible boxes (Optimized and all Robustness) unchecked. 

For each “Sort by” setting on the Collection page, I was left with the best 100 for each sort method, and then copied all to the Portfolio for easy Summary Stats availability for each sort. 

As a Return/DD fan, I think I am suddenly a fan of Average HPR.

Attached are the results ($10,000 initial balance, 0.10 lot size, GBPUSD M1, 500,000 bars (~16.5 months) of my broker data). 

Do these generally match with what others have found?

Post's attachments

2019-04-14 19_56_13-Collection compilations by sort results.xlsx.png 47.74 kb, 1 downloads since 2019-04-15 

You don't have the permssions to download the attachments of this post.

Re: Collections compilations results using different "Sort by" settings

Good job! This the intended way to use EA Studio.

You can pass the strategies through Monte Carlo if you don't do it during the generating process.

Re: Collections compilations results using different "Sort by" settings

The results look excellent!

Spreadsheet is a gret tool, thank you.

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....

4 (edited by Blaiserboy 2019-04-18 13:43:37)

Re: Collections compilations results using different "Sort by" settings

question:

Did you run the Validator 16 times to get those results or just change the filter for the initial result.?

I am pretty impressed with your spreadsheet results and want to try to make a comparison.

Thanks

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....

Re: Collections compilations results using different "Sort by" settings

I guess I can just gather the results and place them on a sheet, I have no idea as to how to combine the results, I have never learned about spreadsheets..

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....

6 (edited by Blaiserboy 2019-04-18 16:12:26)

Re: Collections compilations results using different "Sort by" settings

This is on EURJPY  240 minute

I am not sure which result I prefer to use.  lol

I did not have as many results to start with, just 474, so I am unsure as to the accuracy.

And many of the results are really similar... I have no knowledge of statistics, hard for me to judge  lol

Post's attachments

EURJPY 4 hour April 19.jpg 120.69 kb, file has never been downloaded. 

You don't have the permssions to download the attachments of this post.
My 'secret' goal is to push EA Studio until I can net 3000 pips per day....

Re: Collections compilations results using different "Sort by" settings

Hi,

Thank you for the kind words, gentlemen.

Yes, I ran 16 concurrent instances of EA Studio on my 20-CPU computer, running the Generator for each instance with the Sort By set for Return/DD.  After roughly 3 days, I saved the Top 100 results in the Collection tab for each instance, resulting in 16 files.

Then I set the Sort By filter for the first option, imported all 16 files using the Validator, in the process squeezing out the worst 1500 and leaving the top 100 Strategies when sorted by that option, then Added the Strategies to the Portfolio to see the overall stats, and manually recorded them in a spreadsheet.

I then changed the Sort By filter to the next available option and repeated the sequence 11 more times (only showing 11 because the Profit Per Day results were identical to the Net Balance results).


Thank you for doing the same with your H4 chart files.  I would have guessed that the results might show less variance from that what I might call "higher perspective", but am surprised to see how they are generally more similar to each other than my M1 chart files.  I don't suspect your lesser number of  strategies had much to do with it.

Of course there are different philosophies to trading and it is very nice that to have a tool like EA Studio which allows using and exploring such a variety, and so very efficiently.

Successful investing to you,

Re: Collections compilations results using different "Sort by" settings

Part 2

What a difference a development phase Sort By decision can make for combined GBPUSD M1 results.

Being curious if a different Sort By choice would make much of a difference in the combined results table, I followed the same routine,  but selected R-Squared as the choice during Phase 1, as I generally thought I saw more consistent balance line chart growth among the better of them than any other Sort By choice.

Interestingly, some R-Squared Stats still did not fare well—it had no top results and several worst results-- when compared to several other Sort By choices during the squeezing-out-the-worst of Phase 2.

Looking at the results this time, I am more a fan of the Sharpe Ratio results instead of the Average HPR in the first experiment.

All of them can make money, but some combinations of a Phase 1 Sort By with a particular Phase 2 Sort By may generally be a better choice than others.  Your Mileage May Vary.

I will attach both tables together below for easier comparison.

Post's attachments

Collection compilations by sort results 2.xlsx.png 114.36 kb, file has never been downloaded. 

You don't have the permssions to download the attachments of this post.

Re: Collections compilations results using different "Sort by" settings

Dunn wrote:

I saved them, then uploaded all 1600 strategies to EA Studio using the Validator with all of the visible boxes (Optimized and all Robustness) unchecked.

Why did you decide not to also filter with optimization and robustness tests?

Re: Collections compilations results using different "Sort by" settings

tonyD wrote:

Why did you decide not to also filter with optimization and robustness tests?

Good question.  Several reasons.

(Bear in mind I am working with M1 charts, so some of my initial observations and thoughts might not apply the same to some of the higher time-frames.)

Speed was a very significant factor.  Generally, Optimization and Robustness tests seem unpleasantly slow (especially on 500K bars), to a beginning user just trying to assemble even a small battle-ready army of EAs.  This computing time might be better spent calculating millions of other strategies, a few of which might be even better.

--

Optimization specifically, I am not currently doing,  in order to avoid the chance of getting overly-optimistic indicator settings, which might work great with the current data set, but might generally be worse with other sets. I’m thinking random might generally be safer…unless not! :-)

This also seems inter-related with:

After seeing very attractive strategies disappear during development when "resolve correlations automatically" is checked, I decided I wanted to make that decision myself, to remove or keep them, so I have that disabled.

And because Optimization of some of the best strategies might make them all the same and then delete many of them, multiplying the problem, I also decided I wanted to just leave the indicator settings as randomly chosen which worked well.  If multiple strategies happened to use the same indicators but used different randomly generated settings, keeping them all as-generated might provide some degree of risk spreading.

--

Walk Forward Validation. During development, after visually inspecting and deleting all of which I did not like the smoothness of balance line growth, only "good" strategies remained.  I like the concept of Walk Forward, but I noticed all my best strategies passed Walk Forward. 100%.  I thought I would save a lot of time by not doing this, at least for the best, smoothest strategies.
--
Monte Carlo validation.  About 87-95% of my "good" strategies were passing.  I thought the performance penalty outweighed the value here.  I might come back to this later after I have amassed a large army of EAs for possible whittling down for refinement.
--
Multi-Market validation. My initial thinking is that I do not like the concept of this very much, especially for M1 charts, and especially for a mixture between Forex and other instruments such as indexes and commodities.

11 (edited by tonyD 2019-04-22 13:09:28)

Re: Collections compilations results using different "Sort by" settings

Dunn wrote:
tonyD wrote:

Why did you decide not to also filter with optimization and robustness tests?

Good question.  Several reasons.

(Bear in mind I am working with M1 charts, so some of my initial observations and thoughts might not apply the same to some of the higher time-frames.)

Speed was a very significant factor.  Generally, Optimization and Robustness tests seem unpleasantly slow (especially on 500K bars), to a beginning user just trying to assemble even a small battle-ready army of EAs.  This computing time might be better spent calculating millions of other strategies, a few of which might be even better.

--

Optimization specifically, I am not currently doing,  in order to avoid the chance of getting overly-optimistic indicator settings, which might work great with the current data set, but might generally be worse with other sets. I’m thinking random might generally be safer…unless not! :-)

This also seems inter-related with:

After seeing very attractive strategies disappear during development when "resolve correlations automatically" is checked, I decided I wanted to make that decision myself, to remove or keep them, so I have that disabled.

And because Optimization of some of the best strategies might make them all the same and then delete many of them, multiplying the problem, I also decided I wanted to just leave the indicator settings as randomly chosen which worked well.  If multiple strategies happened to use the same indicators but used different randomly generated settings, keeping them all as-generated might provide some degree of risk spreading.

--

Walk Forward Validation. During development, after visually inspecting and deleting all of which I did not like the smoothness of balance line growth, only "good" strategies remained.  I like the concept of Walk Forward, but I noticed all my best strategies passed Walk Forward. 100%.  I thought I would save a lot of time by not doing this, at least for the best, smoothest strategies.
--
Monte Carlo validation.  About 87-95% of my "good" strategies were passing.  I thought the performance penalty outweighed the value here.  I might come back to this later after I have amassed a large army of EAs for possible whittling down for refinement.
--
Multi-Market validation. My initial thinking is that I do not like the concept of this very much, especially for M1 charts, and especially for a mixture between Forex and other instruments such as indexes and commodities.

Good stuff! As a newbie myself I would go along with everything you said. At first I thought I would generate the very best strategies so I used Reactor with the optimization, Monte Carlo and walk forward all switched on and of course it took a long time to generate a sizable number of strategies. I dumped that in favor of generating as many strategies as possible. My current favorite is to filter by R-Squared. And I agree about optimization I think the idea of applying a basket for trading isn't helped by over optimization. Do you plan on trading them as individual strategies or in a portfolio? I found that putting 20 into a portfolio seemed about right. I've decided to do an additional filtering after final selection based on good equity curves but just using Monte Carlo and see how that goes. I too don't bother about multi market validation. I doubt whether many would get through the validation process.

Re: Collections compilations results using different "Sort by" settings

Thanks for letting me know you are/someone else is generally finding and thinking what I am, giving me some reassurance that I am not too far off-track.

I, too was doing the full Reactor sequence (after I discovered it and what it does), but after a couple of days of that, seeing the small number of strategies making it through all check-points was very discouraging compared to what I had earlier seen using just the Generator.  I thought it might be better to save those checks as a final sequence for only the very best strategies instead of every strategy, and use that computing time to generate millions of other strategies instead.

My current thinking on Portfolio size is: yes, the top 10 or 20 are probably going to be the best, with the rest maybe only being some degree of OK, if the generator has only been running a few days.  But, seeing the value of Portfolio Stats of 100 strategies combined giving a much higher Return/DD than 20, perhaps running the generator for several weeks or even months might eventually deliver 100 strategies which are all good, so that is my current goal.  Each week of work seems to bring me closer to where I think I need to be to get started.

I tripled my computing power for this project, yet I still need a lot more!  Rumors of a Dual 64-core Threadripper 3990WX computer (128 cores) fill my dreams.  :-)