forex software

Create and Test Forex Strategies

forex software

Skip to forum content

Forex Software

Create and Test Forex Strategies

You are not logged in. Please login or register.


Forex Software → Express Generator → Validate isn't importing previous strategies

Pages 1

You must login or register to post a reply

RSS topic feed

Posts: 8

1 (edited by aaronpriest 2022-12-10 15:01:57)

Topic: Validate isn't importing previous strategies

This used to work, but isn't for me anymore. I just tested to be sure. I generated strategies for 15M on 5 pairs and have 16 to 23 strategies in each .json file under the /collections folder. I ran it again with validate_then_generate = true and input = ./collections/ Each pair will say "Validating strategies: 84, then generating", but none of the strategies that just passed will be added to the new collection; where I might have had 16-23 strategies per pair a few minutes ago, I now have 1-2 max until more new ones get calculated and ascended. I'd expect the previous 16-23 to be immediately validated and added if the criteria and dates haven't changed, and they used to be added but aren't anymore. Also, the lines of loaded: x records from *****.son files are different on each pair, it's not showing the same files for each pair, even though it still says the same number of "Validating strategies: 84", so maybe it just randomly says which files it is loading but still loads them all in the folder regardless? I don't know, but it's a discrepancy I noticed. At any rate, I'm not accumulating strategies anymore, I'm basically starting out from scratch every time I generate now. Often I'll have less than I did before when running again, instead of adding more to my existing collections.

Secondly, I can't combine collections in EAS either. I thought maybe I could just upload 3 or 4 runs of separate .json files, reject duplicated strategies, and combine more strategies that way too, but it doesn't work. If I upload 23 strategies, and then hit upload again on another file of 16 strategies hoping to add more, it just removes the 23 and gives me 16 instead, so I can't combine the two files. They can't all be duplicates because my first run gave me 23 and my second run gave me 16, so why would loading 23 in EAS and then loading 16 more only give me a total of 16 unless it was removing the previous 23 first?

Can you guys test this as well and see what you get? Thanks!

2 (edited by aaronpriest 2022-12-10 15:21:53)

Re: Validate isn't importing previous strategies

UPDATE: So this probably has something to do with in sample and out of sample testing. I just tested without doing any IS/OOS and strategies I just generated were validated and added back before generating new strategies just fine on the second run (although I still can't combine .json files in EAS). I'm not sure why I can't get it to work when doing in sample and out of sample testing though.

Here is an overview of my workflow, maybe I've made a mistake somewhere...

Here's part of my .cmd file where I'm fetching data and then generating in three steps:

REM Fetch Forex Data
node .\bin\fetch.js ^
 --fx-rates true ^
 --symbol %symbol% ^
 --period %period% ^
 --max-data-bars 30000 ^
 --time-zone 3 ^
 --check_for_update true ^
 --automatic_update true

REM Generate In Sample
node .\bin\gen.js ^
 --symbol %symbol% ^
 --period %period% ^
 --data-file ./data/Premium/%symbol%_%period%.json ^
 --settings sample-in.ini

REM Generate Out of Sample
node .\bin\gen.js ^
 --symbol %symbol% ^
 --period %period% ^
 --settings sample-out.ini

REM Generate Full Strategy
node .\bin\gen.js ^
 --symbol %symbol% ^
 --period %period% ^
 --settings sample-full.ini

Here's the relevant part of my sample-in.ini where it should load and validate strategies that already passed all three stages earlier. It's loading strategies from /collections where it should pass any that just passed, but they don't. And then it would overwrite the /sample-in folder.

server = Premium
input = ./collections/
output = ./sample-in/IS_[SYMBOL]_[PERIOD].json
output_replace = true
validate_then_generate = true
data_start_percent = 0
data_end_percent   = 70

Then sample-out.ini:

server = Premium
input = ./sample-in/
output = ./sample-out/OOS_[SYMBOL]_[PERIOD].json
output_replace = true
validate_then_generate = false
data_start_percent = 70
data_end_percent   = 100

And finally sample-full.ini:

server = Premium
input = ./sample-out/
output = ./collections/COL_[SYMBOL]_[PERIOD]_[YEAR]-[MONTH]-[DAY]-[HOUR]-[MINUTE]_str_[COUNT].json
output_replace = false
validate_then_generate = false
data_start_percent = 0
data_end_percent   = 100

Since the last two stages aren't being given a data-file, then they should automatically be validate only, right? Even then, it's the first stage that isn't passing already validated strategies, where if I run an almost identical single stage pass without doing in and out of sample, it validates and passes along previous strategies fine. So there's something I'm doing wrong with in and out of sample, or a bug. Any ideas?

3 (edited by geektrader 2022-12-10 15:26:02)

Re: Validate isn't importing previous strategies

For EAS it's normal behavior. Each Upload of a JSON collection file clears the current collection in EAS. You simply do it via the Validator in EAS instead. Just add all your JSON collections to the Validator and run it on the same history data. Then they will be merged into a new collection. It was always like that by the way.

Re: Validate isn't importing previous strategies

geektrader wrote:

For EAS it's normal behavior. Each Upload of a JSON collection file clears the current collection in EAS. You simply do it via the Validator in EAS instead. Just add all your JSON collections to the Validator and run it on the same history data. Then they will be merged into a new collection. It was always like that by the way.

Ah, yes, that makes sense. It's since switching to Express Generator that I've changed my EAS workflow and forgot that step. Any thoughts on the validation with in and out of sample with Express Generator?

Re: Validate isn't importing previous strategies

I haven't used this feature of EG at all, so just could comment on the EAS part. I am sure Popov will chime in on this though.

6 (edited by aaronpriest 2022-12-10 16:13:39)

Re: Validate isn't importing previous strategies

geektrader wrote:

I haven't used this feature of EG at all, so just could comment on the EAS part. I am sure Popov will chime in on this though.

I might have found a better way honestly. Don't use IS / OOS at all. Generate on one full year and add it to a folder, then run a validation on that folder for the last 30 days, or 7 days, and add those to a different folder. You are over-optimizing of course, but then picking what is working in current market conditions and hoping they continue for another day or two (referring to the other thread "Definition of insanity..."). At any rate, if you leave data_start_percent = 0 and data_end_percent   = 100 then validation works properly, so maybe it's a bug?

Re: Validate isn't importing previous strategies

Hello AaronPriest,

> If I upload 23 strategies, and then hit upload again on another file of 16 strategies hoping to add more, it just removes the 23 and gives me 16 instead,

The Collection was specially designed to remove the current strategies and then to load the new ones.
As GeekTrader said, we use the EA Studio's Validator to join and validate multiple collections.

...

As you properly noted, you can do it also via Express Generator by giving multiple input collections or collection folders.

Please report if you find any misbehaviour.

8 (edited by aaronpriest 2022-12-14 13:38:22)

Re: Validate isn't importing previous strategies

Popov wrote:

Hello AaronPriest,

> If I upload 23 strategies, and then hit upload again on another file of 16 strategies hoping to add more, it just removes the 23 and gives me 16 instead,

The Collection was specially designed to remove the current strategies and then to load the new ones.
As GeekTrader said, we use the EA Studio's Validator to join and validate multiple collections.

...

As you properly noted, you can do it also via Express Generator by giving multiple input collections or collection folders.

Please report if you find any misbehaviour.

Yes, that part was answered, but the bug about IS / OOS remains. If you use data_start_percent = 0 and data_end_percent = 100, then validation works, but it doesn't seem to if you use partial data for in-sample. None of the previously passed strategies will be validated and added back, even though they passed moments before (if testing this with back to back runs).

Posts: 8

Pages 1

You must login or register to post a reply

Forex Software → Express Generator → Validate isn't importing previous strategies

Similar topics in this forum