• Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 8,233

Topic: Multi-Market validator in Express Generator

Hello Traders,

I've started working on a Multi-Market functionality for Express Generator.

Why is it necessary?

I'm using Express Generator on two servers to generate and validate strategies for the Top 10 Robots App.

My current workflow is:
1) Generate strategies for each symbol and period (16 symbols and 4 periods). The Generator exports collections with capacitty of 1000 strategies.
2) Validate the strategies against the Acceptance Criteria (AC). The exported collections are up to 100 strategies.
3) Validate the AC collections with Monte Carlo (MC).
4) Cross-validation - Validate passed MC collections with data from another server sequentially. I validate M15, M30, and M30 on one additional server, and M5 on 5 servers. M5 collections also go to an application called Prop Firm Robots.
5) All new strategies are validated with already collected strategies.

My Acceptance Criteria are rather strict, and only a few strategies can progress through the workflow.

Now I decided to change the approach. Instead of throwing off the strategies at the beginning of the workflow, I want to reduce the AC and to move it into the Application itself.
Instead of strict AC, I want to use a Multi-Market validation, like:
1) Generator
2) Monte Carlo
3) Multi-Market
4) Cross-validation
5) AC in the applications

The problem is that the current implementation of Express Generator makes it easy to do sequential validations:
- generate
- test on the market A
- which pass A, test on market B
- ...

We cannot test strategies on 5 markets and take those that pass 3 markets.

I'm going to solve that problem this week.

Popov's Website

  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 8,233

Re: Multi-Market validator in Express Generator

New Multi-Market script is introduced: .\bin\mm.js

Intended usage:

With a custom settings file:

node .\bin\mm.js --settings mm.ini

With CLI options:

node .\bin\mm.js ^
  --input  Collection-[SERVER]-[SYMBOL]-[PERIOD].json    ^
  --output Collection-[SERVER]-[SYMBOL]-[PERIOD]-MM.json ^
  --server Premium     ^
  --symbol EURUSD      ^
  --period M15         ^
  --test-symbols auto  ^
  --count-of-tests   5 ^
  --min-passed-tests 3

By specifying the symbols manually:

node .\bin\mm.js ^
  --input  Collection-[SERVER]-[SYMBOL]-[PERIOD].json ^
  --output Collection-[SERVER]-[SYMBOL]-[PERIOD].json ^
  --output-replace true ^
  --server Premium      ^
  --symbol EURUSD       ^
  --period M15          ^
  --test-symbols EURUSD GBPUSD AUDUSD USDCHF USDJPY  ^
  --count-of-tests   5  ^
  --min-passed-tests 3

The input and the output formats will be as in the Generator "gen.js".

For example, the upper --output-replace true serves to override the same strategy instead of adding a suffix to the name or using another naming format.


The parameters "server", "symbol", and "period" determine the data origin, symbol information, and placeholders in the input/output templates.

The actual Multi-Market data are specified in the "--test-symbols" parameter.

There are two options:
- "auto" - the program will select symbols automatically
- concrete symbols:  EURUSD GBPUSD AUDUSD USDCHF USDJPY

Popov's Website

  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 8,233

Re: Multi-Market validator in Express Generator

Thee new tool can fetch data and load collections

https://image-holder.forexsb.com/store/ex-gen-multi-market-loading-data-and-collection-thumb.png

It loads an input collection in a similar way as "./bin/node.js".

The parameters ""--server",  "--symbol", "--period", and "--input" serve to determine the input collection. I'm using template variables in the example, but it can be a simple collection file path.

It shows loaded 28 strategies here. I'll make it showing the collection name also.

The Multi-Market markets are set as a combination of "--test-servers", "--test-periods", and "--test-symbols". The program will load a combination of them (Cartesian product ) until it reaches the "--count-of-tests" value.

It loaded 8 markets because I set 4 symbols, 2 servers, and 1 period.

Here is more extreme example:

https://image-holder.forexsb.com/store/ex-gen-multi-market-load-a-lot-of-data-thumb.png

Here I intentionally skip fetching fresher than 20 minutes data. (loaded during the first example).

I mark the Monday complete.

Happy Trading!

Popov's Website

  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 8,233

Re: Multi-Market validator in Express Generator

The Multi-Market shows some signs of life.

Here, I generate 100 strategies.

Generator command:

express-generator> node ./bin/gen.js `
    --server Premium `
    --symbol EURUSD `
    --period M15    `
    --output Coll_[SERVER]_[SYMBOL]_[PERIOD].json

Generator output:

                        ..:: Express Generator v2.69 ::..

    Market : Premium Data EURUSD M15
    From   : 2022-02-11 20:15, To: 2026-02-17 12:00, Bars: 100000
    Spread : 10, Swap long: -21.08, Swap short: 13.08, Commission: 6 USD
    Account: 10000 USD, Leverage: 100, Entry: 0.01 lots

    10358 ┤                         ╭╮    ╭────╮╭─     Net profit      353.77 USD
    10328 ┤                        ╭╯╰─╮╭─╯    ╰╯      Profit per day  0.24   USD
    10298 ┤                     ╭──╯   ╰╯              Profit factor   1.34   
    10269 ┤          ╭──╮  ╭────╯                      Max drawdown    77.19  USD
    10239 ┤         ╭╯  ╰──╯                           Max drawdown    0.75   %
    10209 ┤        ╭╯                                  Return/drawdown 4.58   
    10179 ┤       ╭╯                                   Win / loss      0.70   
    10149 ┤     ╭─╯                                    Max stagnation  414    days
    10119 ┤    ╭╯                                      Max stagnation  28.29  %
    10090 ┤   ╭╯                                       R - squared     64.48  
    10060 ┤  ╭╯                                        Max cons losses 4      
    10030 ┤╭─╯                                         Count of trades 509    
    10000 ┼╯                                           Avrg pos length 171    bars

  - Ascended: 343, Calculated: 17606, Time: 03:00 of 03:00, Coll: 100 

Collection exported: collections/Coll_Premium_Data_EURUSD_M15.json

Then I run the new Multi-Market tool. The count of tests is determined by the combinations of servers, symbols, and periods. (18 in this example).

express-generator> node ./bin/mm.js                   `
    --server Premium --symbol EURUSD --period M15 `
    --input Coll_[SERVER]_[SYMBOL]_[PERIOD].json  `
    --count-of-tests 20                           `
    --test-servers MetaQuotes Premium             `
    --test-periods M5 M15 M30                     `
    --test-symbols EURUSD GBPUSD AUDUSD

It fetches historical data:

         ..:: Express Generator Multi Market v2.69 ::..

Fetched: MetaQuotes AUDUSD M5,  From: 2024-10-11 01:25, To: 2026-02-17 12:30, Bars: 100000
Fetched: MetaQuotes EURUSD M5,  From: 2024-10-10 22:45, To: 2026-02-17 12:30, Bars: 100000
Fetched: MetaQuotes GBPUSD M5,  From: 2024-10-11 01:35, To: 2026-02-17 12:30, Bars: 100000
Fetched: MetaQuotes AUDUSD M15, From: 2022-02-08 04:45, To: 2026-02-17 12:30, Bars: 100000
Fetched: MetaQuotes EURUSD M15, From: 2022-02-08 04:00, To: 2026-02-17 12:30, Bars: 100000
Fetched: MetaQuotes GBPUSD M15, From: 2022-02-08 05:15, To: 2026-02-17 12:30, Bars: 100000
Fetched: MetaQuotes AUDUSD M30, From: 2018-01-30 08:30, To: 2026-02-17 12:30, Bars: 100000
Fetched: MetaQuotes EURUSD M30, From: 2018-01-30 07:00, To: 2026-02-17 12:30, Bars: 100000
Fetched: MetaQuotes GBPUSD M30, From: 2018-01-30 09:00, To: 2026-02-17 12:30, Bars: 100000
Fetched: Premium Data AUDUSD M5,  From: 2024-10-14 17:30, To: 2026-02-17 12:00, Bars: 100000
Fetched: Premium Data EURUSD M5,  From: 2024-10-14 20:25, To: 2026-02-17 12:00, Bars: 100000
Fetched: Premium Data GBPUSD M5,  From: 2024-10-14 09:30, To: 2026-02-17 12:00, Bars: 100000
Fetched: Premium Data AUDUSD M15, From: 2022-02-11 18:30, To: 2026-02-17 12:00, Bars: 100000
Fetched: Premium Data EURUSD M15, From: 2022-02-11 20:15, To: 2026-02-17 12:00, Bars: 100000
Fetched: Premium Data GBPUSD M15, From: 2022-02-11 08:30, To: 2026-02-17 12:00, Bars: 100000
Fetched: Premium Data AUDUSD M30, From: 2018-02-08 07:00, To: 2026-02-17 12:00, Bars: 100000
Fetched: Premium Data EURUSD M30, From: 2018-02-08 10:30, To: 2026-02-17 12:00, Bars: 100000
Fetched: Premium Data GBPUSD M30, From: 2018-02-07 21:00, To: 2026-02-17 12:00, Bars: 100000

Then it calculates the input collection against each market. It shows progress and stats for each calculations.

...
Market : Premium Data GBPUSD M5
From   : 2024-10-14 09:30, To: 2026-02-17 12:00, Bars: 100000
Spread : 10, Swap long: -9.96, Swap short: -0.28, Commission: 6 USD
Account: 10000 USD, Leverage: 100, Entry: 0.01 lots

 - Passed: 7, Calculated: 100 of 100 

Market : Premium Data AUDUSD M15
From   : 2022-02-11 18:30, To: 2026-02-17 12:00, Bars: 100000
Spread : 10, Swap long: -11.18, Swap short: 2.52, Commission: 6 USD
Account: 10000 USD, Leverage: 100, Entry: 0.01 lots

 - Passed: 14, Calculated: 100 of 100 

Market : Premium Data EURUSD M15
From   : 2022-02-11 20:15, To: 2026-02-17 12:00, Bars: 100000
Spread : 10, Swap long: -21.08, Swap short: 13.08, Commission: 6 USD
Account: 10000 USD, Leverage: 100, Entry: 0.01 lots

 - Passed: 100, Calculated: 100 of 100 

Market : Premium Data GBPUSD M15
From   : 2022-02-11 08:30, To: 2026-02-17 12:00, Bars: 100000
Spread : 10, Swap long: -9.96, Swap short: -0.28, Commission: 6 USD
Account: 10000 USD, Leverage: 100, Entry: 0.01 lots

 - Passed: 19, Calculated: 100 of 100 
...

The Multi-Market application shows stats when it finishes.

Multi-Market validator ready!
  9 of 100 strategies passed  1 of 18 tests
 23 of 100 strategies passed  2 of 18 tests
 20 of 100 strategies passed  3 of 18 tests
 12 of 100 strategies passed  4 of 18 tests
 16 of 100 strategies passed  5 of 18 tests
  6 of 100 strategies passed  6 of 18 tests
  6 of 100 strategies passed  7 of 18 tests
  3 of 100 strategies passed  8 of 18 tests
  4 of 100 strategies passed 10 of 18 tests
  1 of 100 strategies passed 12 of 18 tests

ToDo:
- make it possible to select the test symbols automatically
- make it export the output collection
- inspect the code for issues and best practices
- test for proper work on my workflow
- prepare a release version and publish

Popov's Website

  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 8,233

Re: Multi-Market validator in Express Generator

The Multi-Market tool can automatically set testing symbols and export a collection of the strategies that passed.

Here is a demo command:

express-generator> node .\bin\mm.js                  ^
    --server Premium                                 ^
    --symbol EURUSD                                  ^
    --period M15                                     ^
    --max-data-bars 100000                           ^
    --input Coll_[SERVER]_[SYMBOL]_[PERIOD].json     ^
    --test-servers Premium                           ^
    --test-periods M15                               ^
    --test-symbols auto                              ^
    --count-of-tests 8                               ^
    --output Coll_[SERVER]_[SYMBOL]_[PERIOD]_MM.json ^
    --min-passed-tests 5

With the following parameters "--symbol EURUSD", "--test-symbols auto", and "--count-of-tests 8", Multi-Market will set 8 symbols automatically.

         ..:: Express Generator Multi Market v2.69 ::..

Fetched: Premium Data EURUSD M15, From: 2022-02-14 14:15, To: 2026-02-18 06:00, Bars: 100000
Fetched: Premium Data GBPUSD M15, From: 2022-02-14 02:30, To: 2026-02-18 06:00, Bars: 100000
Fetched: Premium Data AUDUSD M15, From: 2022-02-14 12:30, To: 2026-02-18 06:00, Bars: 100000
Fetched: Premium Data NZDUSD M15, From: 2022-02-11 20:45, To: 2026-02-18 06:00, Bars: 100000
Fetched: Premium Data EURGBP M15, From: 2022-02-14 13:30, To: 2026-02-18 06:00, Bars: 100000
Fetched: Premium Data EURAUD M15, From: 2022-02-13 23:15, To: 2026-02-18 06:00, Bars: 100000
Fetched: Premium Data EURCHF M15, From: 2022-02-13 23:15, To: 2026-02-18 06:00, Bars: 100000
Fetched: Premium Data EURJPY M15, From: 2022-02-14 06:45, To: 2026-02-18 06:00, Bars: 100000

Multi-Market fetch ready!

When completed, it will export a collection of strategies that pass at least 5 of 8 tests.

https://image-holder.forexsb.com/store/exgen-mm-export-collection_mm-thumb.png

Popov's Website

  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 8,233

Re: Multi-Market validator in Express Generator

--test-symbols auto

The "auto" option is useful when we use the Multi-Market validator in an automated workflow with input strategies with various symbols.

We can use --test-symbols auto only when the main symbol of the input collection is a valid Forex symbol.

The program selects the test-symbols in the following order:
- symbols with the same quoted (underlying) currency "USD" for EURUSD
- symbols with the same base currency "EUR" for EURUSD
- symbols with quoted currency equal to the base currency of the main symbol
- symbols with base currency equal to the quoted currency of the main symbol
- some of the rest symbols

The symbols are selected from the list of symbols provided by the server.

The count of the symbols is determined by the count of the combinations of test-servers and test-periods, and the required count-of-tests.

Popov's Website