forex software

Create and Test Forex Strategies

forex software

Skip to forum content

Forex Software

Create and Test Forex Strategies

You are not logged in. Please login or register.


(Page 7 of 8)

Forex Software → Express Generator → Express Generator - News and Updates

Pages Previous 1 5 6 7 8 Next

You must login or register to post a reply

RSS topic feed

Posts: 151 to 175 of 180

151

Re: Express Generator - News and Updates

> What's the essence behind "programming style"?

Converting from EcmaScript 6 style to EcmaScript 5 style. Then collecting all variable declarations at the top of a function.

For example, replacing 'let' and 'const' with 'var' (deprecated) and collecting at the top of the function:

// Modern
function foo() {
    for (let i = 0; i < 10; i++) {
        const b = i * i;
        ...
    }
}

// Deprecated style
function foo() {
    var i, b;
    for (i = 0; i < 10; i++) {
       b = i * i;
        ...
    }
}

Probably the benefit comes from the JavaScript engine having to create(allocate) a new variable "const b" for every loop.
It only sets a new value to "b" in the second case.

Replacing 'for of'  with 'for i'

// Modern
for (const elem of array) {
    elm ... 
}

// Deprecated style
var i, elem;
for (i = 0; i < array.length; i++) {
    elm = array[i];
    elm ... ;
}

I do all this automatically.
- compile the project source code with TypeScritp compiler to "target" = "ES5".
- re-arranging the code and shortening the variable names with 'uglifyjs --compress --mangle"

The above commands also convert the modern "class" constructs to the "prototype" style.


I changed EA Studio to ES6 two years ago. However, Express Generator has a very different (more conservative) algorithm. For that reason, I decided to test to convert it to ES5.

I made the same test with EA Studio. It shows a 3%-5% performance benefit.
I'm reluctant to publish such deprecated code in the browser for EA Studio, but I'll decide in the following days.

If you want more info, watch the old "Douglas Crockford" videos: https://www.youtube.com/watch?v=JxAXlJE … YUILibrary. They are very funny for geeks.
I watched all of them years ago, and he inspired me to start making backtesting engines for the browser smile))

Trade Safe!

152

Re: Express Generator - News and Updates

Uploaded Express Generator v2.29

New features: Optimizer implemented.

The Optimizer can be used in a workflow with the Generator or when validating imported collections.

New options:

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Optimizer                        ;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

enable_optimizer     = false
numeric_values_steps = 20
optimize_protections = false

Please note that the optimization takes time, and if you optimize collections, you must set a proper working time. For example, you can disable the timer with: `--max-working-minutes 0`.


You can use both the Optimizer and Monte Carlo.

node .\bin\gen.js --symbol EURUSD --period M30 --enable-optimizer true --enable-monte-carlo true

https://image-holder.forexsb.com/store/express-generator-optimization.png

The above screenshots show that 14 strategies were optimized, and two of them passed Monte Carlo and ascended.

Trade Safe!

153

Re: Express Generator - News and Updates

Uploaded Express Generator v2.30

New features:
- added --sort-by option. It sets the sorting criteria for the Collection.
- added --optimize-by option. It sets the metric the Optimizer optimizes.

Both options accept the following parameters:

NetBalance, Profit, ProfitFactor, ReturnToDrawdown, RSquared, Stagnation, WinLossRatio

Example use:

node .\bin\gen.js --symbol EURUSD --period M30 --enable-optimizer true --optimize-by ReturnToDrawdown

Trade Safe!

154

Re: Express Generator - News and Updates

Uploaded Express Generator v2.31

Updates:
  - optimized R-squared formula. The R-squared value of Express Generator is much closer to or completely matches the R-squared value shown in EA Studio.

Trade Safe!

Re: Express Generator - News and Updates

Thanks for the updates Mr. Popov! I didn't find the time yet to test it, but will ASAP.

156

Re: Express Generator - News and Updates

Uploaded Express Generator v2.32

R-squared matches completely between EA Studio and Express Generator.

Trade Safe!

157

Re: Express Generator - News and Updates

Uploaded Express Generator v2.33

This release comes with fixed backtest stats formulas for some edge cases.

Trade Safe!

Re: Express Generator - News and Updates

Thanks for the updates :-) For some reason, it doesn´t fetch 2.33 if running:

node .\bin\fetch.js ^
  --fx-rates true   ^
  --check_for_update true ^
  --automatic_update true ^
  --update_check_interval 1

Results are:

C:\PortablePrograms\ExpressGenerator>!update_fx-rates_and_app.cmd
         ..:: Express Generator Fetch v2.32 ::..
FX rates updated!

C:\PortablePrograms\ExpressGenerator>!update_fx-rates_and_app.cmd
         ..:: Express Generator Fetch v2.32 ::..
FX rates updated!

Thanks.

159

Re: Express Generator - News and Updates

Uploaded Express Generator v2.34

Updates:
- optimizer considers the  Generator's "stop_loss_range_min" and "take_profit_range_min" options.
- formula for calculating correlation in the Collection was slightly modified.

>  For some reason, it doesn´t fetch 2.33 if running:
   Most probably, I forgot to upload it smile

Trade Safe!

160

Re: Express Generator - News and Updates

Uploaded Express Generator v2.35

Fixes:
- enabled all Exponential MA indicators by default.
- fixed creating Collection folders recursively.
- fixed the code for adding suffixes to the collection file to work for thousands of collection files.

Now you can store your collection in subfolders as follows:

node .\bin\gen.js --output .\collections\[SERVER]\[SYMBOL]\[PERIOD]\cool-str.json

// => Collection exported: .\collections\Premium_Data\EURUSD\M30\cool-str_(xx).json

Trade Safe!

161

Re: Express Generator - News and Updates

Uploaded Express Generator v2.36

Fixed issues:
- guaranteed that the indicator period range will not be invalidated from the --max-indicator-period setting.
- fixed the MACD SMA period. The issue was reported here: https://forexsb.com/forum/topic/9443/ea-macd-error-message/ by felix-brenkel.

New features:
- introduced two new options: --use-trade-start and --trade-start. These options set the earliest time the backtester will open a position. It is useful when we want to simulate trading in a short period or want to match the MetaTrader backtest.

- The options --data-start, --data-end, and --trade-start may accept a negative number as an alternative to a date text. The value shows the number of days before the current date. This variant is helpful if we want to have a running data range.

For example, load data from the last year only:

node bin\gen --symbol EURUSD --period M15 --use-data-start true --data-start -365

Load data from the last month, but trade only for the last week to validate a strategy

node bin\gen --symbol EURUSD --period M15 ^
          --use-data-start       true  ^
          --data-start              -30  ^
          --use-trade-start     true   ^
          --trade-start              -7   ^
          --min-count-of-trades  0   ^
          --input .\collections\foo.json


The new settings have the following defaults:

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Data Horizon             ;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

max_data_bars  = 100000

; Use these parameters to set the data range between "data_start" and "data_end".
; Set from where the actual trading starts with "trade_start".
; Text: "2023-05-25 00:00 UTC", "2023-05-25 00:00", "2023-05-25" or "25 May 2023 00:00 UTC"
; Negative number as a number of days before the current date: -365, -30, -7

use_data_start  = false
data_start      = "01 Jan 2020 00:00 UTC"

use_data_end    = false
data_end        = "30 Jun 2023 00:00 UTC"

use_trade_start = false
trade_start     = "01 Jun 2023 00:00 UTC"

Trade safe!

162

Re: Express Generator - News and Updates

Uploaded Express Generator v2.37

Fixes: The MACD indicator matches the one from EA Studio
- SMA period is hardcoded to 9
- SMA period is not considered when calculating the necessary bars for the indicator.

Now MACD generated by Express Generator and MACD generated by EA Studio will be plotted similarly in EA Studio and MT.

163 (edited by Popov 2023-07-01 15:18:28)

Re: Express Generator - News and Updates

Uploaded Express Generator v2.38

New feature:
- Added an --input-match option.

"input-match" is useful when importing particular collections from a directory with multiple collection files.


Match a single phrase

Let's have many collections in the ".\collections" directory and want to load only the ones on EURUSD.
We can do it by using a matching word. Express Generator will load only the collections with "EURUSD" in its path.

node .\bin\gen.js --input .\collections --input-match EURUSD ...


Example for cross-validation: load collections from AUDUSD and validate on NZDUSD

node .\bin\gen.js  ^
          --input .\collections      ^
          --input-match AUDNZD ^
          --symbol NZDUSD        ^
          --period M30                ^
          ...


Match multiple phrases

We can use multiple matching phrases:

node .\bin\gen.js --input .\collections --input-match GBPUSD H1 ...

Express Generator will load only those collections with "GBPUSD" and "H1" in their file path.

When multiple phrases are used, the collection path must contain all of them.


Input match placeholders

Edit:  The --input-match placeholders are changed to [SERVER], [SYMBOL], and [PERIOD] in Express Generator v2.40.

We can also use placeholders such as [SERVER], [SYMBOL], and [PERIOD] to match the corresponding data.

node .\bin\gen.js  ^
          --symbol AUDNZD    ^
          --period H1              ^
          --input .\collections  ^
          --input-match [SYMBOL]  [PERIOD]^
          ...

In the above example, the Express Generator loads only the collections with  "AUDNZD" and "H1" in their file path.


Trade Safe!

Re: Express Generator - News and Updates

Popov wrote:

- Added an "--input-match" option.

This helps so much! It greatly simplifies my scripts when batching multiple symbols or pairs at once, thank you!!!

165

Re: Express Generator - News and Updates

Uploaded Express Generator v2.39

New features:

- when we have --validate-then-generate true, the --input collection may not exist.

This allows scenarios like that:

node .\bin\gen.js ^
  --input       collection-name ^
  --output      collection-name ^
  --validate-then-generate true ^
  --output-replace  true

The above example uses the same collection for input and output. The Generator loads the collection, validates it with newer data, and generates new strategies. When it finishes, it writes the new collection in the same file without adding a suffix.

The purpose of the new feature is to allow the above example to run the first time when the actual collection doesn't exist yet.


- The --server parameter may be an absolute path.

Usually, we use --server to load data from our broker from the "data" subfolder.

node .\bin\gen.js ^
  --server  MyBroker ^
  --symbol  EURUSD   ^
  --period  M30

Now, we can set the absolute path to the MetaTrader's Files folder:

node .\bin\gen.js ^
  --server  C:\Users\Fibonacci\AppData\Roaming\MetaQuotes\Terminal\2127B03FC49A446D047\MQL4\Files ^
  --symbol  EURUSD   ^
  --period  M30

166

Re: Express Generator - News and Updates

Uploaded Express Generator v2.40

This release comes with improved functionality for locating input and output collections.


Breaking change:

  --input-match placeholders are now:  [SERVER], [SYMBOL], and [PERIOD]


New features:

-improved the way the --input and --output compose the corresponding collection path. Now we can give a directory structure, which will be treated as relative to the main "./collections" directory.

-the input and output placeholders are case insensitive.

-we can give a phrase with several placeholders to the --input-match parameter.


Examples:

Express Generator sets a proper ".json" extension and uses the ".collections" directory if necessary.
This works for both --input and --output

--output .\collections\collection.json

// Collection exported: ./collections/collection.json


--output .\collections\collection

// Collection exported: ./collections/collection.json


--output collection

// Collection exported: ./collections/collection.json

Using subdirectories and placeholders for both --input and --output.

--output [server]/[symbol]/[period]/collection

// Collection exported: collections\Premium_Data\EURUSD\M30\collection.json


--input [server]/[symbol]/[period]/collection

// Loads the previously created collection file

Using --input-match with placeholders

--input ./collections --input-match [server] [symbol] [period]

// Match a phrase with placeholders
--input ./collections --input-match [server]_[symbol]_[period]

Trade Safe!

167

Re: Express Generator - News and Updates

Uploaded Express Generator v2.41

New features:

- input and input-match can use the [YEAR], [MONTH], and [DAY] placeholders.
   This feature allows loading or matching collections by the historical data's time.

- the [YEAR], [MONTH], and [DAY] placeholders are replaced with the actual time of update of the historical data. This time is not affected by the Data Horizon or OOS parameters.

- added a max-collected-strategies parameter. It can stop the Generator when the Collection reaches a certain size.

- the home directory of Express Generator is cleaned. The README has links to useful resources.

168

Re: Express Generator - News and Updates

Uploaded Express Generator v2.42

This release improves the work with settings files.

New features:

  - the settings parameter can accept a filename without extension. It will add the ".ini" extension if it is missing.

  - we can provide several setting files. It makes it easy to have separate contextual ".ini" files and to load them when they are needed.

Examples:

Let's have a "eurusd.ini" for specific EURUSD settings.

symbol     = EURUSD
spread     = 10
swap_long  = -1.7
swap_short = -2.3
commission = 4

We can have similar files for the other currency pairs: "gbpusd.ini", "jpyusd.ini", ...

Then, we can have a separate file for the generator workflow - "gen.ini".

max_working_minutes = 30
collection_capacity = 300
min_profit_factor   = 1.4

Now it is easy to run several instances:

start node bin\gen --settings eurusd gen --period M15
start node bin\gen --settings eurusd gen --period M30

start node bin\gen --settings gbpusd gen --period M15
start node bin\gen --settings gbpusd gen --period M30

Trade Safe!

169

Re: Express Generator - News and Updates

Uploaded Express Generator v2.43

Fixes:

- the Data Horizon parameters (data-start, data-end, trade-start) use the data update time when they are given as numbers. The Express Generator is more consistent now. It will provide the same results when calculating collections on older data.

- minor fixes of the "fetch" when downloading Premium Data. Now the "point" is shown as 0.00001 instead of 0.00000099999 in the json files smile

- some refactoring and cleanup of the gen.js file. I'm trying to make the infrastructure files as clean and simple as possible because I expect it to be possible for the users to modify them for their own needs.

Trade Safe!

170

Re: Express Generator - News and Updates

Uploaded Express Generator v2.44

- prevented a crash when Express Generator runs without a visible terminal window. This is helpful when we automate it to run as a background process. I'll make such an example soon.

- Added alternative settings option "max-working-time". It acts as "max-working-minutes". This is because I frequently spell it that way, which has caused errors.

- the "init" option accepts an absolute path to a collection. This works best to load a collection from another folder or drive.

- fixed the way Express Generator loads data files. Now it loads files exported from the newest MetaTrader 5 without problems. It appeared that MT5 saves the files encoded as UTF-16 LE (it was UTF-8 before).

Trade Safe!

171

Re: Express Generator - News and Updates

Uploaded Express Generator v2.45

This version successfully loads data files from MetaTrader 5 in cases when the Broker's name is written in non-English characters. Something like "Сбогом и Благодаря за Рибата".

Trade Safe!

172

Re: Express Generator - News and Updates

Fixed wrong decimal digits in CADCHF's symbol info.

Please update the local symbol info with the following command:

node .\bin\fetch.js --symbol-info true

Trade Safe!

173

Re: Express Generator - News and Updates

Uploaded Express Generator v2.46

Fixes:
- updated endpoints for loading forex rates, symbol info, and Premium data.
- code cleanup.
- updated the symbol-info file with the fixed CADCHF price scaling.

Trade Safe!

174

Re: Express Generator - News and Updates

Uploaded Express Generator v2.47

- Express Generator can backtest startegies wiht a real spread. The spread provided as a settings parameter is used as a minimal Spread.
- "fetch" is connected to the new Premium Data feed. It comes with spread information for each bar, realistic swaps, and a preset commission of $6 per lot per close.
- Each strategy in the collection has information on when it was last modified. It can be used in new features.
- Introduced "forward" testing.
- the strategies come with a unique "strategy hash" number. It is used to ensure no strategy is pushed to the Collection twice. It can later be used for calculating a permanent Magic number.
- Added new "silent" mode for both "gen" and "fetch".
- The Level of the "Moving Average of Oscillator" indicator can be negative.

New options:

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Forward testing          ;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

; Validate strategies on new unseen data.

use_forward_testing = false
preload_data_bars   = 0

As mentioned before, the strategies generated with Express Generator have a record of the time they were modified.
It allows us to accomplish "forward" testing on new data.

We can calculate the strategies on newer data if we have old strategies or use the "data-end" option with the "use_forward_testing" option.

Let's have strategies generated with:

use_data_end    = false
data_end        = -30

These strategies are generated with data up to 30 days back.

Now we can set:

use_forward_testing = true
preload_data_bars   = 500

If we load the previous collection, the generator will calculate the strategies only for the last 30 days. It is very useful to have a running OOS validation.
The "preload_data_bars   = 500" forces the Generator to preload 500 bars before the OOS data. It is necessary to have enough data to preheat the indicators. If this option is not set, the Generator will preload 300 bars.


New "silent" mode:

; Prevent showing output in the console (except errors)
silent = false

The "silent" option is available for "gen" and "fetch'. When it is set to "true", the program will not output messages. It will output only if an error occurs.

The "silent" mode is useful for automated workflow because we can redirect the output of our scripts to an "error.log" file.

This is a very big update, so issues are possible, Please report any misbehaviour.

Trade Safe!

175

Re: Express Generator - News and Updates

Uploaded Express Generator v2.48

This release comes with an updated symbol-info.json file.
It fixes an issue with the DAX 30 (DEUIDXEUR) symbol, which was parsed wrongly.

Posts: 151 to 175 of 180

Pages Previous 1 5 6 7 8 Next

You must login or register to post a reply

Forex Software → Express Generator → Express Generator - News and Updates

Similar topics in this forum