• Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 8,169

Topic: Update Historical Data

Express Generator can fetch historical data from our Premium Data server.


TITLE Update Data

node .\bin\fetch.js ^
  --fx-rates  true  ^
  --symbols       EURUSD USDJPY GBPUSD USDCHF ^
  --periods       M15 M30 H1 ^
  --max-data-bars 100000 ^
  --time-zone 0

PAUSE

How it works:
--fx-rates  true - sets the fetch command to update the FX exchange rates. They are needed to convert the profit into the account currency.
--symbols   EURUSD USDJPY GBPUSD USDCHF - these are the symbols we will download
--periods   M15 M30 H1 - periods to download for each of the above symbols
--max-data-bars 100000 - the maximum number of bars to download
--time-zone 0 - "fetch" can change the data timezone. We specify the timezone shift in hours. Here 0 means UTC.

https://image-holder.forexsb.com/store/express-generator-fetch-premium-data.png

Post's attachments

update-data.cmd 190 b, 6 downloads since 2022-10-28 

You don't have the permssions to download the attachments of this post.

Popov's Website

  • Registered: 2023-03-21
  • Posts: 5

Re: Update Historical Data

Hi Popov,
Hope you are fine.
I have a question:
I am trading on ICMarkets.
ICMarkets servers are set on UTC+2.
When using the time_zone fetch setting in express generator should I set it to 2 or 0?

  • Registered: 2023-05-05
  • Posts: 16

Re: Update Historical Data

Hi Popov,

Why do the naming structures of the JSON files in the data folder differ from each other? for example:
\express-generator\data\Premium\EURUSD_M30.json
\express-generator\data\Darwinex\EURUSD30.json
\express-generator\data\Eightcap\EURUSD30.json

  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 8,169

Re: Update Historical Data

There is particular historical (and not very logical) reason for that.

The first Forex Strategy Builder I created around 2007 was able to import historical data from MetaTrader 4 in CSV format.
The easiest way to do it was to press Ctrl+ S on chart.

The export files are EURUSD1.CSV, EURUSD5.CSV, ..., EURUSD1440.CSV

FSB Pro uses similar functionality since 2012 (first production release in 2014).
It made it very easy to set a data path directly to the MT's "files" folder and simply to save CSV filers from the charts.

Later, I introduced data files in JSON format for EA Studio (and now for FSB Pro also). The JSON files have the benefit of containing symbol information like spreads, swaps, commissions, count of digits ....

When I start the work on Express Generator, I connected it to the Premium Data feed server.
I was pissed of it one day spending hours to find a bug of loading the data because I was thinking that EURUSD1.json is D1 period, but it was actually M1 (simply because USD ends with D). Then I made the Premium Data feed files like EURUSD_M1.json..

This worked well until I decided to make it possible to import data in JSON format from our broker. Of course, the easiest way was to use the Data Export scripts from EA Studio (because every EAS trader have them already) and also to set the data path to the brokers "files" folder. Obviously, the exported format of EA Studio was EURUSD1.json smile

That's why we will suffer forever.
(Recently tested one of the first releases of FSB from about 2005 and it still works. So that EA Studio and Express Generator will work in that way for many more years ahead)

Popov's Website