====== Fetcher (fetch.js) ======

**fetch.js** downloads historical market data, updates Forex rates, refreshes symbol information, and checks for Express Generator updates.

If you are new to ExGen, begin with [[quick-start|Quick Start]] and return here for full option coverage.

===== What Fetcher Does =====

Fetcher can run one or more of these jobs in a single command:

  * Download Forex rates (**fx_rates**)
  * Download symbol information (**symbol_info**)
  * Download historical market data (**symbol**, **period**)
  * Check and optionally install Express Generator updates (**check_for_update**)

If all jobs are disabled, Fetcher exits with "Nothing to fetch!".

You can fetch data only from servers provided by the developers (Forex Software Ltd).

Currently available servers are: //Premium//, //Dukascopy//, //MetaQuotes//, //Darwinex//, //Eightcap//, //BlackBull//.

Note: Premium data is composed from Dukascopy tick data sources.

===== Quick Start =====

Recommended first run order for new users:

  * Update metadata first (FX rates and symbol info)
  * Fetch historical data
  * Continue with strategy generation in [[gen-js|Generator (gen.js)]]

==== Fetch historical data ====

<code bash>
node ./bin/fetch.js --server MetaQuotes --symbol EURUSD --period M15
</code>

Example output:

<code>
..:: Express Generator Fetch v2.xx ::..
Fetched: MetaQuotes EURUSD M15, From: 2018-02-02 09:45, To: 2026-02-20 21:30, Bars: 200000
Data fetch ready!
</code>

==== Update local metadata (FX rates + symbol info) ====

<code bash>
node ./bin/fetch.js --server MetaQuotes --fx-rates true --symbol-info true
</code>

Example output:

<code>
..:: Express Generator Fetch v2.xx ::..
FX rates fetch ready!
Symbol info fetch ready!
</code>

==== Check for updates only ====

<code bash>
node ./bin/fetch.js --check-for-update true --update-check-interval 0
</code>

===== Settings and Priority =====

Fetcher reads options in this order (lowest to highest priority):

  * Default file: ''./bin/fetch.settings.ini''
  * User settings file(s): ''--settings my-fetch.ini'' (or multiple files)
  * CLI options: for example ''--time-zone -4''

Higher priority overrides lower priority.

When using several files with ''--settings'', each next file overrides keys from the previous one.

Settings names are flexible in INI and CLI:

  * snake_case
  * camelCase
  * PascalCase

Aliases are supported:

  * ''symbol'' and ''symbols''
  * ''period'' and ''periods''
  * ''settings'' and ''setting''

===== Command Reference =====

==== Core options ====

^ Option ^ Type ^ Default ^ Description ^
| ''--server'' | string | ''Premium'' | Data server. Allowed values: Premium, Dukascopy, MetaQuotes, Darwinex, Eightcap, BlackBull. |
| ''--symbol'' / ''--symbols'' | string or list | empty | One or more symbols to download, for example ''EURUSD GBPUSD''. |
| ''--period'' / ''--periods'' | string or list | ''M30'' | One or more periods: M1, M5, M15, M30, H1, H4, D1. |
| ''--time-zone'' | integer | ''0'' | Time zone offset in hours where UTC = 0. |
| ''--max-data-bars'' | integer | ''0'' | Maximum bars to keep. Use 0 for all available bars. |
| ''--if-missing-or-older-than'' | integer | ''0'' | Re-download data only if file is missing or older than N minutes. 0 disables this rule. |
| ''--silent'' | boolean | ''false'' | Suppress regular console output (errors are still shown). |

Network behavior note:

  * If a fetch request fails with a transient network error (for example ''socket hang up''), Fetcher retries automatically after 3 seconds, up to 2 retries.

==== Metadata options ====

^ Option ^ Type ^ Default ^ Description ^
| ''--fx-rates'' | boolean | ''false'' | Downloads ''fx-rates.json'' in ''./res''. |
| ''--symbol-info'' | boolean | ''false'' | Downloads symbol info file ''./res/<server>.json''. |

==== Update options ====

^ Option ^ Type ^ Default ^ Description ^
| ''--check-for-update'' | boolean | ''true'' | Enables update check at startup. |
| ''--automatic-update'' | boolean | ''true'' | Installs update automatically when a newer version is found. |
| ''--update-check-interval'' | integer | ''24'' | Hours between update checks. |

==== Settings file options ====

^ Option ^ Type ^ Description ^
| ''--settings'' | string or list | One or more user INI files that override default settings. |
| ''--setting'' | string or list | Alternative spelling for ''--settings''. |

===== Output and Files =====

  * FX rates are saved in ''./res/fx-rates.json''
  * Symbol info is saved in ''./res/<server>.json''
  * Historical data is saved in the data directory under ''./data''
  * Update check timestamp is stored in ''./bin/res/update-check.json''

===== Related pages =====

  * [[fetch-forex-rates|Forex rates and symbol info]]
  * [[fetch-data|Fetch historical data]]
  * [[fetch-update|Update Express Generator]]
  * [[settings-files|Settings files]]