Topic: Historical Forex Data - Development

Hello Traders,

I'm happy to share that our new History Data Service has already a working prototype.

There is still much to do, but I really hope to be able to make a public beta available within a few days.

I'll post daily updates on my progress.

Trade Safe!

Re: Historical Forex Data - Development

Hello Traders,

I'm glad to share a very early development release of the Historical Forex Data service.

You can test it here: Historical Forex Data



https://image-holder.forexsb.com/store/historical-forex-data-app-thumb.png



It is in an Alpha stage and may work or may not work at all. However, I'll do my best to make it usable within the next few days.

Currently, it composes and export data files, but we have to thoroughly check the formats and the correct data.
The timezone adjustment is not implemented yet.

Please fill free to test it and to share your ideas and opinion.
I have a goal to make it completely ready until the end of March.

Trade Safe!

Re: Historical Forex Data - Development

Nice, testing it now.

Re: Historical Forex Data - Development

Fixed EA Studio JSON format export.

Re: Historical Forex Data - Development

That is great..

Re: Historical Forex Data - Development

nice work popov...it will be nice to change the prefix name..

now it come with FS DukasCopy EURUSD M15

what about when i want to have the old data and update new data so its better to change name prefix.

Re: Historical Forex Data - Development

It is better to organise the data in folders. I found out that after the export, it is a good idea to store the files on another place and to have the Download folder empty. If it is not, the Windows save the new files with a suffix: EURUSD1.csv, EURUSD1 (1).csv, EURUSD (2).csv ...

The data with changed file names work well in EA Studio because it reads the data information from the file content. On the other hand, FSB Pro will not be able to find the files. (Prefix may work for FSB Pro because we can set it in the Symbol settings.)

Re: Historical Forex Data - Development

Hello Traders,

The project has its own page on the website: Historical Forex Data

Changes:
- implemented timezone correction
- fixed MetaTrader CSV export format
- fixed Excel CSV export format
- the file names are made user-friendly. Example: EURUSD_H1.json for EA Studio and EURUSD60.csv for FSB Pro.

Trade Safe!

Re: Historical Forex Data - Development

I was curious about what symbols are the most upload by the EA Studio users. To find out that, I grouped the symbols and sorted by count. There were about 20 different suffixes and prefixes and I grouped them to some extent.

Here are the winners:

EURUSD                   2300 
GBPUSD                   1375 
USDJPY                   1327 
AUDUSD                   869  
USDCAD                   811  
EURJPY                   791  
EURGBP                   719  
USDCHF                   698  
GBPJPY                   694  
NZDUSD                   527  
XAUUSD                   477  
AUDJPY                   427  
AUDCAD                   372  
EURCHF                   352  
EURAUD                   337  
GBPAUD                   311  
CADJPY                   310  
AUDNZD                   303  
GBPCHF                   291  
EURCAD                   283  
BTCUSD                   278  
NZDJPY                   262  
CHFJPY                   241  
AUDCHF                   241  
CADCHF                   228  
GBPCAD                   224  
EURNZD                   221  
GBPNZD                   189  
NZDCAD                   171  
NZDCHF                   169  
XAGUSD                   135

Here is the full list: https://gist.githubusercontent.com/Popo … s-statstxt

I'll set the most used symbols in the data feed service.

Re: Historical Forex Data - Development

it will be also nice to have some indices. dax, dow jones. gold, silver

Re: Historical Forex Data - Development

I'll add the ones that have longer data.

...

I'm also surprised by the fact of how active are our users. There are more than 2000 different copies of EURUSD uploaded at the moment from currently active licensed and trial users.

There are 139972 data files of active users.

Mhm. It is good!

Re: Historical Forex Data - Development

Great work Popov, the data feed is stupidly fast to retrieve data, we appreciate the hard work you are doing!

13 (edited by zezo 2020-01-15 14:06:26)

Re: Historical Forex Data - Development

Hi Popov, i download EUR/JPY data on 1min and 5 min charts but it is whith many gaps. https://i.ibb.co/NFX6njX/New-Bitmap-Image.png

Re: Historical Forex Data - Development

Great work Popov, the data feed is stupidly fast to retrieve data, we appreciate the hard work you are doing!

Thank you! My goal was to make the fastest service for historical data. I hope the traders to find it useful.


Hi Popov, I download EUR/JPY data on 1min and 5 min charts but it is with many gaps.

Thank you for the report! It must be due to some updates or changes of the data collection service.
I'll run the service to recalculate the complete EURJPY series during the night.
I may do that for all the symbols for any case.

Re: Historical Forex Data - Development

Added three new symbols: AUDJPY, AUDNZD, GBPJPY.
EURJPY is rebuilt. Now the data looks good. However, what @zezo reported can be leftovers from his old data. Probably deleting the old data before importing in MT is a good idea. I'm not sure because I have no experience with that.

Now the service looks better. I'll set it composing new data every morning at 03:00 GMT. Later I'll add more markets. Now we have 19. I'll try to make them 30 next month.

Re: Historical Forex Data - Development

I'm experimenting with adding more feature:

Price Chart - to be able to review the data and to check the correct timezone.
Data preview - a data table
Statistics - to see for gaps or missing bars
Settings - to make customization for the exported files.

https://image-holder.forexsb.com/store/historical-forex-data-price-chart-preview-thumb.png

I'll be on a business trip next week and will continue the work when returning back to Bulgaria.

Trade Safe!

Re: Historical Forex Data - Development

I'll start adding settings today.

I think to add:
- filename prefix
- filename suffix
- server name

Please share your ideas.

Re: Historical Forex Data - Development

Popov wrote:

I'll start adding settings today.

I think to add:
- filename prefix
- filename suffix
- server name

Please share your ideas.

Are you referring to the function of saving data folder?

If it is, then maybe it's nice to have a description box so that I can comment and put remarks on the data etc

Re: Historical Forex Data - Development

The data files cannot have comments. It will break the content format. The files itself also cannot have comments.
The only option is to have the comment in the file name.

Re: Historical Forex Data - Development

Hello thanks for this data. Very useful. I download Daily Excel csv but when i try to import this data metatrader cant read that. Its show nothing. Where is the problem.

Re: Historical Forex Data - Development

Try the "MetaTrader CSV" format. It slightly differs from Excel's CSV format.

Please read the MetaTrader data import instruction written in the Historical Forex Data