1 (edited by mentosan 2021-02-19 16:52:48)

Topic: Premium Data + 2H = Dukascopy Data

Hello people smile,

I compared the Premium Data history with Ducaskopy Data and I observed that there are minimal difference between bars values and anyway the Premium Data history is 2H earlier shifted. This difference make also big differences between EA Studio and backstesting into MT4.
Into attachement you cand see the difference between 2 bars.

How do you handle this?



Please help !

and thanks!

Post's attachments

1.jpg 207.97 kb, file has never been downloaded. 

You don't have the permssions to download the attachments of this post.

Re: Premium Data + 2H = Dukascopy Data

We work on adding a "Time zone" option in EA Studio. It will be available within two weeks.
You may download and import data from the Data download app with the correct time zone for now.

3 (edited by mentosan 2021-02-19 17:37:19)

Re: Premium Data + 2H = Dukascopy Data

But why is a 2H difference compared to Dukascopy? also the bars values are a little bit different...

If I compare the MetaTrader-demo with JFD, there is no difference.

Is it possible that the Premium Data is from a history source different from Dukascopy?

Re: Premium Data + 2H = Dukascopy Data

The Premium Data in EA Studio and the Historical Data App takes data from DukasCopy with UTC time zone.
This is 100% sure.

Please check the Settings / Timezone option in the App. It is UTC by default, but you may have changed the option.

How do you compare the data with DukasCopy? Are you sure the other data are UTC?

I have more than 100 tests that examine different aspects of the data collection service including the timezone.
All tests pass. A run them for you now.


> date
Fri 19 Feb 2021 09:59:16 PM UTC

> premium-data-feed@1.0.0 test
> micro-tester

Found: 10 test files in: /home/forexsb/apps/data-service/test

1) Run test file: bar-helper.test.js

BarHelper
    cloneBar(bar)
         returns a similar bar
    barToString(bar, priceScale, volumeScale)
         converts a Light Bar to a tab separated line of text
    barsToCsv(bars, priceScale, volumeScale)
         Bar one is correct.
         Bar two is correct.
         Bar three is correct.
    concatBars(baseBars, newBars)
         If the newSet is empty, return the baseSet
         If the bars does not overlap, concat sets.
         If the bars overlap, result 3 bars.
         If the bars overlap, use new set.
         If the bars overlap, result is correct
    cutBars(bars, maxBars)
         When given a higher number, returns the barList
         Cut bars correctly from the beginning
    cutToTime(bars, toTime)
         When given a higher time, returns the barList
         Returns first bar
         Returns correct bars
         When the time is not exact, get from previous bar
    generateLtfBars(sourceBars, periodMinutes)
         Generate M5 bars
         Generate M15 bars
         Generate M30 bars
         Generate H1 bars
         Generate H1 bars from M5
         Generate H1 bars from M15
         Generate H1 bars from M30
    getBarDto(bars, symbolSettings, barMeta)
         gets the correct dto
    getFromTime(bars, fromTime)
         When given a higher time, returns an empty list
         Returns last bar
         Returns correct bars
         When the time is not exact, get from next bar
    ticksToBars(ticks)
Passed: 28 of 28, Failed: 0 (5 ms)

2) Run test file: buffer-helper.test.js

BufferHelper
    barsToBuffer(bars) <=> bufferToBars(buffer)
         a round trip conversion gets a similar input
Passed: 1 of 1, Failed: 0 (1 ms)

3) Run test file: data-analyser.test.js

DataAnalyser
    getMissingBarsHours(bars, period)
         when no bars, returns an empty list
         given M1 with no gaps, returns an empty list
         given overnight successive times, returns an empty list
         given one gap, returns the gap
         given four gaps, returns the gaps
         given multiple gaps, returns the gaps
         solves gaps on H4
Passed: 7 of 7, Failed: 0 (0 ms)

4) Run test file: data-series.test.js

DataSeries
    patchBars(bars, patchBars)
         Patch exact intermediate gap match
         Patch smaller than gap
         Patch at the end of the series
         Patch at the beginning of the series
Passed: 4 of 4, Failed: 0 (0 ms)

5) Run test file: dukas-helper.test.js

DukasHelper
    isValidMarketHour(tradingSession, time)
         Given an invalid time, returns false
         Given a valid time, returns true
    getWantedFileTimes(tradingSession, fromTime, toTime)
         Includes fromTime and excludes toTime
         It excludes the out of market hours
Passed: 4 of 4, Failed: 0 (0 ms)

6) Run test file: loop-helper-2.test.js

7) Run test file: loop-helper.test.js

LoopHelper - provide unblocking execution

8) Run test file: lzma.test.js

9) Run test file: time-helper.test.js

TimeHelper
    ceilHour(time)
         does not change an round hour
         rounds time to the next round hour
    floorDay(time)
         does not change a round day
         rounds time to the same round day
    floorHour(time)
         does not change an round hour
         rounds time to the same round hour
    getDay(date)
         gets a day of month text
         gets a day of month text with a trailing zero
    getHour(date)
         gets an hour text
         gets an hour text with a trailing zero
    getMonth(date)
         gets a month text (starting from 0)
         gets a month text with a trailing zero (starting from 0)
    getMonthHumanized(date)
         gets a month text
         gets a month text with a trailing zero
    getYear(date)
         gets a year text
    lightTimeToTime(lightTime)
         gets the time in minutes from the millennium
    lightTimeToTimeText(lightTime)
         gets the date-time text
    timeToLightTime(time)
         gets lightTime - minutes from the millennium
    timeToText(time)
         gets the date-time text
Passed: 19 of 19, Failed: 0 (1 ms)

10) Run test file: trading-session.test.js
     Called sequentially: 1
     Called async: 1
     Called sequentially: 2
     Called async: 2
     Called sequentially: 3
     Called async: 3
Passed: 6 of 6, Failed: 0 (6 ms)

LZMA
    lzma.decompress(buffer, callback)
         There is a tick
         Time is correct
         Bit is correct
         Volume is correct
Passed: 4 of 4, Failed: 0 (0 ms)

loop-helper
    runLoopLoop(worker, aArgs, bArgs, callback
         is called for all args
Passed: 1 of 1, Failed: 0 (1 ms)
Loaded M30, bars: 168911
AUDCAD data loaded: 235.194ms

Trading session match
     AUDCAD
Passed: 1 of 1, Failed: 0 (58 ms)
Loaded M30, bars: 176278
AUDCHF data loaded: 36.57ms

Trading session match
     AUDCHF
Passed: 1 of 1, Failed: 0 (53 ms)
Loaded M30, bars: 176522
AUDJPY data loaded: 72.258ms

Trading session match
     AUDJPY
Passed: 1 of 1, Failed: 0 (46 ms)
Loaded M30, bars: 175968
AUDNZD data loaded: 32.463ms

Trading session match
     AUDNZD
Passed: 1 of 1, Failed: 0 (49 ms)
Loaded M30, bars: 174639
AUDUSD data loaded: 33.888ms

Trading session match
     AUDUSD
Passed: 1 of 1, Failed: 0 (43 ms)
Loaded M30, bars: 175680
CADJPY data loaded: 25.67ms

Trading session match
     CADJPY
Passed: 1 of 1, Failed: 0 (87 ms)
Loaded M30, bars: 176524
CHFJPY data loaded: 27.751ms

Trading session match
     CHFJPY
Passed: 1 of 1, Failed: 0 (47 ms)
Loaded M30, bars: 168502
EURAUD data loaded: 23.095ms

Trading session match
     EURAUD
Passed: 1 of 1, Failed: 0 (51 ms)
Loaded M30, bars: 175846
EURCAD data loaded: 34.473ms

Trading session match
     EURCAD
Passed: 1 of 1, Failed: 0 (48 ms)
Loaded M30, bars: 176518
EURCHF data loaded: 54.973ms

Trading session match
     EURCHF
Passed: 1 of 1, Failed: 0 (45 ms)
Loaded M30, bars: 176510
EURGBP data loaded: 25.214ms

Trading session match
     EURGBP
Passed: 1 of 1, Failed: 0 (49 ms)
Loaded M30, bars: 175814
EURJPY data loaded: 34.386ms

Trading session match
     EURJPY
Passed: 1 of 1, Failed: 0 (45 ms)
Loaded M30, bars: 176519
EURUSD data loaded: 27.248ms

Trading session match
     EURUSD
Passed: 1 of 1, Failed: 0 (82 ms)
Loaded M30, bars: 176343
GBPAUD data loaded: 26.834ms

Trading session match
     GBPAUD
Passed: 1 of 1, Failed: 0 (50 ms)
Loaded M30, bars: 176512
GBPJPY data loaded: 25.296ms

Trading session match
     GBPJPY
Passed: 1 of 1, Failed: 0 (50 ms)
Loaded M30, bars: 176510
GBPUSD data loaded: 32.565ms

Trading session match
     GBPUSD
Passed: 1 of 1, Failed: 0 (46 ms)
Loaded M30, bars: 176447
NZDUSD data loaded: 25.142ms

Trading session match
     NZDUSD
Passed: 1 of 1, Failed: 0 (86 ms)
Loaded M30, bars: 176506
USDCAD data loaded: 30.066ms

Trading session match
     USDCAD
Passed: 1 of 1, Failed: 0 (44 ms)
Loaded M30, bars: 176523
USDCHF data loaded: 26.721ms

Trading session match
     USDCHF
Passed: 1 of 1, Failed: 0 (51 ms)
Loaded M30, bars: 175814
USDJPY data loaded: 24.008ms

Trading session match
     USDJPY
Passed: 1 of 1, Failed: 0 (49 ms)
Loaded M30, bars: 83172
BRENTCMDUSD data loaded: 13.897ms

Trading session match
     BRENTCMDUSD
Passed: 1 of 1, Failed: 0 (39 ms)
Loaded M30, bars: 118008
XAGUSD data loaded: 15.911ms

Trading session match
     XAGUSD
Passed: 1 of 1, Failed: 0 (38 ms)
Loaded M30, bars: 170965
XAUUSD data loaded: 36.337ms

Trading session match
     XAUUSD
Passed: 1 of 1, Failed: 0 (79 ms)
Loaded M30, bars: 46229
BTCUSD data loaded: 2.706ms

Trading session match
     BTCUSD
Passed: 1 of 1, Failed: 0 (14 ms)
Loaded M30, bars: 39697
ETHUSD data loaded: 7.478ms

Trading session match
     ETHUSD
Passed: 1 of 1, Failed: 0 (8 ms)
Loaded M30, bars: 75659
DEUIDXEUR data loaded: 8.335ms

Trading session match
     DEUIDXEUR
Passed: 1 of 1, Failed: 0 (22 ms)
Loaded M30, bars: 74898
GBRIDXGBP data loaded: 9.691ms

Trading session match
     GBRIDXGBP
Passed: 1 of 1, Failed: 0 (21 ms)
Loaded M30, bars: 76219
USA30IDXUSD data loaded: 10.157ms

Trading session match
     USA30IDXUSD
Passed: 1 of 1, Failed: 0 (21 ms)
Loaded M30, bars: 74948
USA500IDXUSD data loaded: 8.496ms

Trading session match
     USA500IDXUSD
Passed: 1 of 1, Failed: 0 (21 ms)
Loaded M30, bars: 75480
USATECHIDXUSD data loaded: 8.315ms

Trading session match
     USATECHIDXUSD
Passed: 1 of 1, Failed: 0 (22 ms)
Test ready

Re: Premium Data + 2H = Dukascopy Data

Now compared the data:

"Download Historical Forex Data" vs "EA Studio Premium Data" vs "TickStory Lite". We have a complete match:


https://image-holder.forexsb.com/store/premium-data-tick-story-match-thumb.png

6 (edited by mentosan 2021-02-20 11:44:17)

Re: Premium Data + 2H = Dukascopy Data

This is very interesting.

The picture bellow is directly from the Dukascopy Chart EurUsd H1.


Is it possible to be different because I'm looking into a Demo Dukascopy account?

Post's attachments

2021-02-20 11_35_52-855785139_ Dukascopy-demo-1 - Demo Account - [EURUSD,H1].jpg 74.48 kb, file has never been downloaded. 

You don't have the permssions to download the attachments of this post.

Re: Premium Data + 2H = Dukascopy Data

I also checked dukascopy's online chart. Is exactly the same as Premium Data.
That said, the fault is in my MT4. Maybe because it's a demo account. I will clarify with the Dukascopy support department and I will bring the result here as well.

Re: Premium Data + 2H = Dukascopy Data

I have experience that MetaTrader changes the data timezone depending on the timezone of the local machine.
For that reason, I have my trading machines on UTC.

It is also possible for the Demo data to come from a Proxy server with a +2 H timezone.