1 (edited by mentosan 2019-03-18 20:03:07)

Topic: Broker with best data History

Hi,

I opened an account by Forextime (FXTM) and I tried to use their history in order to generate strategies with EA Studio.
The problem is that there is a H4  gap of 4 years (10.2013-11.2017). I already tried everything, I also contacted support ...They gave me a blind copy-paste answer  without any connection to  my problem.

Now, I have to change them and the question is, which broker has an accuracy data history? Pls help

Re: Broker with best data History

hey i use 99% Quality data from dukascopy . you can download with tickstory

Re: Broker with best data History

yes I use now also Tickstory, unfortunately the import into the MT4 is still too complicated, since always all data must be completely transferred. Hope that will eventually work as an update

4 (edited by ats118765 2019-03-19 07:04:41)

Re: Broker with best data History

rantampla wrote:

yes I use now also Tickstory, unfortunately the import into the MT4 is still too complicated, since always all data must be completely transferred. Hope that will eventually work as an update

I personally think it is worth considering paying for Tick Data Suite https://eareview.net/tick-data-suite that simplifies the process after installation.

All you then have to do is map the software to your MT4 installation and you are good to go. You simply select 'Tick Data Suite' and ensure  your brokers symbols are mapped against Dukascopy's.

I agree with a member here that suggests that it may be beneficial to include Dukascopy tick data in the broker selections to allow for longer range data across all timeframes.

Post's attachments

Tick data Suite.PNG 94.29 kb, 1 downloads since 2019-03-19 

You don't have the permssions to download the attachments of this post.
Diversification and risk-weighted returns is what this game is about

Re: Broker with best data History

Is the dukascopy data compatible with any broker? I understood that the only reliable solution is to have your own broker history data....

6 (edited by ats118765 2019-03-20 06:39:22)

Re: Broker with best data History

mentosan wrote:

Is the dukascopy data compatible with any broker? I understood that the only reliable solution is to have your own broker history data....

I tend to use Dukascopy to confirm my data mining efforts of EA Studio. As I am looking for long range testing I use whatever broker offers the longest range data over a medium range segment with a significant OOS component to develop very simple strategies with an underlying broad logic. Sometimes I use Metaquotes, Sometimes FXOpen and sometimes my personally loaded Pepperstone data.

Following the data-mining efforts I then test collection results on longer range Dukascopy tick data to check for consistent results during the data-mined test period and periods out of sample to that to see if it still holds up.....and then I have a final test across my preferred broker demo accounts before it goes to live incubation with micro-lots for about 6 to 12 months.

I use a portfolio collator to compile and rebalance my live portfolios from all this testing which sifts through thousands of different data-sets.

If you are looking for linear equity curves of mean reversion (negative skew) then you would need to adopt a different technique to this method as you would be more interested in preciseness than robustness hence different broker sources can cause grief in this area. My entire focus in data mining is on robustness as opposed to future predictability.....hence others with different approaches would recommend totally different approaches to this.

Diversification and risk-weighted returns is what this game is about

Re: Broker with best data History

Hello. Can you more say over "Portfolio-Collator"?

Greets, Marcus.

8 (edited by mentosan 2019-03-20 10:30:02)

Re: Broker with best data History

Ats118765, I apreciate your effort to make such a description but my question is a little bit more "human". fit the duckscopy history data bars to the history data of the brokers? Do you have "almost" the same results as with the original data history? I'm talking about big timeframes H4 ...

9 (edited by ats118765 2019-03-20 15:30:35)

Re: Broker with best data History

mentosan wrote:

Ats118765, I apreciate your effort to make such a description but my question is a little bit more "human". fit the duckscopy history data bars to the history data of the brokers? Do you have "almost" the same results as with the original data history? I'm talking about big timeframes H4 ...

Yes. Was that "human" enough? There will be variation amongst the return distributions caused by a vast array of factors such as GMT offset, Variable Spread, Variable SWAP and Slippage etc. that you won't pick up in these test results so I am more concerned about the materiality of the broad variations between return streams. I am not after preciseness. If the variation is significant and departs from the logic of the strategy mapped against the market condition then I drop the strategy. and do not proceed with it.

Mostly these EA's generated by EA studio stand up against a variety of different data sources on the higher timeframes such as H4 and above. Yes there will be differences between broker data sources but they will be very highly correlated in general to give an indication to their overall robustness in the market condition that the data presents to the strategy.

Materiality becomes more obvious as you step down into lower timeframes such as H1 and M30 and below....but I keep away from these timeframes due to this.

I am not after curve fit results in which you cannot apply logic. If the market price is trending and the EA has been configured to be a trending design, then I want to see my equity curve rise. If market conditions are flat and my trend following EA is still performing then I drop it as it defies the design logic.

Diversification and risk-weighted returns is what this game is about

10 (edited by ats118765 2019-03-20 15:40:08)

Re: Broker with best data History

rantampla wrote:

Hello. Can you more say over "Portfolio-Collator"?

Greets, Marcus.

Hi Marcus

When you compile a large collection of strategies across a broad range of markets and timeframes you will find that each strategy possesses a unique risk-weighting. For example Strategy A may have a compound annual growth rate of say 6% and a maximum drawdown of 12%. Strategy B may have a compound annual growth rate of 2% and a Maximum drawdown of 3% etc etc etc.

The variation between the return/risk profile will be significant and this is attributed to the standard positon sizing applied of say 0.1 lot as opposed to the risk characteristics of the strategy itself.

If you simply compile these strategies into a portfolio with no further global treatment then some strategies will dominate the overall performance of the portfolio and some strategies will unduly bias the under-performance as well. You will find that you will be very limited in your ability to scale this up to make the most of your finite capital.

To remove this bias associated with standard position sizing, you need to standardise the risk weighting of each strategy in the portfolio.

To do this you adopt a standard drawdown for each strategy in the collection and apply a multiplier to the CAGR which is then used to as the multiplier for your position sizing.

For example we have 2 separate strategies in a hypothetical portfolio.

Strategy A - Position Size 0.1 Lots CAGR 6% Max Draw 12%
Strategy B - Position Size 0.1 Lots CAGR 2% Max Draw 3%

Assume a 10% Max Draw for both strategies

then

Strategy A = 10/12 * 0.1 Lots = 0.08 Lots CAGR 5% Max Draw 10%
Strategy B = 10/3 x 0.1 Lots = 0.33 Lots CAGR 6.7% Max Draw 10%

Adjust the position size accordingly so now you have equal risk weighted strategies where any original bias from the prior position sizing impact is removed.

This is the first level of treatment.

I have attached a graphic of the difference between a raw portfolio of 50 strategies and a risk weighted one. Have a look at the overall portfolio results of CAGR and Max Draw to see this principle in action.

Thereafter for further portfolio treatment you look at methods to reduce the adverse correlated impacts of the collection. Not all correlation is bad. You only need to take care of unfavourable correlation. It is a lot like leverage. There can be favorable leverage and unfavourable leverage. You need to  compare the equity curves of your different strategies to eliminate the weaknesses at the portfolio level.

Once this is all done, then you can scale up with position sizing at the 'global' portfolio level to bring home the bacon.

Now refer back to the process we just took. You will note that as the drawdowns of different strategies get larger, the position sizing applied to that strategy decreases. This is how you manage a portfolio of strategies without having to make a decision to drop a strategy if it is under-performing. The risk weighting treatment will do that itself. I risk weight every 6 months or so and continuously add strategies to the collection. In a way it is very much like applying the 'theory of natural selection' to your data mined strategies without having to allow for discretionary decision making in portfolio adjustment. Your portfolio progressively builds and its composition favorably adjusts to emerging market conditions. provided that you continuously data mine to include the new data sets.

Post's attachments

RiskWeighted.PNG 398.82 kb, file has never been downloaded. 

You don't have the permssions to download the attachments of this post.
Diversification and risk-weighted returns is what this game is about

Re: Broker with best data History

ats118765, big big thanks for this information.
it is not easy, but it is a new idea for me, i take this and test it with that.
Thank you smile

Re: Broker with best data History

rantampla wrote:

ats118765, big big thanks for this information.
it is not easy, but it is a new idea for me, i take this and test it with that.
Thank you smile

No probs Marcus. It is a tough game but there are some tricks that can be used at the portfolio level to make it an easier one.

Cheers

Rich

Diversification and risk-weighted returns is what this game is about