geektrader wrote:

Hi Hannah,

sorry for the long delay in getting back, I´ve been extremely busy ever since the new year started. First off: thanks for the kind words, I much appreciate them. However, I am no magician either, although I have some great EA´s, it does not mean that they cannot and will not fail - I am ready to remove them the day they exceed their historical drawdown / "equity curve behavior". Markets are ever changing everyday and even perfectly looking systems might stop working "tomorrow" - just go with the flow and have no hard feelings to get rid of systems that you´ve got a bad stomach feeling about.

Apart from having good systems, you have to be on top of things for the technically execution too. That means having the right broker with the lowest "total costs" (spread + commission + average slippage) and of course you should be able to withdraw your money should you make profits (you will be surprised how many brokers did NOT pay my profits in the past with shady reasons and all the work did not pay off because they simply denied withdrawal and deleted the profits. Yea, Forex is a freaking HARD business, if not the hardest. First you have to beat the markets (extremely difficult, but EA Studio will help), then you have to beat the broker, so to say. And all that in an enviroment where 80%+ of retails traders lose (you can easily see that this is no myth because all the EU brokers now have to list the losing percentage of accounts on their main-site right away). Would I go 10 years back and have the choice again if "to do" Forex with the knowledge I have now, I would NOT do it again at all - there are absolutely easier ways to make money with a guarantee to actually make some, because in Forex you can spend years and only loose money instead, hehe. But surely there is a passion for creating automated systems, data analytics and coding which keep me in the game. Enough rant, I think everyone knows what they´ll get if they decide to trade Forex now :-)

As you´ve been asking for a benchmark to go for, here are stats of my "top" systems that have been created with EA Studio (maybe it helps anything so you know what to look for for your own ones). All of them are trading 1 lot fixed.

https://drive.google.com/open?id=1wY6Cm … 3IkHUYbTWS

Cheers,

Geektrader

Hi Geektrader thanks for sharing your reports. May I know how did you use Quant Analyzer to generate such a report for strategies that were created with EA Studio?

Quant Analyzer supports MetaTrader 4 and 5 reports for importing.

https://strategyquant.com/doc/quantanal … tanalyzer/

If you are using an unsupported format it is possible to create your own format using the General CSV Loader as mentioned at the bottom of that link.

Hi Hannah,

sorry for the long delay in getting back, I´ve been extremely busy ever since the new year started. First off: thanks for the kind words, I much appreciate them. However, I am no magician either, although I have some great EA´s, it does not mean that they cannot and will not fail - I am ready to remove them the day they exceed their historical drawdown / "equity curve behavior". Markets are ever changing everyday and even perfectly looking systems might stop working "tomorrow" - just go with the flow and have no hard feelings to get rid of systems that you´ve got a bad stomach feeling about.

Apart from having good systems, you have to be on top of things for the technically execution too. That means having the right broker with the lowest "total costs" (spread + commission + average slippage) and of course you should be able to withdraw your money should you make profits (you will be surprised how many brokers did NOT pay my profits in the past with shady reasons and all the work did not pay off because they simply denied withdrawal and deleted the profits. Yea, Forex is a freaking HARD business, if not the hardest. First you have to beat the markets (extremely difficult, but EA Studio will help), then you have to beat the broker, so to say. And all that in an enviroment where 80%+ of retails traders lose (you can easily see that this is no myth because all the EU brokers now have to list the losing percentage of accounts on their main-site right away). Would I go 10 years back and have the choice again if "to do" Forex with the knowledge I have now, I would NOT do it again at all - there are absolutely easier ways to make money with a guarantee to actually make some, because in Forex you can spend years and only loose money instead, hehe. But surely there is a passion for creating automated systems, data analytics and coding which keep me in the game. Enough rant, I think everyone knows what they´ll get if they decide to trade Forex now :-)

As you´ve been asking for a benchmark to go for, here are stats of my "top" systems that have been created with EA Studio (maybe it helps anything so you know what to look for for your own ones). All of them are trading 1 lot fixed.

https://drive.google.com/open?id=1wY6Cm … 3IkHUYbTWS

Cheers,

Geektrader

Hi Geektrader thanks for sharing your reports. May I know how did you use Quant Analyzer to generate such a report for strategies that were created with EA Studio?

I've always remembered this post as an inspirational reference, I decided to search it again but there are no news since some months ago. Would be so nice to have an update or even new success stories from other members. Unfortunatelly I can't post mine because I haven't any.
What about geektrader? He used to be very active in this forum. Did his strategies continued with their curves or folded?

I find it is very important to diversify across different markets when adopting trend following techniques as there can be long periods in any single market where trends rarely present themselves. If you are restricted to a few instruments you can find yourself spinning your wheels uselessly with building drawdowns.

Risk weighting is incredibly important especially since EA studio operates off standard lot sizes for market selections. Like you I use historic max drawdown as a basis to adjust position sizes for each strategy to risk weight them.

The chart below provides an example of a compiled portfolio that does not use a risk weighting approach (a raw portfolio). Notice how 3 instruments significantly bias the results of the overall collection of 14 instruments.

Now here is a risk weighted solution applied to the same 14 instruments using the drawdown as a basis to adjust position sizes.

Risk weighting produces a significant boost in performance result for the total portfolio.

Here is the raw portfolio result.

...and here is the risk weighted performance result.

The difference is significant.

Mike I can now pretty well manage my drawdown profile through adjusting position sizes at the global portfolio level. My most recent blog on the website gives an idea how to do this.

I actually have no long term profit target as I avoid prediction in it's entirety. By focusing on the risk management side of the portfolio, profits are just a symptomatic outcome associated with the fact that markets do tend to trend at points in time. I never know when that will occur....so I just let the portfolios do their thing...as I know that when markets decide to trend I will be riding them. My worst fear is the fear of missing out on a trend....so I diversify as much as I can.

I trade bundles of portfolios that contain many strategies within them......and the more the merrier.

As I risk weight the portfolio, you will find that as a portfolio builds a larger drawdown, the position size is decreased....hence I never actually turn strategies or portfolios off at all. I continually add new portfolios to the collection. It is a never ending process. The only think that changes is that you need to progressively build up your VPS servers and broker live accounts as your portfolios build. At the moment I am operating off about 10 separate VPS accounts each operating off a separate live broker account ....and this is likely to continually grow as time progresses and the workflow processes continue. I use FX Blue to do most of the heavy lifting with live trade data collection which I import into my data analysis tools.

Make sure that if possible you use an account that offers rebates off volume transactions as this can be a nice bonus for high trade activity.

You seem to have a very similar ambition in your trading goals that I do which appear very realistic.

Nice catching up Mike.

Cheers

Rich

I will have a look at your website and have a read through thanks for that!

I don’t primarily trade the M30 I trade M30 and up. Anything lower I find becomes to choppy, all my M30 strategies primarily trade the Euro/US sessions and are flat before rollover to avoid paying swaps as well as I don’t like being in the market longer than I need to be with finding entries on the shorter timeframes. Longer trend based approaches I let ride like yourself.

2 things I think we really agree on:
1: it’s all about diversification, trading across different markets and timeframes allows for smoother results and reduces drawdowns (if done correctly).
2. Risk weighting! I for one risk weight all my strategies however I like to use live test results, and continuously risk weight, I Am playing with different ways to do this currently however the most basic approach in my eyes is sizing my positions based on historic drawdown of a given strategy ie if over the last N months a particular strategy has traded 0.1lot for a 10% drawdown and I assess my ‘max drawdown’ allowance is 5% I’ll half the position size of that particular strategy.
It is constant work and I believe the real secret is not creating profitable strategies but how to manage a portfolio.
How aggressive do you want to be?
Can you handle large drawdowns without sweating?
What is your longer term profit target?
How many strategies are you currently trading?
There are so many questions Which need to be answered by each trader to be profitable and a lot of it is actually finding what sort of trader you are.

I for one am in it for the long haul I like small drawdowns however with that comes small profits, I’m not comfortable with taking on to much risk, id rather make a few % a month than try and double my account in a month and blow up.
I try and play the game of statistics and analysis.
I believe there is only one thing in the market we can control and that’s our risk. Control your risk in terms of position sizing, and diversification and the rest will come.
Trial and error with EA Studio and try every which way to create strategies you can possibly think then gather hard evidence of what works and what doesn’t and it will all start to fall in place.

I like that approach a lot ATS! Also please add to your FF thread! I read over it several times and find it very informative!

Hi Mike

I don't frequent FF anymore as I was just not getting much valuable info from content related to what I do...and there was too much bickering on the forum.....so I let the Data Mining Thread of mine lapse. My workflow has changed a bit since then....but hopefully some of it was helpful.

I now post on my website at https://atstradingsolutions.com/ and involve myself with some of the professional fund managers on Linked-In plus lately Twitter.

I plan to write up some good content on particular approaches to data-mining using EA studio and I have already written some content that may be helpful to those interested in trend following/momentum approaches.

I like your approach you have nutted out Mike. You obviously have spent considerable time with it. Since you are trading on M30 I totally understand where you are coming from in some of your solutions that reduce slippage and execution issues. :-)

I am currently benchmarking my research efforts against the top CTA's in the trend following/momentum space. By closely monitoring their performance, you can see what are reasonable versus unreasonable expectations. Interestingly the approach I now use pretty well mimics their performance.

Below are the long performance results since 1 Jan 2000 for the Trend Following Index that comprises about 60 of the top trend following CTA's in the world. Note the CAGR and Max Drawdown for the 19 year period. This provides an important benchmark that defines whether my test results are pie in the sky or not.

https://i1.wp.com/atstradingsolutions.c … .png?ssl=1

Here are the results of my currently compiled portfolio which spans 13 instruments (FX, Commodities, Indexes, Metals). The equity curve very closely mimics that produced by the trend followers that gives me confidence in it's future robustness. The equity highs and drawdown are almost perfectly aligned with that of the professional CTA's...but still need a bit more diversification to more precisely mimic them.

I am adding to the portfolio as I go and plan to settle on about 30  instruments across asset classes.

https://i0.wp.com/atstradingsolutions.c … .png?ssl=1

You also need to note that the CTA's performance results are net of management and performance fees so the CAGR pre fees would be slightly higher and the Max Draw of the TF Index slightly lower if we were comparing apples for apples.

I plan to scope out the workflow of the processes I currently undertake on my blog so hopefully will get a bit of interest in the methodology and hopefully will be able to share notes with those on a similar path.

Cheers

Rich

I for one have several different approaches and theory’s about data mining strategies,
I have built strategies using over 200,000+ bars of data with 20% OOS with the OOS results not varying more than 5% r squared and balance line stability than IS. Those particular strategies do not trade often however they are looking promising in live testing however there is not enough results to establish a result just as yet.

My latest method is to create a portfolio of 50+ strategies using 10,000 bars of data all IS on M30 timeframe using strict Monte Carlo testing to weed out the over fitted strategies. I have just started testing these in the last two weeks with some good results so far.
This goes against everything I believe should work however I am very mindful of the fact that the market doesn’t care about what I think should work which is why I keep a log and all the settings used to create different portfolios so over time I can come to some sort of conclusion as to what works well and what doesn’t.

On every strategy however I run an entry time indicator as a preset to eliminate the strategy from trading around rollover (midnight for brokers on the international Timezone standard for forex GMT+3 incl daylight savings I believe)
I also put a spread protection in which i refer to my brokers average spread and adjust accordingly.
I also like to turn my strategies off around high impact news events as these can have a huge impact of over all results.

As a newbie to this I'm wondering how to make a success story out of this because I can't predict the future. And that's the problem of course. I just produced some very good EA's running on data from 2003 to 2012. I made a portfolio EA of 20 of the best ones and it shows a very satisfactory profit and a great equity curve. But when I run that same EA on OOS data from 2013 to 2019 it's a flop, a real loss maker. So what am I doing wrong?

Hi Tony

Here is just an idea that I use to make sense of strategies that I data mine using EA studio. It is important to note that I am not after predictive modelling techniques that many favour in using this form of data mining software. If I was, then the workflow process and design logic would be completely different. I am looking for price following techniques as opposed to predictive techniques like mean reverting systems.

I always define a broad design logic within which I then run my workflow process within. This logic is critical as you need to have a good understanding in what market conditions your EA's perform or under-perform in. Remember it is the market condition that determines your profitability in this game....not the performance results of your strategy. If market conditions are favorable then your strategies should also perform. If unfavourable then they should not perform....however most importantly, they should still protect your capital and not lead to excessive drawdowns .

If your EA's design logic performs significantly differently to the market condition, then they are most likely curve fit results which will fall off the cliff when going live.

RESTRICTING THE DESIGN LOGIC

Before I commence my workflow process I first need to establish broad design parameters which restrict the data mining solutions that can be produced.

For example, I prefer momentum/trend following models and therefore ensure that acceptance criteria allow for realistic return/drawdown ratios which I base from industry benchmarks (CTA trend following funds). My return to drawdown ratio over a 10-20 year backtest regime is  therefore realistically set to 0.5.

I also accept a small minimum count of trades as I am only after those models which infrequently trade during high volatility regimes. I also set realistic minimum profit factors (>1.1) that must possess at least a very small edge. You will find that many people suggest a higher sample size is essential....but this is not the case for trend following/momentum models. You actually want to decrease the sample size and avoid over-trading.

I restrict my indicator list to a small class of classic momentum/trend following indicators that I fully understand.

I also use a preset indicator using a long range 200 period SMA filter to ensure that my trades are during more exotic volatile market regimes. Furthermore I tend to operate on the longer term timeframes (D1 and H4) where noise and frictional costs such as slippage is less prevalent.

I also apply a tight stop to each strategy (which I visually determine per instrument using MT4) and a trailing stop condition with no profit targets applied. This ensures that all strategies which are generated are trend following in nature and 'cut losses short and let profits run'. It also ensure that all strategies have positive skew which is important for risk management purposes.

Once these broad design conditions have been set, I then data mine on as much data as I can get my hands on.

DATA MINING SETTINGS

I always test using 30% OOS  and use optimisation across the IS component only, For example let's assume I am data mining to develop strategies for implementation post 1/1/2016. I will therefore select a data range of say 1/1/2000 to 31/12/2015 as my data horizon and allow for 30% OOS in this time series.

I also use a stringent Monte Carlo validation of 90% on (randomise indicator parameters) to eliminate the potential for curve fit results.

I only use multi-market validation when I have multiple instruments with very similar volatility profiles (in terms of pips). Unfortunately I cannot use this feature greatly until Popov includes volatility based position sizing into the selection criteria.

VALIDATION

I run the test on 4 sessions per instrument over about 48 hours to generate a large number of valid strategies. I then pass these into the validator to further compile and eliminate correlated strategies. I then have a check of the equity curves for each strategy and visually eliminate those which appear to have very adverse volatile profiles.

So lets say that I have generated 110 strategies....after validation and a visual check, this might reduce to say 20-30 strats that I am happy with.

MT4 TESTING

I then consolidate these successful strats into a portfolio EA and run this through MT4 on my brokers account and then include local currency treatment and slippage to make things more realistic.

This then generates an equity curve for the portfolio.........and here is the important bit.

You then need to visually map the equity curve result to the market data to see if the design logic of the portfolio makes sense. For example, if your EA's are based on trending/momentum design logic, then you want to ensure that your portfolio is profitable during these periods. If market conditions are not trending, then your portfolio should not be performing well and should be stagnating or incurring drawdowns.

If your portfolio is performing well during non trending market conditions....then it is highly likely to be curve fit.

Here are a few examples of how I map the portfolio equity curve to the market condition. If the mapping process makes sense, then you can be confident that you are not the victim of curve fitting or 'convergent' EA's that are highly likely to fall over in the future.

https://i1.wp.com/atstradingsolutions.c … .png?ssl=1

https://i1.wp.com/atstradingsolutions.c … .png?ssl=1

https://i2.wp.com/atstradingsolutions.c … .png?ssl=1

If you compare performance results post 1/1/2016 against the market data then the performance results make sense and you should be happy with it.

Then you do the same process to a vast collection of other instruments to have portfolios across many asset classes.

Cheers

Rich B

I have the same question as this. I wanted to replicate your process but finding a source of data will be a problem for me.

Thanks,

P.S.: Each of these 4 strategies does (and each new must) have a R-Squared > 80 on at least 10 other pairs (32 years H1 too), so each has to pass 320 years of OOS-data on 10 different pairs (at least!) and on top has to pass that OOS test with a very good R-Squared value and with the exact same indicator parameters as the main symbol (no further optimizations). So each of these 4 strategies does exploit an edge that exists on almost any pair for the last 32 years, and that´s what makes them so stable.

Hi geektrader,

what youre doing is great! thank you so much for sharing your work, have been on a similar mass data path but abondoned it a while ago...:)!

1) Just a quick question about the data... there are quite a few providers of data out there, from where exactly are you buying the data from? i see its from 1987... would be really kind if you shared the source. maybe youve done some homework on the quality of the data?, and i know just how many issues there are with historical data etc.. (whats the quality of the data like or do you not worry too much about it when there is so much ?

2) Another question is that when you actually test on at least 10 pairs? does that mean you test on a lot more pairs i.e. 15... do you use any other specific tools to  find which of the pairs out of the 15 are actually good >80? 

thank you,

GM

Good luck to everyone and thanks again for the kind words :-)

P.S.: And of course big thanks to Mr. Popov, I have to say that, although other platforms have much more options, the actual systems that EA Studio finds surpass pretty much any of the "more complicated" platforms I am using.

"The key to ensure that you have a positive mathematical expectation in the future is to not restrict your system's degrees of freedom.

This is accomplished not only by eliminating, or at least minimizing, the number of optimizable parameters, but also by eliminating, or at least minimizing, as many of the system rules as possible.

Every parameter you add, every rule you add, every little adjustment and qualification you add to your system diminishes its degrees of freedom.

Ideally, you will have a system that is very primitive and simple, and that continually grinds out marginal profits over time in almost all the different markets.

Again, it is important that you realize that it really doesn't matter how profitable the system is [by itself], so long as it is profitable. The money you will make trading will be made by how effective the money management you employ is.

The trading system is simply a vehicle to give you a positive mathematical expectation on which to use money management. Systems that work [show at least a marginal profit] on only one or a few markets, or have different rules or parameters for different markets, probably won't work real-time for very long..."

Hi Guys,

thanks for the kind words :-)

Congrats GT. Great work mate and very nice workflow. You had my head nodding off my shoulders with your detailed account of your processes.

All the best :-)

Rich