Awesome, that worked a charm, thank you!

Thanks Rich, I am starting to like that approach more and more. I see the map to market as meaning that we are effectively waiting for the market to come to us, rather than chasing every blip on the chart. Not only is that much less frenetic than finding tightly focussed strategies, we obviously are not optimising every 10 minutes to capture the changing conditions. Since I have been optimising everything, it took a mindset change to actually turn that function off but I was surprised to see the approach still finding strategies.

I just wanted to check something with you, if that's OK, for this approach. I am in the process of building up the library of core strategies. I am looking for 6-8 "core" strategies which utilise either one or two trend-following indicators with an additional entry indicator chosen by the software, (usually) no exit indicator and a wide stop loss which is there for comfort (and could be changed according to ATR later).

It is much simpler to find entries with the single indicators, e.g where the price is above the moving average over a certain period (50, 100, 150, 200) and the exit where the price falls below the SMA for 0.5 or 0.25 of the entry (e.g. entry at 100 SMA; exit at 25 or 50 SMA) etc - that finds loads of strategies. Im happy with the SMA entries and have around 9,000 of these in some standard combinations.

Second case, above 25 ADX over certain periods (7, 14, 21, 28); then ADX over 50 for certain periods and then ADX over 75 for certain periods etc. Works OK but less strats above 75 are available but that's perhaps the lack of trends - I need to test this on alternative currencies to make sure.

I then wanted to find Donchian but its not working out that well. If I choose "bar opens above upper band" for Donchian 200, I get 362 trades from 1971 - but 99% of these are from 1971 to 1985 ... Its the same if I add in MA, or an ADX - all the trades are bunched together.

- Do you think this is normal?
- Should I look for Donchian on a different currency instead?
- Could I ask what settings you use for Donchian here? There's no middle line available on EA so I am using "opens above upper band after opening below" in order to identify the crossover

I dont think this is the data but i cant seem to figure out why Donchian is giving me so much issue

Hi Minch

That's it mate. Don't be fooled by randomness. In data mining we are looking for a bias in the data series as opposed to price action 'form' itself. Form is inherently random...but repeated form (auto-correlation) suggests that a signal may be embedded in it.

For example here is a randomly generated price chart. I know because I generated it randomly.

Here is the same price chart with serial correlation embedded in it.

You cannot distinguish the bias with a low sample size using visual methods. The only way to do this is by plotting one chart against the other (which you cannot do when trading the right hand edge of the chart)

It is the bias we are after....and not the overall form. In the bias lies 'causality'. So trend followers are not simply looking for trending price (this could be random or non-random)....but are hunting for the 'divergent' bias in the price series.

So the way we accommodate this is through trade sample size. Over many trades....the bias becomes visually self-evident. A bit like tossing a biased coin. You need many flips to conclude that there actually is a 'causal' bias in the series.

For example you cannot deduce an edge in the example below from 500 trades.

But you certainly can visually see the edge in action over 10,000 trades.

Market data is asymmetrically oriented where noise is more prevalent on the lower as opposed to the higher timeframes.

Price movement commences at the tick....and until sufficient ticks are generated, you cannot tell whether there is a bias in the underlying market data. That is why trend followers stick to longer timeframes. The principle of destructive and constructive interference of price data dampens the noise and amplifies the signal. You will not be able to discriminate between noise and signal on the lower timeframe....but the monthly chart clearly spells out where the overall direction of bias has been all along.

Mapping to market on the longer term timeframes (eg. D1) is the way we confirm that the strategy is active during major trends where bias is more self-evident:-) 

Nice mate. Just remember that you want value variation over a chosen parameter set. You want a diversified set of values to capture the broadest range of trending condition you can. The future is uncertain....and trends could take a variety of different forms.

You need diversity of stops, trailing stops, entry conditions etc. to capture different trajectories of trending condition (aka bias. The impact of noise around the bias creates many different forms of trend)

Seems fine Minch. I actually only vary value sets on SMA cross rather than ADX. An 'ADX rising condition' is all I use. I am always trying to simplify my parameter sets...... so ADX actually is not regarded as a parameter set in my technique, but rather a momentum contributor present in all models (so long as it is rising).

That is normal mate. Trending periods are clustered with long periods of mean reversion or noise in between. They key to addressing this is by market diversification across asset classes. Usually something is trending somewhere and someplace. The key thing to note is that you want to avoid over-trading so the longer the look-back the lower trade frequency....which is good as it encourages you to only catch trends of significance (where bias is more likely to lurk).

A Donchian break is a far more specific entry signal than a SMA crossover. The SMA crossover can occur anywhere in the price series and is far more frequent....but a Donchian break is specifically configured to the look-back.

Simply try and generate as many trend following strategy variations as possible and keep them all. It is only later at the portfolio compilation phase that you apply a systematic process to compile the best risk-adjusted collection....and you will be very surprised in the selections it chooses from your robust candidates.

All my Donchian break variations performed exceptionally well between 1970 to 2000....and then really slowed down post 2000. Welcome to adaptive markets. They are still really useful to have however but step out to longer look-backs to create variation.

The reason for why I really like them is that they ensure I don't get 'Type 2' errors. Those errors arising from missing an opportunity that turns into a windfall outlier. SMA crossovers frequently get the Type 2 error....while Donchian breakouts theoretically never do :-)

Great stuff Minch :-)

To give you an idea I currently use two core trend following methods and 2 supplemental momentum methods in my trend following arsenal. Ideally you need many more of these core models...but you have to start somewhere :-)

These 2 broad different trend following system designs revolve around two separate entry methods. One uses a momentum breakout method of entry (using the Donchian) and the other a more classic trend following entry method (using SMA crossover technique). Both of these are found in the trend following literature of the professional FM's.

These core methods are the major 'drivers' of long term wealth building.

The supplemental variations are used to simply smooth the volatility of these two core methods by targeting shorter term trend segments and take some of the profit off the table with the very long trending conditions and deal with the unrealised losses that occur when the trend ends. They are not the drivers of profitability but simply used to smooth the curves. Ideally with significant trading capital you would elect to diversify more widely using the core trend following models only......so the supplemental variations are really only helpful if you cannot diversify wide enough in system design.

You will find that the supplemental methods often 'hedge' the longer term models when it counts. For example.....the long term trend following models can build significant unrealised profits before the 'trend ends' and we often find that when a long term method is now at the end of the trending phase and you start seeing some of that great profit now being eaten away ....the supplemental methods are already locked into the 'turn' in a counter-trend direction to the longer term models and building profits. The hedging process at the end of the long term trend helps to smooth the equity curve at the portfolio level when they are all consolidated.

What is critical is the need for 'diversification of core systems and parameter variations'. This makes you far more robust in being able to target a very large array of different trend forms and segments of the trending condition. This diversification gives robustness in an uncertain future where the 'exact form of trend' is not known....and also serves to significantly increase trade frequency at the portfolio level. Each solution has low frequency as that 'required' condition is few and far between....but together as a diversified portfolio you have many trending opportunities.

So in the first stage of my workflow process I am simply focused on finding a vast array of 'robust' solutions that have stood the test of time to see if there is any design weakness in the assumptions (without biasing the result with current costs such as SWAP, spread etc)........and have a sufficient array of robust performers to take me into the next phase of the robustness process where I select the better recent performers in this reduced subset of opportunities that respond to adaptive trending conditions and also have current trading costs embedded in them.

Across 31 markets, I generate currently a total possible 61,515 permutations which equates to 1,905,965 return streams. The very first stage of robustness testing before I take them to multi-market reduces this set from 1,905,965 to  1,338,876 return streams offering positive expectancy (Gross profit basis). Currently approx 70% pass the very first test across 35 years of market data before they are assessed for their multi-market potential to further test whether an edge (bias is present)

Within this passed set lies extensive diversification and positive expectation which is all you need to build powerful robust portfolios. Provided you have a weak edge in the result at 0.01 lots (for EA Studio) or preferably 0.5% trade risk.....then that is all we are initially after. You do not attempt to apply leverage to that 'edge' with position sizing techniques and target desired drawdown thresholds until you are at the portfolio level of processing.

It is like trying to create the strongest basket with individual threads but during the robustness phase you are looking for the strongest threads to start the process of basket-weaving.

Doing it this way will ensure your EAs won't fall off the cliff when you implement them live. You have 31 x 35 years of market data that tells you that this simply is very unlikely and significantly builds your confidence in letting them run unhindered and not be tempted to intervene in the process by knee jerk reactions whenever they enter very natural drawdown phases.

You will also find that the end result over the long term are performance results that match the trend following FM's as opposed to performance results that are unbelievable 'profitable' in backtest.

Hi Minch

I would leave them as dictated by the strategy generator.

Attached is a file of a typical robust Donchian breakout system applied to EURUSD. You will see the settings applied in yellow. This gives you an idea of the broad settings and ranges of these settings that can navigate really long term data sets.

Rank the listing by MAR to see the better performers.

Cheers Matthew. As you will deduce the Trend Following game is played with many diversified systems with each system only contributing a small handful of trades that clearly attack the trending condition. The range of systems is to allow for trend trajectory variation in a particular market yet each system applied to that market has low correlation with each other.

You do this on a market by market basis until you are comprehensively diversified in both systems and markets. This gives you increased trade frequency as a comprehensive portfolio that strictly attack the trending condition and keep away from the noise and mean reverting environment.

An then hey presto......it just works without you have to continuously replace your strategies. Rebalancing is done on an fairly infrequent basis when compared to 'convergent' methods that need to be continuously rebalanced and even then..you don't know the true risk you are holding and whether it 'all of a sudden' heads towards risk of ruin :-)