Hi Steve

Thanks for the post.

I thought I would add my two bobs worth....but I am only a newbie to this forum.

This post is more a 'food for thought' and is based on a few underlying assumptions that members can take with or without a pinch of salt. :-)

I tend to view markets through a prism of 'markets being very efficient' with only occasional alpha being made available through sporadic and generally unpredictable inefficiencies. Under this assumption I tend to throw my ability to predict future performance out with the bath water.

I do not pay much attention to OOS testing as in my opinion this is an outdated form of testing from back in the days when alternative methods of testing for robustness were not available.  I prefer to invest nearly all my time in testing strategies across as broad a range of market conditions as I can find and in the rigorous Monte-Carlo testing of strategy parameters to identify the more robust strategies that can weather the storms of market uncertainty and varying market condition.

The only OOS testing I do is left for my native MT4 broker platform and over a relatively short period to ensure that there are no execution errors or technical issues with the data mined EA that I have generated.

The intent of my search is to simply find those strategies that in unfavourable conditions, do not detrimentally affect the overall portfolio in terms of any undue bias that the singular return stream of the strategy will impose on the collection.

What this means is that all my decisions at the individual strategy level are based on three requirements:
a) That during extended unfavourable conditions, the strategy (return stream) will be limited in it's drawdown impact on the overall portfolio;
b) That the individual return stream has positive  expectancy overall (this is more a statement that the strategy has a good chance of having a slight edge); and
c) That the individual strategy offers correlation benefits to the overall portfolio and assists in reducing overall portfolio volatility.

As far as the overall profitability of the strategy itself, I tend to pay little regard to this feature. That is where I would differ in opinion from your take that 'Too much historical data is harmful'.

There is good data and there is bad data. Data that is not relevant to your native broker platform is meaningless to me and is what I refer to as 'bad data'. What I mean by this is that available alpha is so light in an efficient market but there is so much variability in overall result from slight changes to data such as differences in bar open, variations in spread/swap, quality of historical data....etc etc etc...that the differences can be material to the overall result on the return stream (equity curve) of the strategy. Good data is therefore platform specific data history and as much as you can garner...but once again there will be significant material variations in the result if you are using constant assumptions in your results such as constant spread, swap, slippage etc.

The reason for my preference for as much data history as I can get.... is less about data sample size (although this is generally a useful proxy for robustness)...... but more about ensuring that the strategy has been exposed to as many different market conditions as possible and of differing duration.

If the strategy still has positive expectancy overall (no matter how slight), but will assist in preserving capital, then you can keep this strategy 'turned on' all the time and be available for those unpredictable times when market conditions become favorable and you make hay while the sun shines.

The reason I diversify my portfolio is two fold:
1. Diversification of 'uncorrelated return streams is the major method used to make your portfolio robust and protect your finite capital from impacts of adverse volatility (namely drawdowns); and
2. Diversification of 'uncorrelated return streams' each with slight positive expectancy ensures that there is overall positive momentum in portfolio growth over time at most points in the time-series.

If you select your strategies from those that are performing well over recent times, there is a strong chance that you will be selecting from 'positively correlated' options. This is great when times are good....but exceptionally bad during unfavourable market conditions. Trading correlated return streams in your portfolio exacerbates portfolio volatility as each return stream amplifies positive growth and negative growth as there are all 'in phase'. By adopting this technique you will have to be right on the pulse with knowing when to 'turn on' or 'turn off' those strategies....which in my opinion is a predictive technique and always lagging in nature. There has been a lot of articles written on this topic of 'market timing' but very little success in this area. My preference is for strategies that are turned on all the time. To do this....you need an appreciation of what is 'acceptable limits' of volatility to your strategy so you are not tempted to interfere when entering drawdowns...as they are actually a necessity in the big picture. Benchmarks tend to be the way to go using your long term strategy metrics as the guidelines to assess strategy health during live trading periods.

Also there is a pretty well known 'mean reverting' phenomenon in fund management circles when selecting which funds to allocate your money to. When you have a selection of well established highly diversified long term funds to allocate your money towards, you actually select those funds that are recovering from a drawdown as opposed to those funds that are reaching new equity highs. This is counter to the idea that you should select the current best performing funds from the options available. The reason for this is that these funds have already passed the robustness challenge in that they are highly diversified AND they have stood the test of time over an array of market conditions. Once that challenge is achieved then the optimal times to re-balance your portfolio allocations dependent on when funds are recovering from a drawdown or when they are reaching new high water marks.

Anyway....here is the guide I use when looking at strategies generated by EA studio at the Monte Carlo phase of review.

I tend to view the most likely scenario from the array of simulations generated by Monte Carlo testing as the median solution.  This to me is a more probable result of return stream that would be generated by a particular class of strategy given the high sensitivity of the strategy to minor perturbations in market condition etc etc etc..

Your assumption about the nature of these markets will also influence this decision. For example I believe that the majority portion of any equity curve (which is a derivative signature of market condition) is attributed to market randomness. Under this viewpoint, a single 'best' equity curve is a misnomer and more a result of the way random results compile under optimisation to give you an optimal equity curve. My opinion is that there is only 'weak alpha' available in the markets which relates to the level of 'signals' in the noise and the degree of autocorrelation in the time series. The median choice of return stream from the alternative generations is the most 'probably' outcome of the series and gives a more realistic interpretation of the degree of 'signal in the noise'. The reason for the 'weak alpha statement' is that over long term horizons, the best fund managers in the world can only achieve a certain threshold performance that caps what is pipe dream and what is realistic. A long term CAGR of 15%-20% with commensurate drawdowns of 30-50% is about the best you can get over an extended performance period from the best of the best with heavy duty research teams. I just cannot find audited verifiable results that prove otherwise.

Anyway....back to the Monte Carlo array.......I then calculate an expected return from this median solution to ensure it still has slight positive expectancy and recognise that the optimised return stream is a very unlikely scenario going forward.  What this does is two things:
1) It reduces future expectations to more realistic live trading conditions; and
2) It provides a broad idea of the likely volatility of that return stream and it's degree of 'edge' that you can then use for portfolio construction using correlation as your guide.

I then superimpose the expected return stream profile over other strategies in my potential collection to ensure they are either anti-correlated or non-correlated.

This is a critical step in the portfolio construction process as your portfolio composite must be balanced with different return stream signatures to ensure there is no undue bias that will contribute to exacerbated drawdowns.

Through this method of portfolio construction the aim is to generate your linear upward sloping equity curve by inclusion of individual strategies that have different strengths and weaknesses but in composite mutually work together to create the ultimate blended solution that reduced overall volatility and improves the risk-weighted return.

Once this is achieved you can then go to step two and accelerate these results using position sizing methods that give far greater bang for buck than what could otherwise be achieved by a single strategy solution.

.....anyway...some food for thought.

Cheers

Rich

Hi Guys and particularly Steve

Thanks for making the effort to keep interesting threads like this alive. I find such great tips when sharing info from this kind of discourse. One way diatribes get quite frustrating to me. :-)

Steve.....I see and understand the points you raise regarding optimisation. I am not a big fan of it myself as it exacerbates the propensity for curve fit solutions.

If I use optimisation in my data mining (which I do currently but in a very weak form to generate solutions that pass validation criteria )...then I have to ensure I spend extra effort in the MonteCarlo phase ensuring that symptoms of curve fitting are not present in my final choice of selected robust strategy.

Where I think we may be differing in opinion is in our faith in the validity of the equity curve as being a method to project a future state. I have very little confidence in the historic trajectory being the guide for a future outcome given market complexity and the inability in complex systems to predict what will happen tomorrow let alone what will happen in a years time. I have had to flip my mind given the continued disappointment faced with forcing my expectations onto the market condition......and rather accepting what the market gives to me.......and as a result have reverted my entire emphasis to think in terms of what we can control......namely risk. IF the market is Non-Gaussian.....in other words if anomalies are more prevalent to what a normal distribution would imply....then the profits will simply be an outcome from being present in the market at the unknown time for the systems you deploy to capitalise on that anomaly. By being present at all times (which means you need to preserve your capital base) you will be in a position to capitalise on alpha when it presents itself. You will be too far behind the eight ball if you attempt to predict an arising anomaly. Not only will you be wrong most of the time as valid identification of an anomaly is always a hind-sight measure...but your delay in identifying them will mean you most likely will miss the boat in an efficient marketplace.

Like a hard stop, deciding to 'turn off' your strategy if it is entering a drawdown can unnecessarily introduce drawdowns into your portfolio....... as it is difficult to determine if the impact of 'turning off' is from a truly deteriorating strategy....or simply a natural and temporary drawdown phase that is exhibited by all strategies as no single strategy can navigate all market conditions. When you 'turn off a strategy due to apparent under-performance this inevitably occurs when the strategy is drawn down. This forces the realisation of a loss at this point...where in fact if you had a balanced portfolio, there would be other strategies performing during this market condition that effectively hedge the adverse impact of that strategy. 

My ultimate resignation to accepting that the market is very efficient has forced me into the mindset of creating the unsinkable ship through risk management approaches and then sailing that ship into an uncertain market context. Given the emphasis applied to risk management it is likely to remain afloat and provided it has some positive momentum, will (like the subtle impact of compound interest over time)...lead to wealth generation over time provided you are prepared to ride the volatility and let your strategies breath. Too much tinkering by subjective human biases often overrides this ability.

Robustness tests is all about techniques you deploy to reduce the volatility of returns while capitalizing on any weak 'bias' that may exist in the data. It is less about the equity curve trajectory and more about the nature of volatility in that signature. That gives clues in methods we can apply to 'plug' the weaknesses in the equity curve. The best I can assume from a linear projected equity curve is that volatility over time is less than a more volatile equity curve in navigating past market conditions....and that the positive slope is a potential sign that the strategy has a definitive edge. The quantum of that slope is not an indicator I use to project a future outcome.

It has taken a long time for me coming from the 'school of hard knocks' to come to an opinion regarding to the level of noise in market data. I have always been an ardent critic of the Efficient Markets Hypothesis (EMH) but have finally adopted a resigned stance that in any efficient market there is only a small amount of exploitable opportunity. This is the bread and butter for speculators seeking arbitrage opportunities and 'generally' the arbitrage opportunity is very short lived.

When referring to 'alpha' I am referring to the ability of an active participant such as a trader who through their actions have the ability to outperform the market. The market being 'beta' with any outperformance of that benchmark being alpha. The market (like a portfolio) is representative of the summated entry and exit decisions of the entire population of participants that interact with it. We as traders are seeking to obtain arbitrage opportunities in this overall zero sum game (with the added burden of frictional costs that go to the intermediaries) by outperforming the average participant over the long term and stealing a greater portion of the 'market pie' than our competitors.

What I mean by 'noise' in the market are the price movements in market data that have no enduring potential to influence a future price outcome. For example at the finest level of granularity we have the tick which is a price exchange at the finest resolution arising from the interplay between a small number of participants each with their own price objectives and strategies that impart that information into the market through their entry and exit exchanges. That price movement at the micro scale may or may not have no bearing on the future price trajectory and if it doesn't....then it is classified as noise and effectively can be termed 'random' in that it's presence does not have a bearing on future price action. It is not termed 'random' to reflect the definition of randomness applied to physics but in less formal trading talk it is termed random in that it has no bearing to the future trajectory.   

When you have two random price signatures superimposed on each other, by definition....there is no correlation between the series. As a result you tend to get destructive interference of the signatures when placed on top of each other. However when you have auto-correlated signals buried in the price series, the superposition of different 'correlated' time series on top of each other tends to amplify the ups and downs to turn what originally had a lot of noise present in all the series, to a blended series with less noise and more signal.

What this means is that at the smaller timeframes, there is lot's of noise in the signals....whereas as you step out into the longer timeframes, the signal/noise ratio increases with more signal and less noise. This ultimately has a bearing on the markets where we look for fundamental reasons for price movement in long term data....as the signal bias present in a time series....only reveals itself over a large sample size.

The same effect is achieved through flipping a biased coin. At the smallest timeframes the results of the flips may or may not have any bearing on the long term statistical result. It is only through the Law of Large numbers that ultimately we can detect the bias.

Because of my resigned respect for the EMH, I have the following central assumptions built into the way I tackle them:
1. Markets are competitive and an edge is 'hard to find';
2. It is difficult to achieve excess returns after accounting for risk;
3. A trader with an edge is a rarity;
4. An edge is ephemeral and usually a fleeting condition;
5. The EMH is right most of the time but not all the time...and it is the 'not all the time' where speculators can flourish if they know their stuff. The aim of an efficient market is just that...to be an efficient method of transacting at fair value..hence arbitrage opportunities in an efficient market are a rarity (and treated as anomalies) as opposed to a norm;
6. Weak signals in market data get amplified with the Law of Large numbers but there are valid statistical methods such as the Dickey Fuller test that can within confidence levels determine the degree to which auto-correlation is present in a time-series;
7. You have no hope of simple eyeballing in discriminating between a random price series and a price series with non random signals present. The best techniques to apply in determining this are statistical techniques.   

I will stop there rather than continue with the Tolstoy Epic :-)....but hopefully it gives you an idea of what shapes by philosophies applied to trading....and gives a basis for some of the statements I might make. There is no right or wrong....just different approaches to tackle this moving feast we call a market. We would probably all agree that this is a very tough game indeed. :-)

It's great to chat with you.

Cheers

Rich

Spot on Hannah. Excellent summary Different course for different horses.

In the trading world context matters as conditions are non-stationery and trading styles need to be adaptive. What works yesterday doesn't necessarily work today......but under a diversified portfolio, embedded in that solution or as a strategy response needs to be the ability to address a plethora of different market conditions that can change on a pinhead with no warning.

The bottom line is what we are applying here is the scientific method and the approaches we all use are simply different models. We can only gain access to partial information and as a result, our best models will only ever be approximates....some better than others at different times....but the ultimate arbiter are the performance results produced and as markets adapt, so should our models. What we need to avoid is assuming a particular solution is an enduring solution and we need to be on a continuous journey of improving our models. This may come from small scale refinements to the occasional total rewrite.....but we can take a cue from science in this regard as it is a never ending process of making better models that stand up to the current boundary of empiricism.

Cheers guys

Steve....just to throw a spanner in the works I would choose the second with the following caveat....that even though it has not seen a live account, we assume that it's execution is in accordance with the backtest result. That backtest result derived from data mining methods needs to be treated to ensure it is 'within realistic' bounds of probability (eg. the median result of of a comprehensive MonteCarlo array or 'block bootstrapping' test).

The reason I would always select this option is due to survivorship bias.

Let's assume you are back in 2000 and have funds ready to allocate towards various strategies....in this case fund managers. The choice is to select either:
1. The Fund manager with the best annual return for the past 12 months; or
2. The Fund manager that has had a track record of 20 years plus and offers a steady risk weighted return that is far lower than the annual returns generated by the option 1 guys.

This is effectively your question simply reshaped for the following example. In this following example think of an individual fund as a single strategy.

I have actually looked at both options and there is an approach to effectively guarantee that you will outperform the market and be alive in 20 years time just by using long term risk metrics of risk-weighted return. The former approach that simply looks at 'best profit performance over a short timeframe' is very deceptive as you tend to jump onto these opportunities too late as you needed to first identify these opportunities which means that most of the meat has already been eaten and your data mining competitors are also greedily eyeing off the opportunity or have already jumped on board. Data mining is a pretty new game in town....but even today the competition in this space is significant and enough to ensure that the opportunity arising from recent outperformance is jumped onto quickly and any arbitrage available quickly exploited.

It is a tough game identifying whether a strategy is broken or whether it is entering a drawdown...and that is why alpha is more persistent in an approach that deliberately seeks more volatile strategies that others avoid but can be treated at the portfolio level to reduce overall risk volatility.

Here is an article https://www.raftradingsolutions.com/let … nds-story/ that showcases what you can do in 2000 without any hindsight bias that ensures you achieve this over the long run. If you had simply selected the best profit performers many of them would not be around today and furthermore the need to rotate to different funds each time you rebalance your portfolio will introduce considerable drawdowns into your performance.

The tail risk that exists in any strategy or portfolio relates to it's ability to manage the downside....not the upside.

Regarding the 'simplicity' of indicators generated by EA studio....ideally simplicity is what you after for robust strategies. The more parameters that exist in a strategy (or their increased complexity) reduces the degrees of freedom allowed for by your strategy. What this does is that your strategy then dictates terms to the market (imposed constraints), when in fact the more robust solutions with less parameters impose less dictates on the market but rather responds to the market condition.

For example think of a simple hard stop placement on a unidirectional 'long only' strategy. If the stop is very wide then the overall signature of the equity curve produced by the strategy is more representative of the market condition. For example when the market is bullish, the strategy will be profitable and the equity curve will rise. When the market is bearish, the equity curve will fall.

If the stop gets placed too closely to the entry without sufficient room to breathe, the equity curve shape is dictated by the strategy (a linear descending line represented by continuously being stopped out....and bears no resemblance to the market condition). In other words the equity curve is derived from the strategy as opposed to the market condition. I also believe data is king....but market data and not system derived data. The strategy needs to float on the market condition (respond to the market data) as opposed to dictate terms on how the market should behave to generate the profitability (eg. force a profit outcome by design).   

Andreas Clenow...a very successful fund manager....and many others in the FM space discuss at length the need for the individual strategies deployed in a portfolio to be very simple...such as simple moving average crossovers, Donchian breaks etc.....but that these strategies must be adjusted for the characteristic volatility present in different markets. Complexity occurs at the portfolio 'global level' and not the individual strategy level.

Unfortunately at the moment EA studio spits out fixed lot outputs for a single strategy but over time if EA studio is further developed with inclusions such as volatility adjusted position sizing etc. we will be able to get more heavily into the portfolio management side of things where the real 'free lunch' resides. It is already a fantastic piece of software in it's current form....but the future potential is teasing me :-)

It is essential that statistical or other methods are used to determine what is potentially random versus what is potentially a signal. Programs such as EA studio will identify candidates through validation that meet your specified criteria. You must however supplement this with robust techniques to assess  whether or not the result is simply curve fit to the data or data mined on the signal that may or may not exist.

For example here is a sample equity curve that appears valid as a profitable strategy. Unfortunately the result is simply random and and has no future forecasting potential

Without rigurous statistical testing you could easily fall into the trap of assuming that this strategy will work into the future.

Here is another equity curve that is not random. There is auto-correlation in the data that creates a bias to the data.

Can you visually spot the difference? Would your performance results pick this up? The signal bias in this data series projects into the future but the random element of the series also creates significant variance against the future projection. It is exceedingly hard to confidently identify strategies that have the potential to survive into the future. Robustness testing is an essential prerequisite to apply...but in itself is also no guarantee.

These are the sort of traps we might fall into by placing an undue emphasis on recent profitability. At least with long historical data series the probability for a curve fit result is significantly less that a result derived from a shorter timeframe.

Hi Doc. I hope you had a great trip :-)

For me..I use what is available from the platform I trade off.....and the more the merrier. I also tend to use multi-market tests where possible subject to limitations of fixed lot sizing.

It is all just data to me Doc...so different markets in relation to data mining simply represent different conditions. If I test across 6 different markets for say the last 10 years then it in my opinion represents 60 years of different market data.

But...you probably know by now where I am coming from and all of us have different approaches we take. Certainly I can say my portfolios are robust....but the returns they actually deliver are nothing like the stunning equity curves that are generated from EA studio.....but I tend to favour robustness and sustainability over shorter term objectives.

Sorry Steve....I missed the second query.

Discretionary traders certainly can capitalise on persistent market conditions where price action becomes more predictable operating around a temporarily stable equilibrium point....... and can significantly outperform systematic traders in the short term......but across a longer term array of different market conditions that are defined by non-stationery equilibria, very few discretionary traders survive.

Jarratt Davis had a small hiatus in the sun....yet I cannot find evidence of an extended run. Peter Brandt is another well know technical and discretionary trader who appears to have had a successful record.....however these guys are the exception rather than the rule.

The key to survival across different market regimes is adaptability. Discretionary traders tend to be specialists who excel in a particular trading style that responds to a particular market condition......however the downside to their specialty is that they rarely have an arsenal of alternative approaches that can be applied to different market conditions successfully.

This is where systematic diversified portfolio that are already equipped with strategies configured for alternate market conditions come into their own. They are less proficient in 'making hay when the sun shines' for a single stable market condition and therefore under-perform the discretionary specialists during these times....but over an extended time horizon across a number of different market conditions.......like the tortoise and the hare.....they outlast and outlive their 'flash' discretionary cousins.

I spend a lot of my time on Barclays and IASG reviewing verified performance data to report on fund manager performance https://www.raftradingsolutions.com/pre … june-2018/ and have yet to find more than a very few discretionary traders that have stood the test of time maintaining performance metrics over 10 years plus.

The most successful breed of fund managers are the CTA's which are a subset of the hedge fund segment but depend almost entirely on the repeated systematic application of diversified momentum/trend following strategies across their portfolio's. The likes of Dunn Capital, Winton, Dreiss Research, Transtrend, Fort Capital etc. etc. etc. having stunning long term metrics.   

It is worthwhile spending some time examining their monthly performance results as frequently you see extended periods of losses. As you have mentioned previously, trading on the right hand side of the chart is a different kettle of fish to the linear rising equity curves you see from these guys. Patience and resilience is required and long losing streaks to enjoy the long term benefits of what these guys deliver.

Rich -- thanks for hanging in there with me.  Clearly you are a professional in the field and your knowledge and expertise way eclipse mine by orders of magnitude.

As a research biochemist in a previous life and a developer of wireless diagnostic tools in this one, I've looked at more than my share of data.  And through it all I am the biggest fan of the KISS principle.  I approach forex sort of like a mechanic -- always keeping KISS in mind -- and I think Popov's software gives us the best tools for this type of approach.

The principles and techniques you describe and refer to are way over my head.  That doesn't mean they aren't correct, only I have no means to comment on them.  Also, even if I did understand them, I have zero clue how I would implement and apply them.  Furthermore, at the risk of sounding like a dimwit, I will step out onto the plank and venture to say I don't think they apply here.  Forex is a different type of beast than other types of investments (which could be a topic for a different thread).  I think the primary reason I can't buy into what you describe is I think the stormy periods would cause draw-downs that would eventually wipe out my account before it had a chance to turn around.  Big investors, institutions, banks, etc. can ride out the storms, as you say, but my smallish accounts could not.  Yes -- I know about risk and money management -- and do take care to ensure I live to trade another day.  We trade in the present -- there's no past or future -- which is why current profitability is really all that matters.  And then tomorrow becomes the current and, hopefully, the strategy again exhibits current profitability -- at least that's the plan.

Adapting to current market conditions -- absolutely -- that's why I claim refreshing strategies with recent data is very important.  I mentioned the mechanic analogy -- just like changing your oil every 3000 miles or 3 months (whichever comes first) keeps your car humming, it will do the same for your forex strategies.

Rich -- I thank *you* for the conversation.  One of the best I've had in a long time.

Yes, Steve, without proper weighted risk management, one may not survive the long drawn down and the account can easily wiped out.

I believe Rich, therefore emphasized the importance of Robust EA and proper weighted risk management so that we learn the art of putting together a group of EA that can help keep the account "break even/keep the head floating above water level" during the prolong ranging period (90%) to catch the King hit of 10% windfall.  Without proper weighted risk management, one can blew his account very easily due to over trading, likewise with proper risk management, one may master the art of surviving in the long haul and not just short term success.

Lastly Rich, thanks so much for your delightful contribution and simulated our brains.  Could you kindly write in another separate post about your journey, the path and learning process you took to become where you are now.  I believe a lot of us, who are aspiring to become a fund manager one day, would learn from you and understand the stages of professional growth/milestones, we need to take to be successful.

Can you open another post, something like a personal blog to share whatever you deem is helpful for the aspiring traders.  Can you share the pitfalls to avoid and the road to success that seem so elusive to many of us.

Finally, the pointers I take off from this discussion are:

1) Robust EA - The importance of building more robust EA for long term sustainability (using longer historical data, to expose our EA to be trained in varied market conditions).  Hence, MC and Multi Markets are good tools to use for creating robust EA

2) Divergent EA, though may have terrible short term equity curve, they are by far, more sustainable in the long run compared to convergent (mean reversal EA).

3) Robust Portfolio - for long term survival, it is important to blend different types/diversification of EA to keep a well balance/composite of EA to construct a good, robust portfolio, together with weighted risk management, it is use thereafter to help one maximise the returns/profit.

4) So this is what I would do...

a) In the area of EA development - build more robust EA

b) In the area of Portfolio management - use diversification of EA with uncorrelated strategies/markets to keep a good blend of EA to tide through the storm.  I'm thinking, is it possible, in addition to a robust portfolio, can I still find a sweet spot (by observing my EA's behaviours) to turn on/off my EA (while keeping them running in demo to collect performance data) based on certain metrics?  So instead of having to resign to the fact that 90% of the time, my EA trying to stay a float (of that particular EA, but for a blended portfolio, while some EA are losing, others are winning because the portfolio selects uncorrelated EA/markets), I'm thinking of switching them off so that I don't have to endure the painful experience of bearing with the long drawn DD.

c) In the area of Risk Management - with proper weighted risk management, make my portfolio "unsinkable" so that we can survive over the long haul while many retail traders come and go.

Thanks once again to both Steve and Rich for such wonderful discussion that I certainly look forward for more.

Just one quick note - Some other software I use lets you place the OOS time period at the front of the sample. So you build your system in the closest data - and validate on the older data.  Many ways to skin a cat

I agree with sleytus about the need to optimize using recent data.

But how about the generating of new strategies?
Is it more valuable to use most recent data for that as well, or is it better to use past data for the possibility to have kind of a backtest with the most recent data?

Recently dannnnn_the_man started a new thread:
https://forexsb.com/forum/topic/7542/significance-of-timeframe-adjustment/

He showed a couple of charts where his strategy had been back tested against two different data horizons and was wondering why the charts looked so different.

The answer is obvious -- check-out the above link.  His chart comparison also shows why using too much old data when optimizing (that is, computing indicator settings) results in indicator settings that are poorly matched for the current data horizon.

When you look at your own charts it's so obvious.  I'm surprised so few people take notice -- perhaps it's because we only see what we want to see.  I don't know...