Topic: Generate Thousand EAs or Cherry picking few EAs

Hey, what do you prefer and why ?

1) Generating thousand of EA's with almost no filter then put them on a demo account to select only the best one.
2) Going through a very precise process to generate very few EA's then put them on demo to select only the best one.

Personnally I prefer the first method because I think no matter which amount of filtering we are aplying to our EA's we can never be sure how they will perform in the future so why bother with cherry picking few EA's that might fail on demo testing.
Lets use the power of EA Studio to generate thousand of EA's and test them all on demo account .

Re: Generate Thousand EAs or Cherry picking few EAs

i think you can also save the collection with your data after a time you let the collection now check with the newer data.

or also you can generate strategies with e.g from 2010-2019 after generating you validate them wtith data from 2010-2021 so i think you have an demo test of future ;-)

Re: Generate Thousand EAs or Cherry picking few EAs

Yes, this is an manual Out Of Sample,
But even with that we can't be sure our EA's will not fail immediately after putting them in live.

I've tried EA generated with OOS and Without OOS and the EA's generated with OOS do not did better ..

Re: Generate Thousand EAs or Cherry picking few EAs

> I've tried EA generated with OOS and Without OOS and the EA's generated with OOS do not did better

Because you evaluate the strategy at the end, having the OOS performance into consideration.
This actively removes the effect of OOS.

When you look at the OOS result, it becomes an In Sample result.
By looking into a system, you change the quantum reality. Don't ask me why.

When you run a strategy on a Demo for a month and then decide if it is good for Live, you do the same.
If you ask me how to prevent it, I don't know.

EA Studio only makes it possible to monitor real-live market behaviour with great precision.

Re: Generate Thousand EAs or Cherry picking few EAs

Thank you for your answer.

> When you run a strategy on a Demo for a month and then decide if it is good for Live, you do the same.

I am agree with you, for me running them on a demo account allow me to have an history data that I can use with Strategy Samurai

6

Re: Generate Thousand EAs or Cherry picking few EAs

Popov wrote:

> I've tried EA generated with OOS and Without OOS and the EA's generated with OOS do not did better

Because you evaluate the strategy at the end, having the OOS performance into consideration.
This actively removes the effect of OOS.

When you look at the OOS result, it becomes an In Sample result.
By looking into a system, you change the quantum reality. Don't ask me why.

When you run a strategy on a Demo for a month and then decide if it is good for Live, you do the same.
If you ask me how to prevent it, I don't know.

EA Studio only makes it possible to monitor real-live market behaviour with great precision.

Remarkable observation, Mr. Popov. One of my friends has decided to not to look at the results of his live accounts very often, because he thinks he turns them into losing strategies by just watching them. He claims that his strategies perform well until they are being observed. We joke about this, but there might be some truth behind.

7 (edited by hannahis 2021-05-10 09:27:05)

Re: Generate Thousand EAs or Cherry picking few EAs

mw wrote:

Remarkable observation, Mr. Popov. One of my friends has decided to not to look at the results of his live accounts very often, because he thinks he turns them into losing strategies by just watching them. He claims that his strategies perform well until they are being observed. We joke about this, but there might be some truth behind.


Logic: Don't observe things/person/item.  Observation would make the things/person/item perform badly"
What weird logic is that, it's proaching superstition.

Most inventions and scientific discovery arise from the observation and the studies of the "behavours" of the person/items/things being observed.

Just generate EA and install in demo, don't look at it and after 1mth, Viola!!! Congratulation, your account made tremendous profit.  Come back 1 year later, don't look, you will be a millionaire.


If only this logic is true and applies to kids.  Then kids not under supervision/observation are the most well behaved kids.  And they will turned out fine if we leave them alone Lol.  Such logic would provide an excellent excuse for those neglecting parents.

8

Re: Generate Thousand EAs or Cherry picking few EAs

hannahis wrote:
mw wrote:

Remarkable observation, Mr. Popov. One of my friends has decided to not to look at the results of his live accounts very often, because he thinks he turns them into losing strategies by just watching them. He claims that his strategies perform well until they are being observed. We joke about this, but there might be some truth behind.


Logic: Don't observe things/person/item.  Observation would make the things/person/item perform badly"
What weird logic is that, it's proaching superstition.

Most inventions and scientific discovery arise from the observation and the studies of the "behavours" of the person/items/things being observed.

Just generate EA and install in demo, don't look at it and after 1mth, Viola!!! Congratulation, your account made tremendous profit.  Come back 1 year later, don't look, you will be a millionaire.


If only this logic is true and applies to kids.  Then kids not under supervision/observation are the most well behaved kids.  And they will turned out fine if we leave them alone Lol.  Such logic would provide an excellent excuse for those neglecting parents.

Hi Hannah... I do agree it isn't a very scientific approach... We laugh about it, yet my friend is not looking at his trades anymore just in case wink

9 (edited by hannahis 2021-05-11 03:53:58)

Re: Generate Thousand EAs or Cherry picking few EAs

This is my hypothesis of your friend's behavour...

He is tempted to look when the market situation is bad and hence, true to his hunch (feeling of bad market situation), his EA/account didn't perform well.

While the market is good (probably trending), he doesn't feel the urge to look because his is confident of his EA under such circumstances. 

Hence, it's not that each time he looks at his account, it performance worsen because of him looking at it, it's more because under what circumstances he had the urge to looked at his account.  And thus, it's when market is bad, he tend to worry and look.

Most of us wished we don't have to keep watching our accounts.  It's only when we are confident of our Algo/EA that we can hands off and don't bother it.  We look only when we aren't sure of how our EA perform, otherwise, we would left it alone till we lost confidence in it.

10 (edited by ats118765 2021-05-13 16:47:32)

Re: Generate Thousand EAs or Cherry picking few EAs

The real issue when contaminating OOS data through enquiry is one of ‘selection bias’.

Say we have a strategy developed using In Sample data and then we take it to the OOS environment to see how it fares. In this case it performs well so we accept it for our live trading experience. So far no problem.

However, let us assume it performs badly during OOS and we elect to drop it and not use it, then we run another test using the same data with another strategy and it performs well so we keep that one and use it.

Now we have introduced  'survivorship bias' into our decision making.

The contamination problem arising when making decisions about multiple system return streams using the same data. A Bayesian Bias is introduced into the process….just like the Monty Hall problem.

Definitely worth having a listen to. https://www.youtube.com/watch?v=1RKz9v_0WDo

Diversification and risk-weighted returns is what this game is about

Re: Generate Thousand EAs or Cherry picking few EAs

According to the video (I'm in the middle of it) OOS should be used once and for 1 strat only smile

Richard, have you worked with those methods Larry explains in the video?

Do you know how the strategy development process is evaluated in the form of using permutations? I think this is the main point or value it might offer in terms of the tools we possess. He talks about "going back to the drawing board" and "tinkering" like it is picking up a hammer and some nails and then going to work on a shed, but what we have are hundreds of sheds at a time, so it's quite a difference and helping users to validate their workflow and methodology would be a huge step forward.

Data permutation is different from randomizing data in the Monte Carlo simulation of Studio, right? And what he looks for is diametrically opposed to our current use as well: he wants the original strat to stand out from strats trained on permutated data.

12 (edited by ats118765 2021-05-15 18:54:52)

Re: Generate Thousand EAs or Cherry picking few EAs

footon wrote:

According to the video (I'm in the middle of it) OOS should be used once and for 1 strat only smile

Richard, have you worked with those methods Larry explains in the video?

Do you know how the strategy development process is evaluated in the form of using permutations? I think this is the main point or value it might offer in terms of the tools we possess. He talks about "going back to the drawing board" and "tinkering" like it is picking up a hammer and some nails and then going to work on a shed, but what we have are hundreds of sheds at a time, so it's quite a difference and helping users to validate their workflow and methodology would be a huge step forward.

Data permutation is different from randomizing data in the Monte Carlo simulation of Studio, right? And what he looks for is diametrically opposed to our current use as well: he wants the original strat to stand out from strats trained on permutated data.

It is a punchy video and raises lots of points we need to be aware of when undertaking data mining processes, as we do here. :-(

I have had to develop my own workflow processes to create workarounds for the problems of over-fitting to data and data mining bias that arises through selection processes, but this is outside EA Studio. It needs to be.

Most of Larry's discussion centers on being able to detect the difference between random market data and non random market data. Once that is distinguished then, and only then, you can design system solutions to attack the non-random elements....but only once it has been distinguished.

Our systems need to respond to signals in the 'otherwise' noisy environment. We must be able to detect the signals in the market data.

We don't have that ability in EA Studio as we only can undertake Monte Carlo sims with trade results, not underlying market data.

The problem we face when only working with our trade results is that if those trade results are a response to random market data and not market signals....then we have a problem Houston. This means that we simply cannot tell whether or not our systems developed through these processes possess these statistical biases.

This is irrespective of the amount of post-treatment we apply through Monte Carlo Sims or Walk Forward Methods we use to attempt to remove this bias. These methods can simply further complicate the issue and exponentially magnify the existing bias unless we really know what we are doing. To assume we can simply use these methods without a comprehensive understanding of what we are doing is going to end in tears.

If the market is 'mostly' efficient hence mostly random, then these problems are really big ones for us data miners.

So the video serves as a heads up to make us at least be very careful with what tools we use and how we do it in our data mining experiences.

Some key ideas to consider in addition to Larry's ideas.
1. Adopt Design First Logic without Optimisation to configure your system through Design Principles to deliberately respond to a market condition you are targeting. Don't let the system generator decide for you. It will over-fit to the data which 'mostly is noise'.
2. Avoid reducing collection sizes by performance metrics. If there is randomness in the equity curve you are only exacerbating 'fictional' performance as you are removing the adverse random elements that make up the equity curve in your selection process.
3. Most Monte Carlo methods applied to trade results themselves (as opposed to market data) tell you nothing as they disrupt the serial correlation in the equity curve (related to the market signal) and not the noise element of the equity curve (related to the randomness in the market data). They turn the entire equity curve into noise.
4. Walk Forward should only be used for Predictive techniques that are seeking to capture a repeatable market condition. They are not applicable for trend following or momentum methods. A nice straight equity curve for a single system can only be maintained if market conditions remain favorable over the entire extent of that equity curve. That is only relevant to predictable methods for the period of time that the predictable market condition persists. Every equity curve must display periods when they are performing and periods when they are under-performing as we know that no system can address all market conditions.
5. Straight equity curves for the long term are the result of a portfolio of many different successful systems attacking many different conditions.
6. Always data mine using the most data you can get your hands on. This is the only way you can reduce the impact of randomness in your overall performance results. Data mining over a few years of data is asking for trouble.
7. The market condition determines your ultimate fate. A good system simply allows you to extract the signal from the market data. Bad systems......well they are just bad in every shape and form.

Just some ideas for what they are worth.

Cheers Footon :-)

Diversification and risk-weighted returns is what this game is about

13 (edited by hannahis 2021-05-16 09:11:17)

Re: Generate Thousand EAs or Cherry picking few EAs

Most of Larry's discussion centers on being able to detect the difference between random market data and non random market data. Once that is distinguished then, and only then, you can design system solutions to attack the non-random elements....but only once it has been distinguished.

When you mentioned random market data vs non random market data, are you really referring to data or are you referring to EA that are generated through randomness vs EA generated via trading theory which is not created via randomness by through proper understanding of underlying market behaviours?  At least this is my interpretation of random system vs non random system/EA, ie. EA developed with underlying understanding of market behaviours.


Our systems need to respond to signals in the 'otherwise' noisy environment. We must be able to detect the signals in the market data.

We can only know when it is noise only when the trend or direction of the price movement didn't materialized.  Meanwhile it is termed noise because of market forces, ie. Demand (buy) vs Supply (short) are at play (traders aren't clear of the direction, equal amount of buying and selling to force prices to range at a particular price point.  Till there is a clear break, then price break out of the price point.  Hence trend starts.  So these "uncertainty" are termed as noises but there are part and parcel of market forces, they are signals too, if you understand what market condition it is in, i.e Ranging Market. To the Trend traders, these are noises, to the Ranging Traders, these are signals.


We don't have that ability in EA Studio as we only can undertake Monte Carlo sims with trade results, not underlying market data.


Monte Carlo Simulation, also known as the Monte Carlo Method or a multiple probability simulation, is a mathematical technique, which is used to estimate the possible outcomes of an uncertain event

I don't use MC, I don't understand why we need to introduce more randomness to an already random generator.  That randomly generate systems/EA without any theoretical basis.  My challenge is to find a sound theory System/EA and why would I want to messy it up by messing up with the parameters further...I don't understand the logic behind it.  MC is for uncertain event like weather, but forex is not uncertain....it's driven by market forces, Demand vs Supply, it's not uncertain.  It's the traders/speculators that are uncertain of which direction they are going as a "whole" because there is no "coordinated" effort to decide (that's why banks and liquidity providers can determine the direction) which direction the market forces are going because everyone has different interpretation, different goals, long term gain vs short term gain etc and hence traders are the ones who are uncertain.  But once there is a "coordinated" recognition of where the market is going, prices begin to trend, that's where trader go with the trend and no longer wants to fight with the price point.


The problem we face when only working with our trade results is that if those trade results are a response to random market data and not market signals....then we have a problem Houston. This means that we simply cannot tell whether or not our systems developed through these processes possess these statistical biases.

Yes, we do have a problem here that Popov don't seem to recognized.  The need to generate EA/System that is rule based, ie. Market signals based on trading theory and not rules that are pluck randomly that are illogical to manual traders but somehow survived for a period of randomness or survived in the backtest/software environment and failed when put in live/demo account.


This is irrespective of the amount of post-treatment we apply through Monte Carlo Sims or Walk Forward Methods we use to attempt to remove this bias. These methods can simply further complicate the issue and exponentially magnify the existing bias unless we really know what we are doing. To assume we can simply use these methods without a comprehensive understanding of what we are doing is going to end in tears.

Exactly, to apply so much work to Randomised System is a total waste of time that's why users after all the MC, WF, Acceptance Criteria etc ended up with no or little success.



So the video serves as a heads up to make us at least be very careful with what tools we use and how we do it in our data mining experiences.

Some key ideas to consider in addition to Larry's ideas.

1. Adopt Design First Logic without Optimisation to configure your system through Design Principles to deliberately respond to a market condition you are targeting. Don't let the system generator decide for you. It will over-fit to the data which 'mostly is noise'.

Yes, it's time to allow users in EAS to have a choice to ensure our Generator goes by the trading rules and not some weird randomness.  Rules that I've mentioned in my previous post.  https://forexsb.com/forum/topic/7293/in … cess-rate/


2. Avoid reducing collection sizes by performance metrics. If there is randomness in the equity curve you are only exacerbating 'fictional' performance as you are removing the adverse random elements that make up the equity curve in your selection process.

To me, if it is random, it doesn't matter whether I have 1000 vs 10,000.  They are all useless.


3. Most Monte Carlo methods applied to trade results themselves (as opposed to market data) tell you nothing as they disrupt the serial correlation in the equity curve (related to the market signal) and not the noise element of the equity curve (related to the randomness in the market data). They turn the entire equity curve into noise.

Multi Market validation would be a more useful way of applying "MC" to market data, random market data without messing up your System's theory. Don't confuse Random Market data with Random System/EA.  EA generated by going through all the random curve fitting is a random System.


4. Walk Forward should only be used for Predictive techniques that are seeking to capture a repeatable market condition. They are not applicable for trend following or momentum methods. A nice straight equity curve for a single system can only be maintained if market conditions remain favorable over the entire extent of that equity curve. That is only relevant to predictable methods for the period of time that the predictable market condition persists. Every equity curve must display periods when they are performing and periods when they are under-performing as we know that no system can address all market conditions.


5. Straight equity curves for the long term are the result of a portfolio of many different successful systems attacking many different conditions.


Agree, those going for "perfect" equity are asking the software to look for some random system that can fit into an unrealistic expectation.  To ask for near perfect Equity curve is as good as asking for random system, ie asking the software to look for failure.

"Perfect" equity curve can only be achieved in a portfolio that managed to balance the account's equity curve through the use of different system/EA, diversification of systems and markets (different currency pairs) so that when Trend EA are not performing, the Ranging or mean reversal EA can help to soften the fall and hence Equity curve can look more linear, perfect.


6. Always data mine using the most data you can get your hands on. This is the only way you can reduce the impact of randomness in your overall performance results. Data mining over a few years of data is asking for trouble.

Another way to have more data and to reduce randomness is to use Multi Market validation.  It means, these system are facing many types of market data randomness/conditions/fluctuations, volatilities, that it takes a good system to pass it and hence reduce some form of randomness but not totally.  If any random system pass such vigorous system, then it's purely magical and luck.  1 in a billion and not because of some effective software but just pure randomness.


7. The market condition determines your ultimate fate. A good system simply allows you to extract the signal from the market data. Bad systems......well they are just bad in every shape and form.

Introducing Trading theory in a software determine your "fate" in finding success in your process.  Not having theory in your generating search is like hoping to win in a lottery system.  That's why I tend to use Preset indicators (introduce theory based rules) to increase my chances. Best of luck.

14 (edited by ats118765 2021-05-16 17:26:56)

Re: Generate Thousand EAs or Cherry picking few EAs

Hi Hannah

Most of Larry's discussion centers on being able to detect the difference between random market data and non random market data. Once that is distinguished then, and only then, you can design system solutions to attack the non-random elements....but only once it has been distinguished.

When you mentioned random market data vs non random market data, are you really referring to data or are you referring to EA that are generated through randomness vs EA generated via trading theory which is not created via randomness by through proper understanding of underlying market behaviours?  At least this is my interpretation of random system vs non random system/EA, ie. EA developed with underlying understanding of market behaviours.

I am referring to the market data. If we have random market data then the system results produced will be random results. We still could have a profitable system result but that is from luck alone. Not an edge.

If we have non-random data, then the system results produced are non-random. System performance is a derivative expression of the market data.

To trade with an edge, a system must be able to exploit non-random data. Most market data contains noise (random) with some signal (non random). All trading systems with an edge need to exploit the non-random component of the market data and the resulting performance from this edge must be able to outperform the random component of the trading results.

PS Good point about 'Multimarket testing' :-) It is a great way to increase the trade sample size to test the edge of your system. But Multi-market capability is severely limited in EA Studio. When we apply Multimarket testing in EA studio we are currently limited to similar priced markets with similar volatility (eg. GBPUSD and EURUSD).

Ideally if we could adopt $Risk with ATR based stop/trailing stops as opposed to pip based stop/trailing stop and lot sized position sizing,  then we could test across any liquid market. This would significantly increase the trade sample size to eliminate possible bias.

Diversification and risk-weighted returns is what this game is about

15 (edited by hannahis 2021-05-17 08:05:17)

Re: Generate Thousand EAs or Cherry picking few EAs

I am referring to the market data. If we have random market data then the system results produced will be random results. We still could have a profitable system result but that is from luck alone. Not an edge.


So a market that is more volatile will produce more random market data?

Based on such definition, there is no way we can "remove" such random market data, isn't it?  Because without such random market/volatility, there won't be "losers" and hence there won't be winners cos forex is a zero sum game.

Won't it be a case that because we can't "see" the signal behind the randomness and thus we term it random but actually there is a pattern, only via data mining, such as EAS it can see some form of pattern/trading edge behind the randomness we can't see?

That's part of the reason, I usually let EAS search for the rules and demo test which strategies work in live/forward testing and then take these strategies as Preset indicator to increase the search/generator success with some guided direction (via Preset indicator to serve as the guide).


To trade with an edge, a system must be able to exploit non-random data. Most market data contains noise (random) with some signal (non random). All trading systems with an edge need to exploit the non-random component of the market data and the resulting performance from this edge must be able to outperform the random component of the trading results.

I assume an example of non random data is where there is a clear trend and prices are moving according to the technical indicators "prediction" and random data is where price is ranging and there is no clear direction of where the trend is heading.  Hence a trading system that has an edge is the one that has good entry(open) and exit (closing) conditions that are not easily misled by the false signals/random data.

If my above understanding is correct, then a good entry signal would be one that is not "conflicted" in it's signal.  Conflicted signals/opening/closing conditions that contain conflicting signals such as one condition is rising in it and the other conditions has falling in it.  Or signals that open when fast is lower than slow.  In another words, it's like a trend EA that use counter trend signals, hence conflicted signals. 

Hence a system/strategy with good trading edge is one with accurate entry and exit signal and other signals that will prevent it from entering wrongly in random market data, ie. ranging.

Won't such argument be bias?  It's assuming Trend signal is Non random and Ranging signal as random.  Won't it be a prejudice against convergent/mean reversal/ranging strategies that knows how to exploit on such random data market data?


Ideally if we could adopt $Risk with ATR based stop/trailing stops as opposed to pip based stop/trailing stop and lot sized position sizing,  then we could test across any liquid market. This would significantly increase the trade sample size to eliminate possible bias.

First of all, we all know that Popov has been working tirelessly to improve EAS and has been constantly pushing the envelop to make EAS more efficient.  And I also have been tirelessly pushing his envelop to stretch EAS's potential further often to his dismay (sorry, I'm like the delinquent kid that keep testing parent's boundaries).

So to introduce ATR based stop/trailing stop (which no doubt) will produce awesome outcome to the strategies that can be generated because it is more dynamic and hence takes into account of market volatility instead of the pip based SL/trailing.  However, given the context of the backtesting environment (the need to ensure EA logic can be fully backtested in bar data with reliable outcome), whereby EA signals are mainly depended on bar open/close, how is ATR based stop/trailing possible to be implemented? That would require revamping the whole backtesting engine structure of EAS, isn't it? Or is there a way that you know that can get around or work within the backtesting engine requirement/structure that uses bar data to implement ATR?  How do you resolve this issue? Code your own EA, add in your own code to EAS strategies, then backtest in MT4 etc?


Can you kindly explain how by $Risk with ATR SL/trailing, it will increase the trade sample size and eliminate possible bias? What's the relationship between $Risk and sample size? I fail to see the connection, kindly enlighten me.



Note:
To other readers, kindly understand that I'm always trying to push EAS's envelop and see if there ways to improve it further.  So I may sound like complaining about EAS but I'm not trashing it.  EAS is still the best tool for me out in the market and I won't hesitate to recommend it to anyone (provided the person put some brain into using it and not expect it to be a brainless money making machine) but to those who know how to use it, it's is a gold mine indeed.

Re: Generate Thousand EAs or Cherry picking few EAs

Hi Hannah. Thanks for the discussion.

So a market that is more volatile will produce more random market data?

I think the reverse is probably true but haven't tested it. Volatility implies that there is some causal influence associated with its increase. An unequal distribution between buyers and sellers that exacerbates price moves.


Based on such definition, there is no way we can "remove" such random market data, isn't it? 

I think Larry has a way that he describes in the video. He produces a random price series so he can benchmark that random series against another possible series that has non-randomness in it.

To be clear, I categorically state that a market is not perfectly efficient, hence not perfectly random. There is an edge that can be obtained from non-randomness but it is slight. A lot less than what the equity curves of our data mined EA's suggest.....but it is there.

A good EA should be able to target a non-random pattern and can be 'designed for' however there is no 'intelligence' in data mining processes to do this. A data mining process will just do what it does well....crunch enormous amounts of data quickly to 'curve fit a response to that data'. It cannot 'decipher' whether or not that data that it has matched to is random or not.

A way around this is to find an obvious non-random pattern such as a major trend....or a major series of oscillatory cycles of mean reversion in the market data itself, and then purpose build (design) a solution that you know can address the feature you feel is non-random. Once the design is created then see how it fares on another set of market data and do the map to market test to see how well it copes with that 'major pattern in that environment. Don't let a generator do this itself as it will match to any data arrangement.

I assume an example of non random data is where there is a clear trend and prices are moving according to the technical indicators "prediction" and random data is where price is ranging and there is no clear direction of where the trend is heading.



Good analogies Hannah :-). A trend however can be either random or non-random in nature. It can adopt many different forms.  Distinguishing a possible non-random trend is impossible without hindsight judgement. It all depends if the data series that makes that trend has a serially correlated bias in the series. The only way to address this is 'only take the most obvious trends'. They are the ones that are more likely to possess serial correlation. A generator once again won't pick this up...but design logic can.

Trend following models need a filter to restrict the ability to take all trends.....and just focus on the material trends. The entry into a material trend is not really important, but the filter is. Once you have a filter then you can enter pretty well at any point in a material trend. What is super important is the exit for trend following and a trailing stop rather than a profit target. We can never 'predict' when the trend will end and we really benefit from left and right tail events with no profit target.

Entry is crucially important for mean reversion methods and momentum methods (but not trend following).

The exit is important for all methods.

How is ATR based stop/trailing possible to be implemented?

It is a real coding challenge because it can compromise the architecture of the entire workflow process. You need it embedded in the Workflow architecture right from the get go...otherwise it becomes a major drama.  We have found that in our Workflow design processes. That is why we probably don't have progress here on the request for ATR based stop, position size methods. Popov does a great job but it restricts what we can achieve. If we work within Popov's programming architecture then that is fine, but if we want to stray outside that envelope (like I do) it restricts us. We probably need Popov's reason however, as this could all be descriptive hand-waving.

Can you kindly explain how by $Risk with ATR SL/trailing, it will increase the trade sample size and eliminate possible bias? What's the relationship between $Risk and sample size? I fail to see the connection, kindly enlighten me.

In my trend following models, I only focus on material trends (fat tailed price movements). Their infrequency combined with the fact that when riding them I can be doing so for years means that each system I deploy has a very low sample size. I need to 'prove' my process by increasing my sample size. By using normalisation methods of ATR based $Risk and position sizing calcs. then I can test my models over any liquid market under diversification,. Say 40 to 50 markets.

So I turn a system with a low sample size of say 20 trades over 20 years, to now say 20 trades x 40 data streams over 20 years which increase the sample size to levels that can validate my 'proof of concept' within a statistically valid trade sample size.

Cheers Hannah

As always, nice chatting :-)

Rich

Diversification and risk-weighted returns is what this game is about

17 (edited by hannahis 2021-05-18 09:25:44)

Re: Generate Thousand EAs or Cherry picking few EAs

Hi Rich, thanks for taking your time to teach and explain, it's always such a pleasure to have a discussion with you and I learn a lot from you.

A way around this is to find an obvious non-random pattern such as a major trend....or a major series of oscillatory cycles of mean reversion in the market data itself, and then purpose build (design) a solution that you know can address the feature you feel is non-random. Once the design is created then see how it fares on another set of market data and do the map to market test to see how well it copes with that 'major pattern in that environment. Don't let a generator do this itself as it will match to any data arrangement.


I totally agree if we can add "design logic" or trading logic into EAS such as allow only Trend rules in the opening conditions and prevent anti trend rules (such as falling, slow above fast etc, that are counter trend rules), then we have eliminate half of the non random possibilities that EAS can use and hence increase the chances of using only trend friendly rules in our generator search.  This is not an impossible task for Popov and doesn't need revamping the backtesting engine (in my limited understanding or I've over estimated Popov capabilities).

Meanwhile we can use EAS optimiser to self build our own EA and put the range of parameters for EAS to search for the optimal settings, there again there is no guarantee of ending up with non random strategies, but at least we cut down the probability.


Trend following models need a filter to restrict the ability to take all trends.....and just focus on the material trends. The entry into a material trend is not really important, but the filter is. Once you have a filter then you can enter pretty well at any point in a material trend. What is super important is the exit for trend following and a trailing stop rather than a profit target. We can never 'predict' when the trend will end and we really benefit from left and right tail events with no profit target.
Entry is crucially important for mean reversion methods and momentum methods (but not trend following).
The exit is important for all methods.


I agree that a good strategy need to have 2 components.  1) How to make money 2) How not to lose money.

It's rather easy to spot a trend and enter (to make money) but it's really difficult to stop a Trend EA from bleeding and losing money during not trending markets, hence the importance of having good filters to tell it when not to trade.



if we want to stray outside that envelope (like I do) it restricts us. We probably need Popov's reason however, as this could all be descriptive hand-waving.

Yes, often we wish Popov is in the same page as we and probably he also wished we see what he saw, the limitations and restrictions, it's easy to imagine solutions, it's difficult to implement it.

One of my imagination would be using higher time frames indicator in 1 M time chart.  But instead of using the H1 open/close price to calculate the parameters, I wished it uses instead M1 open/close price.  And hence we can have the stability of higher time frames parameters but the sensitivity of M1 price changes to react to market changes.  This method can be replicated in the backtest engine and because it uses bar data, it still meet Popov requirement for reliability testing.  And ATR SL/Trailing can then be implemented because it would use M1 bar to trail which is far more effective than using H1 bar trailing.

Let's hope Popov can catch a glimpse of what we are imagining and put his coding prowess at work to make our dreams come true...that he is a real expert, always exceed our imaginations, that's why EAS is unique, one of it's kind in the market.

Rich, you are really admirable.  You have a heart to teach and you are always so patient to explain and most of all you are so giving with your time and experience.  Most professional or successful traders don't have time for others but you always make time to value add to the community and that's why I hold you with high regard.  Thanks for being part of this community.  Have a pleasant and fruitful week ahead!  Cheers!

Re: Generate Thousand EAs or Cherry picking few EAs

Spot on with your comments Hannah and many thanks for the kind words. Have a great week as well :-)

Cheers

Rich

Diversification and risk-weighted returns is what this game is about