1 (edited by rjectweb 2018-02-21 20:30:06)

Topic: When would you have stopped this robots?

Please, contribute to this topic.

I'm posting here a couple of EAs that started well but have gone bad in the last days. Both of them showed signs of being worsening, so I could have stopped them from traiding at some point. The problem is when and how to decide when to stop trading an EA.

https://s10.postimg.org/excccjkhh/1_stop.png

https://s10.postimg.org/xcwtacyqd/2_stop.png

One can see for example in the second image that this EA starts worsening at a point (the slope changes and gets flatter) and then the EA finally crashes. If we could foresee this behaviours we could stop some EAs before going bad. The problem is when and how. In this case for example a moving average on the equity could have helped (for example).

What are your ideas about this topic? How do you know/when do you think a robot should be stopped?

Please, post your approaches here so we all might get to a conclusion that could be of help for all of us.
Thanks in advance
Warmest regards
RJ

Re: When would you have stopped this robots?

I'll stop using a robot when the. current stagnation period exceed half of backtest stagnation, that's why I always prefer a strategy with shortest stagnation, and use the longest data available. Off course I have maximum standard of drawdown too.

I hope Mr Popov will add stagnation as one of generator and optimizer criteria.

3 (edited by sleytus 2018-02-22 02:44:08)

Re: When would you have stopped this robots?

I've developed a new app and will release it soon.  It uses a Demo account to drive decisions in your live account.  The same portfolio and strategies are running in both accounts.  The strategies in the Demo account are allowed to trade with no limitations and are used to create an active database of trading results.  When a strategy in a Live account receives a signal it first checks if the WinRatio is >= 0.5 in the Demo account.  If it is, then the trade continues, otherwise the signal is ignored.  In this way strategies are dynamically turned on and off, depending on their current performance in the demo account.  The WinRatio metric is computed over the most recent 10 trades.

The app is called MT4 Sidekick.  I've been using it for a little over a month and it works well.  You still have to create good strategies -- it won't turn losing strategies into winners.  But since it can detect the ebb and flow of a strategy's performance, then it should help a lot when it comes to managing a portfolio of strategies.  It should address exactly the  problem you describe -- I mean, that is the primary reason I created it.

4 (edited by hannahis 2018-02-22 08:47:53)

Re: When would you have stopped this robots?

That's what I've been dreaming for.

Steve, is there a way to increase the number of metric options with check boxes for one to choose how many options to use as trading criteria?

1) Win loss ratio - is not a good criteria for a "normal" breakout EA cos it tends to have a lower win ratio (though not all Breakout EA has that limitations)

2. Max DD - Two schools of thoughts To have the choice to either 1) turn OFF EA when reached max DD or 2) Turn ON EA when reached Max DD

a) 1st School of thoughts - Conventional method is to turn off EA when it reached it's max DD

Firstly, if we were to use this conventional method, we will be constantly throwing away EA whenever they reached their max DD cos if we were to turn off when reached DD, then when will we know when to turn on? 

Picture this - imagine your EA's performance cycle is like Stochastic, do you buy (turn on EA, during max DD) when the market hit rock bottom (max DD) or do you buy (enter long, turn on EA) when market reached it's peak's performance? When a EA reached it's max DD, what is likely to happen? The EA continue to worsen or the EA will start to pick up in it's performance?

Secondly, we need to look at market cycle or market situations instead of purely statistics as our decision making.  For example, if our EA is a Breakout EA and the past few days have been a prolonged ranging, unclear market trend direction, it is expected that the EA will be losing over the past few days/weeks and hence reached or even exceeded it's max DD.

Question - when we have a prolong ranging days, what do we expect at the end of the ranging days? 

Which is more probable/likely to happen next? 1) The ranging days will continue further and hence incurred more DD, or 2) There is going to be a breakout soon?  So each time we turn off the EA near the end of ranging days (becos our EA hit it's max DD), we will be missing out on the coming opportunity when the Breakout is going to happen at the "end" of ranging days.

2) Second school of thoughts - turn on EA when ever it reached it's max DD

This school of thoughts (which I propose) fall in line with how market cycle or EA life cycle behaves.  A breakout EA will have it's peak and then followed by it's DD (because between each breakout, there are a number of ranging days and hence EA will likely reach a certain level of DD).  Hence, it is wise to turn on EA after a certain number of ranging days or when EA reached a certain level of DD.  Cos we are expecting a breakout is about to happen so it's about time to start trading a breakout strategy.

3) What is the reason the EA reached it's max DD?

a) It's part and parcel of the market cycle of the EA (as explain above)

b) The EA trading strategy isn't based on any sound trading theory and hence, it will keep deteriorating.

Thus it is important to understand whether...

a) What type of EA are you using, Ranging EA or Breakout EA so that we will understand the trading patterns, market cycle and hence equity curve and it's performing metric indications.

b) Whether the EA is a profitable one in the 1st place.


3).  PF value - Turn off EA when PF is below certain value or turn off when PF started falling and turn on when PF started to rise.

It's like the PF is an indicator.  When PF started to fall, turn off and when PF started to rise, turn on.

The rationale...

a) Does the EA has a Trading Edge?

A good EA will tend to increase in it's PF value over time (after the "incubation" period such as 1 to 2 months, cos during this incubation period, the EA may have extremely high PF and then start to decline and then it will begin to "settle" at it's "performance" marker, it's "usual" performance standard, from here onward the PF will then fluctuate according to the market cycle as explain above).

A good EA with a trading edge tend to show (sometimes small depending on the EA's trade frequency) incremental rise over time.  In another words, a Breakout EA need to earn more than it loses during ranging period.  Hence, it's PF has to increase over time but if a Breakout EA tend to lose more (during ranging times) than it earn (during breakouts), then it's PF value will decline over time. 

So if you want to know whether to "throw" away an EA, look at it's PF value changes.  Does it increase over time or decline over time.  Then you will know whether your EA has a trading edge (advantage).

b) PF changes signify market changes

If I'm using a Breakout EA and now it's ranging times, my EA will start to lose money and hence the PF value will start to decline.  Hence, turning off my EA with PF value starts to decline (over 2 to 5 days or over a number of trades, if we can specify how many days/trades to calculate the rate of changes), is like turning off a Breakout EA in ranging times.  And when PF starts to increase, turn off (this is tricky cos if we waited too long for PF to rise then turn on, we may miss the breakout instead).

4) Net Profit % change

Likewise, all the above performance metric ultimately is to measure how well an EA make money and hence another "easier" method is to use the Net Profit % change cos it comprise of all the mentioned metrics

Win loss ratio - means that EA started to lose more than it win (when does that happen? For an Breakout EA, it's during ranging times)

Max DD - means EA is having a streak of losing trades and hence reached max DD (when does that happen? Reason same as above for a Breakout EA)

PF value - Declining means the same as above

Hence, I would like to have

1) Turn off EA when it's Net Profit decline at certain %

2) Turn on EA when it' Net Profit begin to rise at certain % or

when EA reached Max DD or

when EA win loss ratio begin to increase or

when EA PF value begin to rise.

Steve, above are just some of the suggestions, of cos it's in your prerogative to decided ultimately what you would like to add in etc.  Whatever it is, this tool definitely will help to improve one's trading results when used in the most appropriate manner.  Looking forward to hear more of your progress in this project.  It's a very promising tool to have.



RJ - These past few weeks have been really tough for a Breakout EA to do well (unless one has some good filters for the Breakout EA to avoid multiple false signals during these periods).  Hence, it is "normal" if your EA is experiencing bad results these past few weeks.  Unless you think your EA doesn't have a trading edge (whereby it is able to sustain those loses during ranging times because it will win more than it lose at each Breakout).

5 (edited by hannahis 2018-02-22 09:04:38)

Re: When would you have stopped this robots?

sleytus wrote:

I've developed a new app and will release it soon.  It uses a Demo account to drive decisions in your live account.  The same portfolio and strategies are running in both accounts.  The strategies in the Demo account are allowed to trade with no limitations and are used to create an active database of trading results.  When a strategy in a Live account receives a signal it first checks if the WinRatio is >= 0.5 in the Demo account.  If it is, then the trade continues, otherwise the signal is ignored.  In this way strategies are dynamically turned on and off, depending on their current performance in the demo account.  The WinRatio metric is computed over the most recent 10 trades.

The app is called MT4 Sidekick.  I've been using it for a little over a month and it works well.  You still have to create good strategies -- it won't turn losing strategies into winners.  But since it can detect the ebb and flow of a strategy's performance, then it should help a lot when it comes to managing a portfolio of strategies.  It should address exactly the  problem you describe -- I mean, that is the primary reason I created it.

Steve, would it be "easier" to just have 1 account whereby all the strategies are in the demo account and this demo account will send trade signals to the live account and the demo account will 1) trade as per normal without any restrictions as you mentioned, 2) Send only trade signals that meet the acceptance criteria

So the demo account serves as 1) A testing account to gather all trades performance metric 2) A trade signal account (copy trade function) account to send trade signals to live account.

As for the Trade Signal function, we can add in some "safety" measures (normally found in typical copy trade software) such as

1) specify the max lot allowed to copy (cos I may be using 100 EA in my demo account for testing but I won't know how many will "pass" the criteria at any given time, I don't want to ended up over trading, exposing my account with too many trades.

2) lot size allocation such as demo account using 1 lot size but copy only at 0.01 lot size etc

3) Copy only profitable trade - even if the EA pass the performance metric criteria, copy this EA if it's position is now in floating profit

4) copy on Long or Short trades

5) copy at certain days/time etc

6) max slippage allowed to copy trade etc (if you put copy floating profit trades, this slippage has to take into consideration how much floating points to start copying and this "amount" of floating profit is part of the slippage allowed.

This is just a suggestions, I'm not a programmer so I won't really know which is "easier" for a programmer but as a user, I feel this would be "easier" cos if for some reason the "checking" function failed in the live account, I'm afraid the live account will then trade all the EA (imagine I've 100 EA running).

So if we can use the demo account as copy signal account to send trades to live account and with all the necessary safety measures in place (for the signal function), we will prevent the live account from over trading cos we won't know how many trades will pass the acceptance criteria and also if the checking function becomes faulty.

Re: When would you have stopped this robots?

Hi yonkuro, thanks a lot for contributing.

Yes, stagnation period and drawdowns, exceding the maximum / historic values should be a good criteria. However, there seems to be a problem when we have hundreds of EAs in a portfolio: in order to make the decissions based on that factors one should have the historic values of drawdown or stagnation periods, but this usually is not the case when we are trading hundreds of EAs in a Portfolio EA.

I mean, in the EAs I posted all what I have to make my real time decission to stop trading or continue trading the EA is the information provided by the trades in my live account. Maybe the last n trades, maybe all the trades, the R factor, the profit factor, the moving average of the balance, the percentage of winners, etc. If we want to make a real time decission, all what we have is some statistics on the trades in our account... It would be great if we could have more information but we don't usually know much about each EA, only the last trades it took.


Hi sleytus, thanks a lot for your answer.

Yes, my primary goal is what you describe in your post: being able to dinamically decide what EAs can go to a live account and which EAs should be stopped (at least for a couple of trades more). I had already thought about the two accounts with the same portfolios, although I was thinking about a shared file containing the tradeable magics, instead of a database.

Your new tool looks great. Thanks a lot for your generosity sharing your tools. I'm using them a lot.

The idea of a using database opens a new world of possibilities. For example, we could "feed" the database with the info/results of the EAs in a backtest. That would give us the historic data of each EA that we could need to make decissions based on statistics during the In-sample period. I mean, comparing the results we have now in the demo/live account with the statistics/ the results we should have according to the backtest (for example, the stagnation period or the DD, as yonkuro said) could give us the reasons we need to stop or continue trading an EA. I think this feature (feeding the database) could be useful. However, I think there might be simpler methods / approaches to my request, and that the trades we have in our live/demo account by themselves could be enough to make our deccissions about stopping or continue trading an EA.

I think having winning trades > 50% of the last 10 trades is possibly not enough in most of the cases. You said "One should build good EAs because this method doesn't turn bad EAs into good ones". Yes, I understand that, but I think one could easily think of an EA where more than 50% of the trades were profitable but the last 10 trades turned into a big loss overall. For example and EA where each loss is a lot bigger than each winning trade (this is typical for scalping) wouldn't work well with this method.

Of course we could reverse the procedure of creating the EAs and start building the EAs according to the method we have to stop them. I mean, if an EA with a lot bigger losses than winners (scalper) is bad for me bacause my method of stopping is what you said (less than 50% winning of the last 10 trades), I could start building EAs where losses and winners are similar in terms of money (Risk/Ratio: 1:1 or similar). This way I adapt my building method to my trading/stop method. I'm not really in favor of that approach. I would prefer to try to find a more general method of stopping an EA without having to think about it in the building proccess.

In summary, I feel 50% winning of the last 10 trades might not be enough for the general case. Do you think it works well with your EAs? Do your EAs have some characteristics? Are they always built the same? MAybe a high % of winners or something? Please share what kind of strategies work well with this method. Thanks.

Thanks once more sleytus for your new tool and your generosity. The new tool looks great. I'm really keen to use it.


Hi hannahis, thanks a lot for your contributions. Much instructive.

In my experience, I'm using EA Studio a lot more than FSB pro. As a result, the EAs I usually build are not "very elaborated", they don't use many filters or indicators, not higher timeframes, etc.. They are "datamining" strategies and not very related/not a consequence of a previous analisys of the market. That's completely ok for me. It doesn't mean my EAs are bad or they don't have good statistics. On the contrary they usually have very good curves when I add them to a portfolio. I'm using portfolio EAs to trade them (all together) and then, after a few trades, I select (so far, manually) the ones I want to move to a live account.

As I said before, I usually don't have much information about each EA. With my building method (I might need to change this) the EAs in a portfolio might be very different ones from others. My point here is that I don't usually have a historic profit factor or historic net profit of each EA, not even an insight about if the EA is a ranging or a breakout EA. I only have a sequence of trades with their profit and losses for each EA, and only with that I have to make a decission about continue or stop trading each EA. A fast and automatic decission for each EA.

It doesnt matter if we use a copier from one EA to another or we use two accounts with the same EAs, the case will continue be the same: what EAs to copy and what EAs not to copy. Thanks a lot for your ideas about this. Reducing profit factor, %winners or net profit during the last trades I guess might be somehow the way to go. Maybe I could calculate this factor by the last 10 trades as sleytus said, but maybe instead of a 50% threshold I could use a different limit (being more restrictive, for example).

Sorry for such a large post. Please continue posting your ideas if you like.
Warmest regards
RJ

7

Re: When would you have stopped this robots?

I think Sleytus has in mind to create two accounts
one demo  and one real
demo will work with an indicator which presents balance/equity start for one ea
We can have this intdicator in fsbpro I thnik

Re: When would you have stopped this robots?

Hello,

I didn't mean to hijack rjectweb's original post.  I thought he posed an interesting question and topic, so I took the risk of mentioning the approach I'm currently working on.

The idea is very simple -- two accounts (demo and live) running the *same* strategies.  The demo account always trades and keeps track of wins / losses.  The live account checks the wins / losses before proceeding with a trade.  The wins / losses are computed using the most recent 10 trades -- and it's adjustable.  In this way one can use the demo account to get an idea whether a strategy is currently a winner or loser.  It all depends on the incoming data, and the incoming data is always changing.

I'll have more to say in a different thread -- for now, let's return to rejectweb's original question.

Re: When would you have stopped this robots?

I love hijackings smile

Steve, what is the probability for a trade to succeed if W/L ratio for previous n trades is above x threshold?

Re: When would you have stopped this robots?

footon wrote:

I love hijackings smile

Steve, what is the probability for a trade to succeed if W/L ratio for previous n trades is above x threshold?

I'm curious to know too.  What's your observations from your own tabulation and also what kind EA are you trading, Breakout EA or Ranging EA?

I think there is Generally 2 Basic Functions when I performance metric, such as PF, Win/Loss etc(of cos there are more, than 2)

Phase 1/Function 1
1) To measure the EA's overall performance over time (in another words, we want to know the EA's "average" performance over a certain period of time.  I usually wait till I got at least 2mth (the longer the better) demo testing to gather the performance metric.  Once I've these statistic, I use it for EA selection and put into a portfolio.  This is rather a "easy" decision making, choose EA with the top performance metric.

Phase 2/Function 2
2) To use Performance Metric as a "Predictive" indicator for deciding when to Turn On and Off EA.  This is by far the most "difficult" factor in my opinion, because by the time we "gathered" the results, It's "Post" results and not "pre"dictive results


Assuming all my EA selected in Phase 1 is done "correctly", in another words, all the EA selected are "profitable" hence I don't have to question the issue whether this EA will "fail" or not.  The question is when to turn off when the market is not "favourable " to my EA's make up/"training"/market cycle.  Such as a ranging period for a Breakout EA.  Hence my "challenge" is how to switch off EA during ranging times when EA isn't performance well?  Hence, which performance metric has the "best" or most appropriate "predictive' factor?


We tend to choose the "best" performance metric to "turn on" EA.  In my opinion, choosing Best performance metric should be done in Phase 1 as part of EA selection process. 

Imagine, using a Breakout EA, we wait for a few days of ranging just to get 1 Breakout.  So by the time I collected back the results that the EA is "making" money and hence win/ratio went up, I already missed out the Breakout and I'll will then be turning it On (after the Breakout period and that mean, I'm actually turn On my EA during Ranging, which is the very period I should be avoid trading, i.e. Turn off my EA.

So the question, is which performance metric is a "best" metric that may "indicative" that my EA is going to start making money?  I think Max DD or turn ON EA when it has lose a number of trades, Consecutive number of loses  (during ranging times, would be a better predictive indication).  Eg Turn ON EA when EA reached Max number of consecutive loses (during ranging time, may signify that Breakout is going to happen next).

I think we tend to mix up Phase 1 and Phase 2 Functionality of performance metric.  Selection of EA for Portfolio and Criteria for Turn on/off EA are 2 different Phases we need to consider them as separative process.  Don't use Phase 1 as decision marker for Phase 2.  Cos for Phase 2 to be effective, we need to see which performance metric has "preventive and predictive" value.

11 (edited by sleytus 2018-02-24 12:58:32)

Re: When would you have stopped this robots?

what is the probability for a trade to succeed if W/L ratio for previous n trades is above x threshold?

Ahhh -- good question.  I get to use my favorite answer -- "I don't know -- it depends"...

I don't have any statistics on this -- this is something that remains to be evaluated and, hopefully, the more people who experiment with it and share results then something interesting will fall out.  My guess is this will vary according to your strategies, trading approach and the market.  I prefer empirical over theoretical, and I it occurred to me the demo account could be used to provide a clue as to how a strategy was trading in the current market.  At this point I just want to distinguish between strategies that are total losers and those that are not.  This is also much simpler than constantly pruning and allows for a strategy to begin working again if it likes the data -- which wouldn't happen if you solely relied on pruning.

This is all new.  And with time, if people share their observations then I can include more "policies".  However, to be honest, I'm not really interested in people suggesting that I try this or that.  I have better things to do with my time.  However, if people do their own testing and can show some results where a particular policy tends to help their results then, sure, of course I'd be interested in giving it a try.  I leave the theoretical to Popov's software, and after that I'm only interested in empirical -- i.e. real testing.  I've noticed people tend not to like to do their own testing because it takes time and you have to be a good detective.  I don't want to fall into the trap of being the guinea pig who tests everyone else's ideas.  As you know, coding does take time. 

Another policy I've added (besides WinRatio) is "consecutive wins".  I've noticed that the majority of strategies exhibit streaks of winning and losing.  I suspect this is due to the fact that strategies are better fitted to certain data patterns compared to others.  And as the market pattern changes, then strategies will perform better or worse.  So, if a strategy has two consecutive wins in the demo account then it gets a green light to trade in the live account -- otherwise it is paused (until it again gets two consecutive wins).  Very simple -- but it matches the data and increases the overall profitability.  Though you miss out on a couple of winners, you frequently also avoid a string of losers.

As long as I'm hijacking, one last point.  Just as there is a "disconnect" between back testing and demo accounts, there is also a disconnect between demo and live accounts.  There is also a disconnect between demo accounts from different brokers.  I don't think it is due to anything dishonest, just that they each are using different data sources and probably add a little magic sauce to fit their business model.  So, using a demo account is not a substitute for creating good strategies -- that should still be the #1 priority.  But this new app can provide some protection if you happen to have a few real losers hidden in your portfolio EA and, also, make your good strategies a bit more profitable.  So -- at worst, no harm is done.  And, at best, profitability can be increased.