• Registered: 2017-05-06
  • Posts: 51

Topic: Change in market conditions

Hi all,
please contribute to this thread with your ideas.

My results over the last two weeks have been bad. Last year I built many EA Studio strategies most of them with a very high winning rate (80-9x% win rate). They have worked quite well until last weeks. I understand to build 80% and higher winning rate strategies that needs (usually) a TP considerably smaller than the SL, and these strategies work better in range periods or they are usually "mean reversion" strategies or "hit and go" strategies and so on. But, as during the last weeks very strong movements in USD pairs have appeared, many of my EA Studio high winning rate strategies have gone bad. I think this can be considered as normal sad

I wanted to ask you all what you think about this, how you deal with these situations and how were your results last weeks.

For example, I guess if Steve's Sidekick users have also high winning rate strategies you might also have experienced similar bad results, although probably better than mine because of the stop trading in the real account once a loss happens and wait for a new winning streak to start trading the real account again. But, please can you post if your results have been bad too?

In the end I guess what happenned last weeks is simply that market conditions changed and thats why the losses appeared. For that reason I always think about detecting market conditions and using EAs adapted to those market conditions. But I think that the strategies in EA Studio, being so really simple, cannot be built on a specific market regime. On the contrary, in Forex Strategy Builder Pro it is a lot easier to build strategies when a higher timeframe (for example) is doing something. So what do you think? Are you able to build in EA studio strategies well adapted (only trading) to specific market regimes or you have experienced something similar: big movements and new trends make most of your EA Studio strategies fail once and again. We are building mainly mean reversion strategies failing with strong trends appear?. Can you build strategies in EA Studio that work the same in one type of market and other and how do you do that? I mean, maybe I could use some specific settings in EA Studio (for example, not TP and SL very different or so on). How are you dealing with this?

oF course my results could have been been better if I had a better balanced portfolio and not so many high winning rate strategies (small TP and big SL). How are you building your portfolio? Do you try to build different kind of strategies to have a balanced portffolio between trend strategies and mean reversion strategies? Is there a way to know if we have a balanced portfolio between different type of strategies?

I usually like high winning rate strategies because it is easy to notice if they are working well. On the contrary trend following strategies are a lot more difficult to ffilter out and determine if they could work in the future although they are failing now. So how do you work with trend following strategies: can you give us some advice on how to build them and when to remove themo from the real account?

Thanks a lot for your contributions all of them are of help
Regards

2 Reply by hannahis (edited by hannahis 2018-05-06 13:17:06)

  • Registered: 2010-11-18
  • Posts: 1,179

Re: Change in market conditions

Hi rjectweb,

Yes past two weeks trading has been bad (generally speaking, even when I'm not using mean reversal trading strategies).  In fact for a trend following trader, the past 6-8 weeks (starting in Feb 25) has been rather tough times to trade.bbIf you look at the monthly chart, the past 2- 6 weeks (you see a rather flat market/price curve) are a long over due correction and hence, the movement was huge (since it was a long over due market correction). 

So don't be discouraged by your EA past 2 weeks results cos I also have a number of trending following EA that did badly too.  Try to find the "optimum" SL setting whereby you know by a certain amount of pips movements, you know your EA has already made a wrong "guess work" so by a certain amount of pips (mine is about 35-50 pips), my EA will have SL to cut the lose and quickly join in the "correct" trade direction/trend.

However, generally I'm not a person who favour mean reversal EA, it's not my cup of tea and I find the potential risk out weigh the reward (and plus I'm not good and I don't have a clear understanding in spotting a "reliable" mean reversal signal).

  • Registered: 2017-05-06
  • Posts: 51

Re: Change in market conditions

Hi hannahis,
thanks a lot for your contribution.

Today EURUSD continued sinking and again any attempt of going long (buy EURUSD) was a difficult task in such a strong SELL-trend. I don't want to bother anyone here with my ideas about market regime detection and sorry for my last posts being so long. I will try to summarize my ideas here:

- EA Studio is a wonderful tool. I have licenses for almost all the trading strategies builders but I almost only use EA Studio. EA Studio is faster than the others and it has the reactor (batch processing - sequence of optimization-montecarlo and multimarket) and it has the portfolio feature. It is great!

- However with EA Studio I cannot build strategies that take into account higher timeframes and therefore it is not easy to try to build strategies more adapted to specific market conditions. That is usually not a problem, but sometimes (these weeks for example) it is so clear that there is such strong trends in USD pairs that I would really like to have some filters to stop trading in these market conditions. With such a strong sell trend, one should't buy (or not trade). My point here is I would like EA Studio to have a few more features to detect this situations like FSB Pro, for example, or more likely I would like if FSB Pro had the greatest features from EA Studio (increased reliability and better GUI, batch processing for Optimization and Montecarlo, POrtfolio feature, etc).

- EA Studio doesn't have much indicators to build strategies on. They seem to be enough tough.
But I (this is only my fault) end up building the same type of strategies that usually work for me. These are usually strategies with smaller TP than SL that have a really high winning rate and for that reason they are usually scalpers or reversal strategies or mean reversal strategies. Those are great for ranging periods or not so strong trends periods, but these days with such storng trends they are (obviously) working bad. My point here is that my portfolios are not well balanced and for that reason I end up winning a lot in ranges and losing a lot in trends. It would be great if we could have a way to determine if a portfolio is well balanced. Correlation in EA Studio helps a lot, but it seems to be not enough. I miss something to help us build better balanced portfolios, mixing different kinds of strategies (trending, breakouts, scalpers, etc).

Overall I'm more than happy with EA Studio, but these days being so bad make me question how I must refine my metodology and build more robust portfolios to try to avoid weeks like these.

Warmest regards
RJ

  • Registered: 2017-03-14
  • Posts: 709

Re: Change in market conditions

Recently Popov added "Long-only" and "Short-only" indicators to EA Studio.  Could those be used to better tailor strategies to specific market conditions?

I did try those a few weeks back in EA Studio, but these days I'm mostly using FSBPro -- so, I can not confirm this from much experience.

5 Reply by hannahis (edited by hannahis 2018-05-09 06:17:34)

  • Registered: 2010-11-18
  • Posts: 1,179

Re: Change in market conditions

Rjectweb,

1. You can "compensate" or subsitute the lack of Longer Time Frame with higher parameters.

For eg. MA 10 for H1 chart is the same as MA 20 in a M30 chart and MA 40 in a M15 chart. 

I understand that there are some indicators that will work differently if you try to convert them into different time chart. 


2. Current Market Situation: The problem you faced in EA studio, you will face the same problem in FSB Pro too

The current market situation is not just an issue of a strong trend (because a strong trend is very easy to spot and hence easy to trade), it's an issue of Short Term Trend vs Long Term Trend. 

Your Short Term Trend will give you false signals that the trend is ready to change and hence you ended up with multiple wrong direction trades because the Long Term Trend forces the short term trend to be "short lived" and thus many of short term trades ended up in SL.

If you are trading in any time Chart D1 or below or you are using very sensitive parameters in your EA, there going to be lots of False Signals during this period.  So even if you use FSB Pro, you can't really "solve" this issue unless you are using very large parameters which works well in this situation because it is so laggy, it won't "mis-fire" or send out false signal.

The problem you are facing is a very common and fundamental problem most traders (in my limited opinion) faced.  If we want responsive EA, we get lots of false signal but if we want stable/laggy EA, we miss out lots of pips to enter later to ensure we get the "correct" confirmation of a big trend and we need a huge SL because the EA is laggy to respond fast to market changes.

It's a double edged sword, you want a fast reactive EA, you get lots of false signal.  It's a delicate balance to strike between responsiveness vs stability (reliability).  Whether you are using EA Studio or FSB Pro, you can't run away from this fundamental issue.  Of cos FSB Pro has more indicators to help you tackle this issue with greater options but it is still an unavoidable problem prevalent to forex.  Whoever can resolve such issues would definitely be a great trader because the real challenge in forex trading (IMO), if not about finding winning formula.  The real challenge is how to find "non losing" formula - a formula that will prevent or cut down the amount of "wrong trades" that led to loses.

For a successful EA, you need 2 trading theories incorporate into 1. 

1.  A winning formula - tells the EA when to Trade and make money (which most of us have, it's the easiest part)

2. A non losing formula - prevents the EA from losing, tell the EA when not to trade to avoid loses.

Most will agree with me, if only we can stop our EA from losing money, we are winning all the time cos we get to keep our profit (and not lose it).  Isn't that so ideal?

After all these years, my biggest challenge is how to find the correct indicators to tell the EA when not to trade.  If I can stop my EA from losing money, that would be my dream come true.

So if anyone (kind souls) have an brilliant suggestions or ideas to contribute whereby you have found an effective indicators to help you manage when not to trade.  I'll be very grateful to hear from you.


3. A Good and timely exit?

A short term EA is "supposed" to be good in whatever market situations, the challenge is the entry and exit accuracy. 

If one can have a timely entry and exit, then it can ride through all sorts of trends/waves.  Hence, it is very important to use the correct time chart for such EA, thus explained why I always stick to M1 time chart.

  • Registered: 2017-05-06
  • Posts: 51

Re: Change in market conditions

Hi hannahis and sleytus, thanks a lot for your contributions. I learn a lot reading opininions and personal experiences.

Please let me continue with this interesting debate

1. You can "compensate" or subsitute the lack of Longer Time Frame with higher parameters.

Well, this is completely true in general. However, I cann't do this with EA Studio (or at least I don't know how I could). In EA Studio all what we can do is select indicators and general rules for strategy creation, but we cannot use a template to build a special type of strategy the way we do in FSB. EA Studio is extremely fast and simple to use, but on the contrary it lacks the flexibility of FSB pro when we want to create some kind of stregies.

In other posts I read you (hannahis) often start your strategies with a predfined idea in mind of how your strategies must perform (a template, locking indicators and so on). This is not possible in EA Studio. EA Studio is great because it is soooo fast. But it is pure datamining (brute-force). I'd like FSB to be as fast as EA Studio, and EA Studio to be more flexible, like FSB.

2. Current Market Situation: The problem you faced in EA studio, you will face the same problem in FSB Pro too

Yes, you are right. And thanks a lot for your ideas here. However, in FSB pro I can at least imagine ways of trying to avoid this situations. For example, I can define a strategy to only trade LONG if MACD in the daily timeframe is bigger than zero, for example. And I can imagine setting this filter before I built the strategy or even after I have the strategy, and this way trying to avoid the periods where I have a DOWN TREND (not to buy when MACD<0 in daily TF). That's only and example.

Thank you for your opinions on the long and short trends. My experience is exactly the same. Although I've usually called Strong or Weak Trends, the idea I think is basically the same. On strong UP or DOWN trends (long trends) trading against the trend is very difficult. It would be better not to trade in this situations (especially for reversal strategies). In the past, when I built my own strategies based on my own rules I used to use the angle of a moving average (ej. angle of a TMA) to determine the strength of a trend. If the trend was so strong I stopped trading. Adding such a filter on a manually generated EA improved the performance because the EA didn't trade when there was a very strong trend. If I have the time I'll try to add a similar filter to a portfolio EA generated in EA Studio. The idea is, ok I have a portfolio EA and it is basically as a simple EA, I can manually edit the portfolio EA file to filter out the trades in strong trend conditions. The portfolio EA generated with EA studio is so good that I can even backtest it in MT4, with and without my additional filter smile


3. A Good and timely exit?...

I do not much about this topic. I've read the posts about exit and open accuracy and it wouud be great to improve this aspects of my EAs. However I wanted to think about this:

A short term EA is "supposed" to be good in whatever market situations...

Well, this is key. I mean, I use H4 timeframe to build many of my strategies using many years of historic data (my brokers'). So I build my strategies across many different market regimes and situations. I've optimized my strategy to be robust enough to have been profitable for the last 9-10 years. So these strategies are ready (should be ready) to survive (even with some losses) all the market conditions. My point here is, ok I already have a good strategy that overall performs well, wouldn't it be better and lose less if I had an additional filter over it to make it not to trade in the worst market conditions?...
I'm not talking about building the strategy with the additional filter, I'm talking about adding the filter later to make the EA not to trade against the stronger trends, for example...

Thanks a lot for your help!
Warmest regards
RJ

7 Reply by geektrader (edited by geektrader 2018-05-09 20:17:12)

  • From: Germany
  • Registered: 2010-04-05
  • Posts: 839

Re: Change in market conditions

Hi Guys,

just wanted to add that I can confirm this. Up from around the start of February 2018, market movements/patterns have changed considerably in several pairs, not just EURUSD, but also GBPJPY, GPBUSD and several other ones. This is not specific to EA Studio / FSB Pro, as I am also using pretty much all the other "Builders" out there (won´t name them here, but you guys know them for sure). You can also clearly see that almost everyone out there is affected by this (of course), if you look around the top performers at Darwinex, MyFxBook, etc., they all took a beating since the start of February 2018. Same goes for my friends which use neural networks and had an almost straight equity line up until February, and then.... Boom...

There is not a lot we can do about it really, whatever program we use to build the systems, whatever method we use (simple strategies with just one indicator, complicated neural networks, whatever), if there is such a hefty change in "movement-behaviour", it is impossible that the strategies will continue working under these completely new conditions. It´s just the same if they teach you how to drive in driving school and once you hit the road, all the rules have changed - you would fail as well.

I can confirm that EURUSD and most other pairs never moved like they do since February 2018, as I am using history data from 1986 to date all the time. All of my systems on EURUSD (just as an example) and from all the trading platforms I have been using to create these, have, just like yours, keep on buying into the recent EURUSD down-move several times and got stopped out heavily as the down-move continued. It´s simply not how EURUSD behaved in the past, there seldom was such a long and straight down-move without any kind of retracement and also not such a long side-ways period before, with so many "false" breakouts that also gave my systems some good beating. Normally EURUSD retraces and never moves down THAT long without any "come back", so that´s what all the systems that we´ve built for it have learned from the past, hence they are continuing to buy, buy, buy and get killed.

How I´ve been (trying!) to tackle this:

1) I am generating all my EAs with 10% OOS at the end, this period (approx 2016 to 2018) has to look good and "straight".
2) After these systems have been generated, I am testing them once again on data from only February 2018 to now and only the ones that at least made some small profit are added to my portfolio.

However, I have not been very successful with this either, as especially EURUSD, keeps on moving very strange and, as it looks, very different once again than from the period February 2018 till ~middle of April 2018. So it´s getting smashed again.

Oh my, Forex is a b*tch, right?

Good luck to everyone!

geektrader's Website

  • Registered: 2017-03-14
  • Posts: 709

Re: Change in market conditions

Excellent post -- thanks...

I experienced a similar downfall with EURUSD around the beginning of April that seemed totally out of the blue.  Your explanation and observations help a lot.

9 Reply by hannahis (edited by hannahis 2018-05-10 05:43:46)

  • Registered: 2010-11-18
  • Posts: 1,179

Re: Change in market conditions

Hi Geektrader,

Thanks for your sharing.  Reassuring to know that I'm not the only one that took all the beating smile

Your explanation helps many others to understand when not to give up so easily cos without such explanation, many would think that there is something "wrong" with their EA (or something wrong with FSB/Studio) and kept throwing away EA and searching new ones only to end up with the same issues all over again.

Unless we try a different method of approach to search or build new EA, our old method won't be sufficient to tackle such "odd" market.  Same input = same output (outcome).

Thanks for sharing some insight on how you intended to tackle the issue.  Hopefully one day you will find an effectively and workable solution that you would shed some insight to us here so that we can tackle and learn from one another.

Wishing you the best of luck to tame the forex beast (Interesting: guys call forex B*tch and gal call forex beast.  Gender bias?)


Regards
Hannah

  • From: Germany
  • Registered: 2010-04-05
  • Posts: 839

Re: Change in market conditions

Haha, yea, gender bias indeed :-) I don´t mind, haha!

You are most welcome and also thanks for sharing your feedback. It´s obvious that everyone is affected, just see the (previous months) top-performers and how they´ve done since end of February - they also either have no trades or are in drawdown. Oh well...

So how do you tackle the current market? Just sit it out? I remember that I did read that you are using strategies with tons of indicators (you said that you are thankful that Popov did increase the indicator limit in FSB to 26 or something). Are you really using that many indicators? Are these strategies working well? I´d fear overfitting with so many indicators (and very few trades to have statistical confirmation?).

Thank you :-)

geektrader's Website

11 Reply by hannahis (edited by hannahis 2018-05-10 15:05:21)

  • Registered: 2010-11-18
  • Posts: 1,179

Re: Change in market conditions

Hi Geektrader,

There are some EA that has about 11 opening and 11 closing rules.  Basically the number of Openings are more to tackle different time frames.

However, I usually use only 2 to 3 different types of indicators.  The 11 openings are a combinations of 2 or 3 indicators in different time frames (not 11 different types of indicators).

Are these strategies working well?  Yes for some and no for some. 

Those that uses multi time frames to tackle ranging times did better (less false signals) but those that are "simple" tend to do badly in ranging times.  I strongly believe that Multi Time Frames EA are more stable and now I'm need to find a better way to handle the ranging times to improve my results even further. 

I think a good EA need to utilize different types of indicators to tackle different aspects of trading.  So besides using multi time frames to monitor different aspect of trading in the spectrum of time.  We need to monitor trading in different spectrum of Price, Volume and Movements/momentum.  Avoid using too many indicators that falls under the same category cos they are effectively measuring the same spectrum/thing. 

1) Indicators that monitor Price movements such as moving averages, oscillators etc
2) Indicators that monitor Volumes
3) Indicators that monitor Price changes, such as Std Dev, ROC, Cum Sum etc

Generally, I understand the use of my indicators, so I don't have issue with over -fitting, although there are some (small) incidents whereby I filter so much, hardly any trade.  But in most cases, I still need to filter more to cut down those multiple entries during ranging times.  So it's still more the case of over trading rather than under trading.

I've some good EA that did well during these 2 weeks (these EA have been tested for past 2-6mths).  Below are just 2 week's performance results (the rest in excel file).  Of cos there are tons of other EA that did badly during these period and I'll usually add in more filters to these "bad" EA to improve them further (if there are worth keeping).  Can't wait to finally put together those good EA into a reasonable trading portfolio.  Just when I was about to go live, this market situation arise (which I am thankful, I haven't gone live yet during this period.  But I usually like such market situations because I rather my EA die now in demo account then let it kill my live account and it's a good "training" for me to develop better EA and at the same time help me in my EA selection process better, results from these few weeks will make it easier for me to know which EA to eliminate).

https://s31.postimg.cc/xi4fb7k7b/4_B.png
results 29 April - May 10.

  • From: Germany
  • Registered: 2010-04-05
  • Posts: 839

Re: Change in market conditions

Hi Hannahis,

thanks for the information and very interesting. Are these all systems generated by either FSB / EA Studio or also by the other platforms (since you are on the forums there too, I won´t mention the names)? What´s your observation which systems by which platform perform the best in general? I am very impressed by the really simple ones (1 indicator IN / 1 indicator OUT) that EA Studio finds and how well they often perform even over 32 years on H4 (at least in the backtest / 10% OOS so far) and will have to explore those further, especially since EA Studio is, by far (if using the fastest browser for your system of course), the fastest Builder I have.

Also, what timeframe are you using? I´ve found that H4 performs the best in all pairs in general with acceptance rate in EA Studio being at 20%. If I switch to H1, using the same acceptance criteria, acceptance rate drops to ~11%, which confirms that H4 is better. The daily timeframe has a lower acceptance rate once again too than H4 does, so H4 seems to be the best choice. That´s also what I´ve found with most other systems builders (including FSB Pro).

As for your portfolio, the ones listed look very nice, that´s great! Are you overall up or down for the last 2 months with the changing markets? Just wondering.

Cheers :-)

geektrader's Website

13 Reply by hannahis (edited by hannahis 2018-05-12 09:29:49)

  • Registered: 2010-11-18
  • Posts: 1,179

Re: Change in market conditions

Hi Geektrader,

I didnt' use the other software cos it's still in beta version.  The current (old version) doesn't allow me to use FSB Pro EA as their custom indicator whereas the new version (still in beta version) allows me to use FSB EA as custom indicator, so I'm still waiting for it to be ready so that I can "export" my FSB EA (as custom indicators) to see whether it can improve my system because it has tick backtest ability.


I don't use EA studio (ES) because I need multi time frames and also the indicators in ES are (limited, doesn't have what I'm looking for).

Maybe one day after I've finished up my FSB Pro projects, I'll give ES a try.  Can you elaborate more how you use ES to build your EA?  And what can you do to ensure a better quality search?  What's the key to successful ES usage?  Is the use of long historical data a key to ES success? or Is it the choice of indicators? or is it the OOS allocation etc?  What advise would you give to me with regard to the use of ES?

As for FSB Pro...

All the EA are build on 1min time charts

And since I use Multi Time Frames, I've a number of time frames in use.

Generally I use D1 or H4 to identify a good trend and lower Time frames to activate the trade, however since the beginning of the year, given the market irregularities, I decided to use lower time frames such as H1, M30 and M15 to tackle the market.

As I've mentioned, "short term EA supposed to do well in all sorts of market situation", so with this in mind, I plan also to develop what I call "universal EA" an EA that can do well in both short term and long term market situations.  If the trend is strong, ride along with it, if it is short trend, exit fast when the trend ends.  It's tough to build such EA cos entry/exit accuracy is the key to success and eliminating false signals is the other road block but it's rewarding once I can finally get it right (still in the process).  So it seems like H1, M30 and M15 are rather a good fit for such EA.   

Furthermore, now FSB Pro Optimizing function have improved tremendously and there are a number of Optimized EA that did really well.  So I would usually choose some of my good EA (as template) and then use the optmizer to fine-tune and generate some good EA.  Interestingly, it seem like those optimized EA with 3-5mths are doing better than those 1 to 3 years.  I haven't use any longer historical data like 32 years yet!  I need to do more sample to really find the good "optimal" historical data to use.

As for my portfolio....

I'm now testing thousands of EA in demo test, cos FSB Pro backtest doesn't seem to give me an accurate "picture" of how well my EA can do...maybe becos my EA uses M1 and other multi time frames and also I use an external Trailing EA for all my EA, as an alternative method for dynamic closing options (instead of just relying on my EA closing conditions), thus with this external intervention (from the trailing EA), I definitely can't rely on FSB Pro backtest for my results (no fault of FSB, since it doesn't give us Additional alternative trailing closing option).

Since I'm demo testing thousands of EA (in 45 MT4 terminals, made only possible, thanks to Sleytus/Steve Portfolio Maker) hence, these EA aren't in any portfolio, their results are all lump together with good and bad EA and I use Fxblue to see each individual results to do my selection, thus I ignore the overall account gain or loses.


As for some good EA that I took out and put them into a portfolio....

The past 1 mth results are down by 3-5%.  It was up by 90% initially (the 1st 2 weeks) but the past 2 weeks, market was so crazy that it dropped from 90% to -3%.  That's about 93% dropped in these 2 weeks....really bad.

I've another portfolio (4.5mths old), overall 9.79% gain (dropped 9.43% in March and 1.69% in month of May).

Now, I'm waiting for my next batch of EA to be ready (after another 2-3mths of demo testing).  Hopefully this new and improved batch will have better results.  Based on the past 2 weeks trading, there are a number of EA that made money despite tough trading situation, so this give me great hope.  keeping my finger crossed as for now.