1 Topic by Alann (edited by Alann 2023-10-04 17:38:05)

  • Alann
  • Licensed Member
  • Offline
  • Registered: 2023-02-02
  • Posts: 109

Topic: EA Studio vs FSBpro Complete Opposite Analysis

I have loaded an EA created by EA Studio into EA Studio and into FDBpro.  I have not used any filtering such as Acceptance Criteria or Monte Carlo or optimizations.  No filtering at all.  Both use Premium Data and Symbol settings are the same.  No commissions.  Date is the same for both.  When I look at the balance line of EA Studio I see a positive slope, when I look at the balance line of FSBpro I see a negative slope.  I do not know what I am doing wrong.  I have examined inputs for both generators and I do not see any differences I might have made in order to produce two complete opposite balance lines.  I know both generators are reliable.  I must be doing something wrong but I cannot figure it out!


https://i.ibb.co/MSP55wx/EA-STUDIO-EURGBP-M15-Screenshot-2023-10-04-104936.png

https://i.ibb.co/202TLGL/FSBPRO-EURGBP-M15-Screenshot-2023-10-04-104748.png

  • footon
  • footon
  • ◄≡≡≡►
  • Offline
  • Registered: 2009-08-17
  • Posts: 2,969

Re: EA Studio vs FSBpro Complete Opposite Analysis

There seems to be difference in trade count, can you confirm?

  • Alann
  • Licensed Member
  • Offline
  • Registered: 2023-02-02
  • Posts: 109

Re: EA Studio vs FSBpro Complete Opposite Analysis

EA Studio count of trades = 25
FSBpro executed orders = 233

How can there be such a difference and why would the difference in the number of trades result in such a massive difference since the start and end dates are the same?  Which is correct, EA Studio or FSBpro?  How do I justify (show they are the same) the two generators?

  • footon
  • footon
  • ◄≡≡≡►
  • Offline
  • Registered: 2009-08-17
  • Posts: 2,969

Re: EA Studio vs FSBpro Complete Opposite Analysis

Executed order number I can see from the screenshot but it's not equal to count of trades. Go to Overview and find the count for Number of trades line in the stat part of the overview.

There is no need for justification, something is off.

If you open Journal in both programs, does the first trade opening time match for both?

Check if indicator parameters are equal in both programs.

If parameters are equal, opening trade is equal, then the problem is probably in one of the indicators, most likely in the closing one, Candle Color. I don't have access to Studio indis, it needs to be checked by Miroslav when he comes back.

5 Reply by Alann (edited by Alann 2023-10-05 00:54:51)

  • Alann
  • Licensed Member
  • Offline
  • Registered: 2023-02-02
  • Posts: 109

Re: EA Studio vs FSBpro Complete Opposite Analysis

FSBpro Number of Trades = 115
First trade for both generators is the same.
Indicators have the same parameters.

  • footon
  • footon
  • ◄≡≡≡►
  • Offline
  • Registered: 2009-08-17
  • Posts: 2,969

Re: EA Studio vs FSBpro Complete Opposite Analysis

Ok, I hope Miroslav saw this, Studio's Candle Color needs to be checked in the first order.

  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 7,988

Re: EA Studio vs FSBpro Complete Opposite Analysis

I'm working on major releases of EA Studio and Express Generator now.
I hope to upload them during the weekend.
Then I'll check this issue.

Please upload the strategy to be able to test it.

Popov's Website

  • Alann
  • Licensed Member
  • Offline
  • Registered: 2023-02-02
  • Posts: 109

Re: EA Studio vs FSBpro Complete Opposite Analysis

Strategy uploaded...

Post's attachments

EA Studio EURGBP M15 37535320.mq4 21.03 kb, 10 downloads since 2023-10-05 

You don't have the permssions to download the attachments of this post.
  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 7,988

Re: EA Studio vs FSBpro Complete Opposite Analysis

Thank you for uploading the strategy!

The main reason for the issue of the overestimated result in EA Studio is not a bug in the indicators or in the backtester.
The signal will be perfectly fine if you check it on the chart.

Your strategy uses Bollinger Bands and trades when the price crosses the borders. It is fine as an idea. However, what we don't see is that the spread can be up to 10 times higher in such cases.

I noticed it recently when I worked with MT5. It uses something called "real spread". It is not exactly "real" because MT5 stores and provides it in a strange way. Anyway, it is dynamic and makes a big difference in the backtest.

I'm working at the moment on providing a "real" spread in the Premium Data and also making EA Studio and Express Generator considering it during the backtest.

I'll upload new releases of EA Studio and Express Generator on Sunday.

Now I'm re-compiling the complete Premium Data set from Dukascopy ticks. It is a very slow process and will take time.
For now, I patched the data series with a pseudo "real spread" from MT, but later, I'll find a better solution.

Popov's Website

  • footon
  • footon
  • ◄≡≡≡►
  • Offline
  • Registered: 2009-08-17
  • Posts: 2,969

Re: EA Studio vs FSBpro Complete Opposite Analysis

Do I get it right that Studio uses dynamic spread, which is higher and that induced overestimated backtest result?

And then why the trade count doesn't match?

  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 7,988

Re: EA Studio vs FSBpro Complete Opposite Analysis

EA Studio will use dynamic spread from the next release.
Currently, EA Studio, FSB Pro, and ExGen use fixed spread.
This causes discrepancies with the actual trading for this strategy because it trades when the peace crosses the BB bands.
It appears that the real spread is much higher in such occurrences.

There was a posted strategy several weeks ago that showed a smooth rising line in all markets (I cannot find the topic now). I'm pretty sure it exploits this issue. If you find it, please send me a link to test it.

> And then why the trade count doesn't match?

I'll inspect that issue next week in detail.

Popov's Website

12 Reply by Alann (edited by Alann 2023-10-14 19:55:15)

  • Alann
  • Licensed Member
  • Offline
  • Registered: 2023-02-02
  • Posts: 109

Re: EA Studio vs FSBpro Complete Opposite Analysis

I have again loaded an EA (same link as above) created by EA Studio into EA Studio and into FSBpro.  I have not used any filtering such as Acceptance Criteria or Monte Carlo or optimizations.  No filtering at all.  Both use Premium Data and Symbol settings are the same.  No commissions.  Date is the same for both.  When I look at the balance line of EA Studio I see a  large draw down followed by a rally, when I look at the balance line of FSBpro I see a downward trend.  I would expect these to testers to have similar backtest.  What is going on?

EA Studio count of trades = 19
FSBpro executed orders  = 112


https://i.ibb.co/CnvrNMb/EA-Studio-Screenshot-2023-10-14-132422.png

https://i.ibb.co/nbp2Qjx/FSBpro-Screenshot-2023-10-14-132603.png

  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 7,988

Re: EA Studio vs FSBpro Complete Opposite Analysis

The noted discrepancy results from a bug in FSB Pro introduced in the last release. It fails to set the symbols' "Pip" property.
The "Cancel Color" indicator uses this property to calculate the minimal candle height.

I'll release a fix on Sunday evening.

Thank you for the report!

Popov's Website

  • Alann
  • Licensed Member
  • Offline
  • Registered: 2023-02-02
  • Posts: 109

Re: EA Studio vs FSBpro Complete Opposite Analysis

How do I update FSBpro with the new fix?

  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 7,988

Re: EA Studio vs FSBpro Complete Opposite Analysis

I just uploaded the new FSB Pro v4.3

It will be automatically updated on the next start (however, it checks for updates once per 24 hours)

You can also download and install the new version over your current installation.

Popov's Website

  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 7,988

Re: EA Studio vs FSBpro Complete Opposite Analysis

It looks like the bug still persists in the released FSB Pro version.

I'll examine it further.

Edit.

FSB Pro still does not set the Pip value correctly for the Premium Data feed. I'll fix it later today.

Popov's Website

  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 7,988

Re: EA Studio vs FSBpro Complete Opposite Analysis

The issue is fixed in FSB Pro v4.3.1.

Popov's Website

18 Reply by Alann (edited by Alann 2023-10-17 13:24:45)

  • Alann
  • Licensed Member
  • Offline
  • Registered: 2023-02-02
  • Posts: 109

Re: EA Studio vs FSBpro Complete Opposite Analysis

Again I have loaded an EA (same link as above) created by EA Studio into EA Studio and into FSBpro.  I have not used any filtering such as Acceptance Criteria or Monte Carlo or optimizations.  No filtering at all.  Both use Premium Data and Symbol settings are the same.  No commissions.  Date is the same for both.  When I look at the balance line of EA Studio I see a  large draw down followed by a rally, when I look at the balance line of FSBpro I see a nice upward trend.  I still see quite a difference in the two balance lines and the final profit is also quite different.  I would have expected the results of both these backtests to be similar.

Which is the more accurate, EA Studio or FSBpro.

EA Studio count of trades = 18
FSBpro executed orders  =  24

Note:  FSBpro did not update its version number when it was launched.  I had to install the latest version over it to get the latest version.


https://i.ibb.co/sHfrG1r/EA-STUDIO-EURGBP-M15-Screenshot-2023-10-17-061010.png
https://i.ibb.co/vYdXZG1/FSBPRO-EURGBP-M15-Screenshot-2023-10-17-060807.png

  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 7,988

Re: EA Studio vs FSBpro Complete Opposite Analysis

EA Studio closes the positions at the next bar's Open. On the other hand, FSB Pro closes them at the same bar's Clase price.

This execution difference causes different results.

Another difference is that EA Studio uses Dynamic Spread by default.
You can switch it off from Tools -> Settings. The option is called "Use dynamic spread".


Besides these differences, I think everything is fine in the latest versions.

FSB Pro exports its own experts that work correctly with its logic.
EA Studio exports other experts who resemble its logic.

Both programs work similarly but not in the absolutely same way.

We have the same count of trades, the same chart shape and a very similar performance (dough Dynamic spread is off)

https://image-holder.forexsb.com/store/eas-vs-fsb-nov-2023-foo-bar-thumb.png

EA Studio is more reliable in that case because it can use a Dynamic spread and shows a more realistic result.

Popov's Website

  • Alann
  • Licensed Member
  • Offline
  • Registered: 2023-02-02
  • Posts: 109

Re: EA Studio vs FSBpro Complete Opposite Analysis

I guess I am having issues with EA Studio.  Here is the version number: Expert Advisor Studio (EA Studio)  v23.10.12.

The image of EA Studio above where the balance line shows big drawdown followed by a rally is with the Dynamic Spread turned off.  Is this the correct balance line with the Dynamic Spread turned off?

Also when I turn Dynamic Spread "on" and go to a different tab and then come back to Tools the Dynamic Spread check box is again unchecked.  It will not remain checked.  Because of this I cannot view the balance line with the Dynamic Spread enabled.  What am I doing wrong?

  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 7,988

Re: EA Studio vs FSBpro Complete Opposite Analysis

Reload EA Studio with Ctrl+F5. The latest version is "v23.10.17".

Popov's Website

  • Alann
  • Licensed Member
  • Offline
  • Registered: 2023-02-02
  • Posts: 109

Re: EA Studio vs FSBpro Complete Opposite Analysis

Everything seems to be working now!

If I understand you correctly the balance line in EA Studio when using Dynamic Spread is a more realistic than the balance line in FSBpro.

Thanks for being very responsive to resolving this issue and in answering all my questions quickly.

Alan,

  • Alann
  • Licensed Member
  • Offline
  • Registered: 2023-02-02
  • Posts: 109

Re: EA Studio vs FSBpro Complete Opposite Analysis

I guess I am still a little confused and I hope you can help me to see clearly.

I understand that in EA Studio if I use the Dynamic Spread option then the balance line will be more realistic than without the Dynamic Spread option box checked.  However, when I leave the Dynamic Spread option box unchecked then the EA created by EA Studio has a similar balance line as to the balance line in FSBpro when I upload the same EA into FSBpro.  So in this case I understand that neither EA Studio or FSBpro balance lines are accurate.

Now here is where I get confused:  When I upload the EA that was created in EA Studio is uploaded into FSBpro I understand the balance line is not the most accurate it looks the same as the balance line in EA Studio when the Dynamic Spread box is unchecked.  If this is the case then when I create an EA using FSBpro how can I feel confident its balance line is accurate?  This is confusing to me!

  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 7,988

Re: EA Studio vs FSBpro Complete Opposite Analysis

Nothing changes in the workflow.

My usual procedure is the following:
- examine the trading rules for contradictions. (Example: close long when RSI crosses 90 upwards.)
- examine strategies with a too-good-to-be-true balance curve even further.
- test the strategy in MetaTrader. (now prefer MT5). If there is a too-big discrepancy, the strategy is questionable.
- demo or live trading with a minimal amount for three weeks. Acceptable performance is the most critical factor.

We never know which strategies will perform best in the market.
I have many cases when I create a simple strategy to test some behaviour, which performs amazingly well.

However, recently, I changed my approach. I prefer a pool of thousands of strategies per chosen market and revalidate them with Express Generator on newer data.

Popov's Website

  • footon
  • footon
  • ◄≡≡≡►
  • Offline
  • Registered: 2009-08-17
  • Posts: 2,969

Re: EA Studio vs FSBpro Complete Opposite Analysis

At first sight it might add to confusion, but in reality it underlines a very important factor of spread. The balance line is accurate, the question is if it represents a realistic outcome of actual market conditions taking into consideration the changing value of spread. One way or another it is better to use higher spread value.

This site offers good insight into spread history: https://www.oanda.com/us-en/trading/historical-spreads/


Alann wrote:

I guess I am still a little confused and I hope you can help me to see clearly.

I understand that in EA Studio if I use the Dynamic Spread option then the balance line will be more realistic than without the Dynamic Spread option box checked.  However, when I leave the Dynamic Spread option box unchecked then the EA created by EA Studio has a similar balance line as to the balance line in FSBpro when I upload the same EA into FSBpro.  So in this case I understand that neither EA Studio or FSBpro balance lines are accurate.

Now here is where I get confused:  When I upload the EA that was created in EA Studio is uploaded into FSBpro I understand the balance line is not the most accurate it looks the same as the balance line in EA Studio when the Dynamic Spread box is unchecked.  If this is the case then when I create an EA using FSBpro how can I feel confident its balance line is accurate?  This is confusing to me!