51 (edited by hannahis 2017-12-29 05:23:44)

Re: Using MyFxBook to analyse strategy entry and exit accuracy

Footon,

Thanks for the steps.  Now I can view the stats

1. What's the rationale behind the Total Accuracy, what's the "ideal" or reasonable value to obtain? 1 is the minimum?

2. When I tried to input my opening conditions and observe how it affects the Entry/Exit Accuracy, I noticed that when I input an opening condition and just by changing the Base Price, I sometimes get NaN% and when I keep adding in new opening conditions, I still get NaN% but if I happened to changed the Base Price in one of these conditions, sometimes there will be values shown instead (I'm using Close and Reverse as closing). 

So my question is 2a) What cause NaN% and

2b) How come with the same opening conditions, if I were to change the Base Price, the value change from NaN% to some numeric values?  I understand that changing Base Price affect strategy's trading outcome, I would have expected the numeric to change in values not from "non existence, NaN%" to something else.  thus back to the 2a question, What cause such observation?


This Accuracy Stats are very useful for users (especially those who prefer to build their own strategy based on rules) to understand what caused our Strategy/trading theory to "fail".  Is it bad Opening conditions (Entry Accuracy Issue) or Bad Closing Conditions (Exit Accuracy Issue). 

In the past, we can only look at the Strategy's overall performance and clueless where (opening or closing) to zoom in to improve the strategy, we didn't know whether it's because we have poor Opening Conditions that cause the entry to be late in identifying a trend/breakout or enter too early (before a breakout) and thus exposing our trades to risk (price fluctuation and ultimately exit at SL).

or

Is it because we have poor Closing conditions that caused premature exit or late exit.  By examining the Accuracy Stats, we now know where lies the weaknesses/strength of our strategy.

Accuracy Stats also shows whether our trading rules (especially opening rules) have adequately reduce "false" signals and thus will resulted to higher Entry Accuracy. 

Likewise, Exit Accuracy helps us understand beside our trading rules, if we were to change our SL, TP or BE, what impact would it have on our Exit Accuracy.  Whether we have used too tight SL/TP/BE and if I were to increase my SL/TP/BE, will I improve the Exit Accuracy by preventing premature exit?  These Accuracy stat shed many important elements to our trading strategy's components that we could use it to identify areas for improvements.

In a nutshell, we want to prevent 1) False Signal that cause premature entries/exits and 2) Late Signal that cause late/slow entry/exit that reduce our profit margin.

Once we know the "weakness" of our Strategy, we can zoom in to the area that needs improvement, ie. work on improving either the opening or closing conditions. 

For example, if we know that we need to work on improving the Exit Accuracy, then we can lock the Opening Conditions to Generate better closing or Untick the Opening Conditions to Optimized the Closing Conditions.

Ideally, I wished the software will use the Accuracy criteria as a searching tool to find the optimal peak/tough rules that hopefully present better predictability in terms of finding repeatable patterns of behaviors.

Re: Using MyFxBook to analyse strategy entry and exit accuracy

I apologize for hijacking this thread and changing the topic.  footon, Blaiserboy -- could you delete my posts?
I've moved my posts to a different thread -- https://forexsb.com/forum/post/48451/#p48451

I don't like it when it happens to me -- and here I've done it to someone else.  Sorry for the distraction...

Re: Using MyFxBook to analyse strategy entry and exit accuracy

Thank you footon!

I played some time with Accuracy. Want to share some results of my recent experimental portfolio of ~40 systems and how much improvement there is. I have found that Accuracy brings some value (better results if chosen higher) to unseen data profits smile However correlation is small, but it is there. Also I did it for optimized and not optimized systems. One more time I got optimized results much better than without it.

I chose to cut down systems beforehand if accuracy will be below 50% for entry and exit, and for total if below 10%. I took these numbers from other ~40 systems portfolio experiment, because these was obvious points after plotting in the scatter charts. So this 2nd experiment is confirming it with unseen data(different portfolio) smile

https://s18.postimg.org/t59rth2ud/accuracy.png

All the systems was data mined since 2012 till 2016 - 06, and out of sample check was used till now. Used various timeframes and pairs. 0.01 lot used.

Re: Using MyFxBook to analyse strategy entry and exit accuracy

hannahis wrote:

Footon,

Thanks for the steps.  Now I can view the stats

1. What's the rationale behind the Total Accuracy, what's the "ideal" or reasonable value to obtain? 1 is the minimum?

2. When I tried to input my opening conditions and observe how it affects the Entry/Exit Accuracy, I noticed that when I input an opening condition and just by changing the Base Price, I sometimes get NaN% and when I keep adding in new opening conditions, I still get NaN% but if I happened to changed the Base Price in one of these conditions, sometimes there will be values shown instead (I'm using Close and Reverse as closing). 

So my question is 2a) What cause NaN% and

2b) How come with the same opening conditions, if I were to change the Base Price, the value change from NaN% to some numeric values?  I understand that changing Base Price affect strategy's trading outcome, I would have expected the numeric to change in values not from "non existence, NaN%" to something else.  thus back to the 2a question, What cause such observation?

1. Total Accuracy is Calculated as:

(exit price - entry price) / (maximum price seen - minimum price seen)
= (110 - 100) / (130 - 90)
= 25%
= The trade represented only 25% of the trade range

   • The formula is reversed for short

2. For the sake of efficiency, stats are calculated only for strats, which show profit.

Important bit - stats are calculated from the bar the trade enters to the bar the trade exits!

Re: Using MyFxBook to analyse strategy entry and exit accuracy

Irmantas wrote:

Thank you footon!

I played some time with Accuracy. Want to share some results of my recent experimental portfolio of ~40 systems and how much improvement there is. I have found that Accuracy brings some value (better results if chosen higher) to unseen data profits smile However correlation is small, but it is there. Also I did it for optimized and not optimized systems. One more time I got optimized results much better than without it.

I chose to cut down systems beforehand if accuracy will be below 50% for entry and exit, and for total if below 10%. I took these numbers from other ~40 systems portfolio experiment, because these was obvious points after plotting in the scatter charts. So this 2nd experiment is confirming it with unseen data(different portfolio) smile

https://s18.postimg.org/t59rth2ud/accuracy.png

All the systems was data mined since 2012 till 2016 - 06, and out of sample check was used till now. Used various timeframes and pairs. 0.01 lot used.

Don't forget to wire me my cut! wink I think we agreed on the percentage before.

Re: Using MyFxBook to analyse strategy entry and exit accuracy

footon wrote:

Don't forget to wire me my cut! wink I think we agreed on the percentage before.

We forgot to agree on exact percentage number smile Please PM what do you think is OK

Re: Using MyFxBook to analyse strategy entry and exit accuracy

hey footon...

I've incorporated the new Accuracy stats into my AccountStatistics.cs file and they compiled and are working.  But I'm not clear how to interpret them.  This thread is so polluted by posts (including mine) that I found it difficult to piece things together.  Would you mind explaining things again -- pretend I'm your grandmother (no offense to her)...

I'm currently looking at a strategy with what I think has good overall statistics -- here are its Accuracy stats:
     Entry Accuracy:   55.67 %
     Exit Accuracy:     70.91 %
     Total Accuracy:   26.58 %

Without going into the formulas for how Accuracy is computed, in layman's terms how are you using Accuracy to help you determine whether or not to include a strategy in a portfolio?

Thanks...

Re: Using MyFxBook to analyse strategy entry and exit accuracy

Hi Steve,

Footon hasn't been feeling well so I thot I'm reply on his behalf since I once asked him the same question and here was his reply to me. 

Entry Efficiency is Calculated as:

(maximum price seen - entry price) / (maximum price seen - minimum price seen)

Exit Efficiency is Calculated as:

(exit price - minimum price seen) / (maximum price seen - minimum price seen)

Total Efficiency is Calculated as:

(exit price - entry price) / (maximum price seen - minimum price seen)

Note:

However, I have reported to him that I have accuracy reading above 100% such as 3000% plus and hence, he said, there is definitely some wrong calculation somewhere and he will try to fix it after he gets well.  Meanwhile hope he recovers soon.

Re: Using MyFxBook to analyse strategy entry and exit accuracy

I'll add my ramblings of a man in fever. For entry it reads out how close it is to best opening point, for long it is the lowest price seen during the trade, vice versa for short. Exit accuracy shows how close is the exit price to most favourable price, that will be the highest price seen for long position. Total accuracy measures the whole range, meaning how much it captures of the whole range seen during the trade.

What we look for is the highest accuracies we can achieve. As Hannah pointed out, there seems to be a bug involved, when I'm in a better shape, I'll address this issue.

60 (edited by sleytus 2018-01-15 19:16:36)

Re: Using MyFxBook to analyse strategy entry and exit accuracy

Thank you Hannah and footon.

footon -- sorry to hear you are not well.  I hope the fever passes quickly.

Re: Using MyFxBook to analyse strategy entry and exit accuracy

Hi Footon,

I hope you are well by now.

Have you found the bug for the Accuracy Metric?  It's not that accurate you know.  Such irony.

Re: Using MyFxBook to analyse strategy entry and exit accuracy

Hannah, no, I am not able to reproduce the bug. I'm not so sure why. Can you trim your strategies that there would be only a couple indicators and still present "off" numbers for accuracies?

63 (edited by hannahis 2018-01-30 16:02:41)

Re: Using MyFxBook to analyse strategy entry and exit accuracy

wow, that's tough Footon.

I'm not sure whether I can do that.  I'll try when I could manage to and send it to you once I've it.

Re: Using MyFxBook to analyse strategy entry and exit accuracy

Do you build strats in certain order? Maybe you can determine at which step those off-numbers appear.
Finally, if you upload the data, maybe I can do something with it.

65 (edited by sleytus 2018-02-18 10:38:43)

Re: Using MyFxBook to analyse strategy entry and exit accuracy

Hey footon -- just wanted to let you know I'm using your Accuracy metric more and more and find it *very* useful.  I mostly monitor 'Total Accuracy'.

Here's how I'm using it...

I'm in the habit of always checking strategies (and portfolios) in MT4 Strategy Tester.  I do this primarily as a sanity check to ensure the code will run in MT4.  One issue that I've rarely seen discussed in the forum is that strategies tested in FSBPro are coded in C#, whereas those exported for use in MT4 are coded in MQ4.  And though porting may go well in most cases, there could be subtle differences. 

Let's say I'm working on 10 strategies that all appear to have excellent stats under FSBPro.  I'll then test them in MT4 Strategy's Tester.  Those strategies whose "Total Accuracy" is 40% or higher almost always continue to show excellent stats under Strategy Tester.  The strategies whose "Total Accuracy" is lower (e.g. 30% or lower) are noticeably worse -- it's not that they are necessarily bad, just that they do not exhibit the same high stats in Strategy Tester as they do in FSBPro.

I don't yet have enough results to extrapolate this observation to Demo or Live Accounts.  But in my hands your metric provides another hoop my strategies have to jump through in order for them to be promoted to the next level.

Once again, thank you...