Hi,
Interesting points here. Will try to add something from my side. Now for me one of the main filters in the strategy testing is to check if strategy behaves similarly on the different brokers data (about 2-3 brokers data sets on the same data horizons) and find that results are not changing drastically. If one data sample shows steep PL curve, and on other it gets drastically worse, I think that strategy got curved around too much and picked up on something too small than "major repeatable market structure", and probably live unseen data will be sure miss too. For me it makes sense, why some great strategy should fail with different brokers data on the same time horizon period?

I think that is not the case. Every strategy will have some shortfall in live trading (unless you created trend following strategy and you are lucky to get unseen live data trending even more stronger than back testing ) Real question is by how much you will get that shortfall and if your strategy still will be profitable? That disconnect is really called "data mining bias" or "curve fitting/over-fitting". There are some tests that promise to fight this: Monte Carlo,  Multi Market, Walk forward analysis, System Parameter Permutation ... Still experimenting about these methods and searching for answer. But this one is not some easy question to be answered quickly ... For now I have found that MonteCarlo (variable shuffling and data randomization) has some positive effect. Walk Forward can be the one too, but this is hard test to pass, also I do not have conclusion about it effectiveness with big portfolio and longer real time period trading (stopped trading these systems on the drawdawn, probably too quickly). For now I am still working hard with System Parameter Permutation / Parameter Sweep test variation, this one is very promising, but unbelievable slow... still want to get more data to draw some conclusions.

With FSB we get pretty much accurate backtest, let it be bar opening, or tick entries. How mr. Popov or Footon would say "accurate interpolation method" Unless, there is increased spread in the real market, FSB does not take it into account. And I am sure it is causing some shortfall to the end PL results. I am recommending to check if your great unbelievable steep PL strategies is not trading midnight spread too much, or big break out events like NFP news releases too frequently

It does not matter how much ticks you had in the bar, or what order it was ticking, it matters that price was in specific point in that bar and it could trigger your strategy entry/exit price which was calculated from the last bar value. Unless there are some scenarios when it is not clear which price was firstly touched in same bar when TP SL or entry/exit point involved, but FSB warns you about it with ambiguous bars and it should be taken seriously that you got probably not trusty backtest in these marked bars. Probably mr Popov or mr Footon could explain here better.

How I said it can be some cases that FSB backtested entries/exits do not match exactly with real market trading: 1. Spread 2. Slippage 3. Mistake/difference in specific indicators code between FSB and MQL , if later is the case there is a need to rise this question and wait for a fix
You always can and actually it is mandatory to check couple real time entries with updated same broker data in FSB and check if they match. What I found that 95% of the time they match perfectly. So backtest is accurate, but it does not mean that it will produce same steep PL curves like backtest did for a reason what I mentioned earlier - "data mining bias". And to get method which completely deals with this (exact same steep profits which was backtested match with real time market profits) is probably worth some trillions dollars. But there is some methods which promises to reduce it impact.

Some point about using OOS check. I have read somewhere that is pretty much the same if you use OOS or all data for strategy creation. Using more data compensates using out of sample check to pass strategy. End results is pretty much the same. I did some experiment in this using two OOS, and found that using first OOS is not productive, it does not warn you about your 2nd OOS potential results, it was random. For now I am using OOS only for method effectiveness research, to skip long demoing part. I totally exclude last year from strategy creation and decision making. After some portfolio or strategy made by some researchable question I check if this method would be profitable on unseen last year data chunk.

Midnight spread problem is specific to broker and strategy. Below are example on my recent H1 timeframe generated strategy, which picked up on trading almost all entries at 00:00 time. Indicators somehow picked that gap/spreading action, and profits are total illusion. To see these things after some time in real money account can be pretty expensive. Also I do not use H4 D1 bar opening strategies for same reason, unless you are sure that your not paying these insane spreads for your entry everytime. In the strategy below I added time filter, which do not allow to trade at midnight,  and all good results are gone.

For other types of strategies, which do not trade this midnight fake price action, spread/slippage effect is probably much lower. But it can be present in some cases, for example, if your strategy trades huge break out spike, it might be that these price spikes are caused by important news like FOMC or NFP (at these trading times spreads get a lot bigger), and if you check your entries with economic calendar and see if you have lots of winners on these news time... you can suspect that backtested profits might be not accurate.

Hidden bad ticks ... I think you refer to events that happens in single bar, when trade outcome depends on lower timeframe price action, which bar area was firstly touched -> TP or SL. Image below what I mean.  On these cases FSB goes to lower timeframe, and checks price action there to see what was happening inside that bar, and if there is still no proof which price was first touched in that bar, you get ambiguous bar warning. So nothing here hidden or bad... all ticks are in the bounds of the bar high and low. Please correct me someone if you see that I am talking false nonsense here

Have a good weekend.

1. I think there is an indicator for maximum spread
2. In my experience, different brokers with same horizon good strategies win.
3. With FSB pro I Use M15.
4. I recommend to try to use add, reverse.
5. Sleytus can use FSB Pro Journal in his MT4Tracker to create statistics except detailed statement
6. How to create Portfolio can be an interested talk
7. An indicator which calculates equity in fsb pro can be interested in combination with other indicators.

Just some information...

Hi,
Thanks GD for mentioning spread filter. Yes, it helps to prevent entries in increased spread time, but only in real time trading. And it is good idea to not forget to add this indicator for each strategy It seems that I am not doing this right now... I wanted to mention times when FSB is not so accurate with backtest and rise awareness for other users. In my example I wanted to isolate few trading hours during increased midnight spread, and show that profits was not real.

Sleytus, I am heavy user of time filter. Not all day is the same in forex trading activity. Asia session is a lot calmer, rangy, huge price movements is rare in these time, and it seems you filtered out these times with your 22-04 filter (to be exact it depends on timezone, but for GMT+2 it seems Asian ranginess is cut off). It means that calmer price action is not so profitable with your strategies. So why not skip some losses? During London and US session trading is a lot more active. It is very logical to filter out calmer/loosing market hours and leave more active trading hours what are suitable for your strategy. With some strategies you can find that opposite is true, and calmer Asian session market is more suitable. Also you can isolate only one or few hours in a day, like London Open time, or US open session, and you will find certain strategies suitable for that market volatility. In short "time filter" opens  some possibilities what you will not find without using it. If you are lazy not to reoptimize strategies, and have some reasoning for this, you always can shift few hours manually in time filter for timezone compensation.

Hi,

In my experience this time is about ~200ms , so not very much ticks. If you get some, you get slipped, and with ECN good execution broker I sometimes get positive slippage. So in the end it pretty much evens out. If you are seeing higher execution times, like couple seconds and bigger slippages you definitely need to change your broker to better one, because it is stealing from you... I had used some nasty brokers, witch sometimes putted some artificial lag and slipped like 10 pips in some cases. You need to avoid these for all costs. You can insert some mql code to EAs output executions times and slippage after each order. I was doing it in old mql days... And found some surprises with some brokers, lag and slippage was a norm for them.  I can recommend some ECN broker what I think is good, but not sure what they say about USA users. If interested please PM me.

Mr. Popov, it would be very nice to get : Execution time, Slippage and Spread output in the log after each entry/exit executed in new FSB versions. It would be tremendously useful to catch NASTY brokers.

An interesting idea and got me to thinking.

I can create an indicator (for free) that will chart two values:
(1) the latency (i.e. delay) between the time a close signal is received and the time the position is actually closed
(2) the delta (i.e. difference) in price at the time a close signal is received and the price used by the broker (or bank) when the transaction is completed

You would run this indicator in both a Demo and Live account.  You would expect the latency and delta price charts to be very similar but, if not, then it may warrant further investigation.

I'm not so sure what's the best.

My guess it it would be the if your broker has high quality data which you can use for strategy creation and optimization.

Unfortunately only very few broker have high quality data available.

My current broker is IC Markets their execution is great no complains about that.
But the history data they are offering is bad.
I just get 2 months of good 1Minute data and that's it and since I know no way of building a live history database I'm always stuck with the last 2 months of data.

I generate my strategies with Dukascopy data (via Tickstory) but I cannot really compare them with the IC Markets data.
The strategies look similar during the last two months but it's not really satisfying.
Backtest woud not work either since I'd have to use Dukascopy for that also since not enough bars are loaded from my broker to make this work.

And I'm afraid there is another problem; many strategies need a certain amount of bars already loaded into MT4 to work.
But the quality of my broker's past data is qualitywise a joke compared to Dukascopy.

So I'm thinking does it make sense to just open an account with Dukascopy?
I mean it takes so much time and effort to create the strategies and then the (live) data is the problem?

What are your thoughts about it?

OK, I better call it data range.
The problem is I have just 2 months of 1 Minute live data.

But I guess I have to deal with it in someway.

;-) I'll keep my mouth shut