Re: Intrabar Scanning
@coffeesnob :
"Given that I don't know of any backtester on the planet that actually works properly, then there isn't really anything to baseline against..................
I would expect that as SMAs can only be calculated one way (as far as I know a simple moving average on the close is still and average over a period, kinda hard to stuff that up) then I would expect the results good or bad to be almost the same. If they are not then this shows at some level that the backtester is inaccurate and then you have to look to testing your strategies an other way. "
I found a backtester (optimizer) that I trust and that I sometimes use as a reference. I've used the "SMA cross strategy method ", too (lol).
Unfortunately this backtester (optimiser) works on another platform and uses a different language than the MQL, so it is a difficult work to make reliable comparisons. (Also appears important issues of historical data, it is evident that must be used exactly the same data). As if not enough inconvenience, its use is expensive (lol).
"Testing on real data is of cause the best way forward, but honestly that will take a hell of a lot of time if you are trying to validate something the generator built, for example the generator generates a strategy which tested with 18K profit over 3 months, it will take at least 2 months to confirm if the backtest was accurate or not. if you have 10 or so such strategies this is 2 to 3 years. I could have been in the market with something if I was pretty confident the backtest was accurate.
Understand please, I dont expect the backtester to be a perfect predictor of future profit. This is plainly never going to happen (its like trying to predict the next number on a roulette wheel, probability is it will be completely random regardless of how many times the ball has landed on 12) but it needs to be close, maybe 60 - 70% accurate in predicting profits.
To solve this problem I use for many years WFA. The principles are simple (see "Robert Pardo, The Evaluation and Optimization of Trading Strategies), but application of these principles requires A LOT of work if done manually. For MT4 exists a software (quite primitive), but it can tell you what is about.
http://www.easyexpertforex.com/walk-forward-metatrader.html
For those FSB generated strategies which I think deserve to make the effort, I make 30 -50 -100 ...manual tests using various reports INS / OOS and various lengths of INS. Using this method I have tested so far ~ 20 strategies generated by the FSB, but I found no one with WFE> = 60%. As I wrote, there are automatic methods that go fast (very fast!) but again, not applicable for MT4 . Handiwork remains the only solution for now.( in case of FSB...)