26 (edited by B52 2014-02-17 10:25:41)

Re: Intrabar Scanning

@coffeesnob :
"Given that I don't know of any backtester on the planet that actually works properly, then there isn't really anything to baseline against..................
I would expect that as SMAs can only be calculated one way (as far as I know a simple moving average on the close is still and average over a period, kinda hard to stuff that up) then I would expect the results good or bad to be almost the same. If they are not then this shows at some level that the backtester is inaccurate and then you have to look to testing your strategies an other way. "

I found a backtester (optimizer) that I trust and that I sometimes use as a reference. I've used the  "SMA cross strategy method ", too  (lol).
Unfortunately this backtester (optimiser) works on another platform and uses a different language than the MQL, so it is a difficult work to make reliable comparisons. (Also appears important issues of historical data, it is evident that must be used exactly the same data). As if not enough inconvenience, its use is expensive (lol).

"Testing on real data is of cause the best way forward, but honestly that will take a hell of a lot of time if you are trying to validate something the generator built, for example the generator generates a strategy which tested with 18K profit over 3 months, it will take at least 2 months to confirm if the backtest was accurate or not. if you have 10 or so such strategies this is 2 to 3 years. I could have been in the market with something if I was pretty confident the backtest was accurate.
Understand please, I dont expect the backtester to be a perfect predictor of future profit. This is plainly never going to happen (its like trying to predict the next number on a roulette wheel, probability is it will be completely random regardless of how many times the ball has landed on 12) but it needs to be close, maybe 60 - 70% accurate in predicting profits.

To solve this problem I use for many years WFA. The principles are simple (see "Robert Pardo, The Evaluation and Optimization of Trading Strategies), but application of these principles requires A LOT of work if done manually.  For MT4 exists a software (quite primitive), but it can tell you what is about.
http://www.easyexpertforex.com/walk-forward-metatrader.html
For those FSB generated strategies which I think deserve to make the effort, I make 30 -50 -100 ...manual tests using various reports INS / OOS and various lengths of INS. Using this method I have tested so far ~ 20 strategies generated by the FSB, but I found no one with WFE> = 60%.   As I wrote, there are automatic methods that go fast (very fast!) but again, not applicable for MT4 . Handiwork remains the only solution for now.( in case of FSB...)

Re: Intrabar Scanning

I develop simple EA's and then backtest the idea in MT4 and use walkforward analyzer as well as EA Analyzer pro.

Takes a bit of time for the coding but gives acceptable results

Re: Intrabar Scanning

Blaiserboy wrote:

I develop simple EA's and then backtest the idea in MT4 and use walkforward analyzer as well as EA Analyzer pro.

Takes a bit of time for the coding but gives acceptable results


Hi Blaiserboy, I have done very complicated strategies as well as simple ones in MT4, Backtested then and used WFA Pro as well, however given this is a forum about FSB Pro it hardly seems totally relevant, FSB/FSB Pro offers a very bright alternative to MT4, the coding interface is much easier, not to mention the fact that it is a common programing language (v MQL which is sort of C) if the backtester can be trusted in FSB Pro then to me this is a very nice solution, mt4 should have been retired years ago in my view.

Re: Intrabar Scanning

coffeesnob wrote:
Blaiserboy wrote:

I develop simple EA's and then backtest the idea in MT4 and use walkforward analyzer as well as EA Analyzer pro.

Takes a bit of time for the coding but gives acceptable results


Hi Blaiserboy, I have done very complicated strategies as well as simple ones in MT4, Backtested then and used WFA Pro as well, however given this is a forum about FSB Pro it hardly seems totally relevant, FSB/FSB Pro offers a very bright alternative to MT4, the coding interface is much easier, not to mention the fact that it is a common programing language (v MQL which is sort of C) if the backtester can be trusted in FSB Pro then to me this is a very nice solution, mt4 should have been retired years ago in my view.


Thanks for your input.

I don't think my message was directed to you, however, I was giving someone my own solution to the current dilemma and the question of accuracy.

Sometimes it is suitable to use more than one method to establish confidence in a system and that is the route that I have taken.

I apologize for not meeting your expectations.

30

Re: Intrabar Scanning

To make things simple, I'm reminded of a post dated 08/11/2013
"In my oppinion FSB(Pro) + a good WFA module ( when/if will be available)= ( can be ) the best product available on the market (with no much money) "

Re: Intrabar Scanning

Probably things will be easier to program once some folks start to contribute to the cause.... We seem to be expecting a lot for free.

32

Re: Intrabar Scanning

"not (too) much money" <> "free"  ; just means a decent price...

Re: Intrabar Scanning

some users have donated to the cause

34

Re: Intrabar Scanning

I am quite skeptical if we talk about human nature. In several forums my name is "The misanthrope". I am ready to pay for a quality product along with all the other users, but I will not donate because I do not know how many others do (LOL!)

Re: Intrabar Scanning

B52 wrote:

I am quite skeptical if we talk about human nature. In several forums my name is "The misanthrope". I am ready to pay for a quality product along with all the other users, but I will not donate because I do not know how many others do (LOL!)

One of the threads has a list and I understand your reluctance.

Soon there will be an updated paid version which will outperform anything that we have seen so far, once that is selling there will be more available for programming and the project is going to take off.

Re: Intrabar Scanning

PeterChristoph wrote:

Hellow
I have the problem that strategies in FSB work good, but when I open the strategy in FSB Pro
it makes very bad results.

Peter, FSB Pro and FSB must give 100% equal results on same data and settings. Please be sure all discrepancy (if any) will be fixed before the final release.

Re: Intrabar Scanning

B52 wrote:

"In my opinion FSB(Pro) + a good WFA module ( when/if will be available)= ( can be ) the best product available on the market (with no much money) "

WFA will be added to the product at a later stage. Probably as a free update after the final release.

Re: Intrabar Scanning

B52 wrote:

Unfortunately FSB Optimiser works completely non-transparent to the user , so I can not know if a backtest is reliable or not.

FSB Pro will provide an easier way to track orders execution.

http://s28.postimg.org/viw8a7lzt/screenshot_154.jpg

39

Re: Intrabar Scanning

Popov wrote:
B52 wrote:

"In my opinion FSB(Pro) + a good WFA module ( when/if will be available)= ( can be ) the best product available on the market (with no much money) "

WFA will be added to the product at a later stage. Probably as a free update after the final release.

If you really intend to write a WFA module  for FSB, I suggest you take a look at WFA module used by  TradeStation. Especially at the concept of "Cluster Analysis".Good Luck!

40 (edited by B52 2014-03-11 09:32:34)

Re: Intrabar Scanning

Popov wrote:
B52 wrote:

Unfortunately FSB Optimiser works completely non-transparent to the user , so I can not know if a backtest is reliable or not.

FSB Pro will provide an easier way to track orders execution.

http://s28.postimg.org/viw8a7lzt/screenshot_154.jpg

For me, non-transparent means lack of information on how to do optimization. (type, criteria, etc.). Some optimizers provide this information (and many others, such as maps) (see pictures attached).
But of course, this information is not vital, especially if there is a module WFA, which lead directly to conclusions.
http://s28.postimg.org/tjn0mdya1/fsb101.jpg

http://s28.postimg.org/51uwyhvpl/fsb102.jpg

http://s28.postimg.org/7uo4iiw21/fsb103.jpg

Re: Intrabar Scanning

I tested the attached strategy of PeterChristoph.
It gives the exactly same results in FSB and FSB Pro
http://s9.postimg.org/p8i9ppzvf/screenshot_155.jpg

If you have other strategies that you think they show different results, please attached them in the forum.

FSB and FSB Pro calculations must be 100% equal on equal inputs.