26 (edited by nquental 2014-04-13 14:11:42)

Re: Swap rates - strange things happen

Popov wrote:

About your screenshot:

http://s7.postimg.org/e1i97054r/screenshot_296.png


- Backtester starts from Open (way point 1)
Later it has two choices:
   1. To go down to Low and upwards to WP 5.
   2. To activate BE trigger WP 2 and to BE stop at WP 3.
That's the reason this bar is ambiguous.

Since this is the Random method, Backtester randomly went to WP 2 first.

If it was Pessimistic, Backtester would make the same, since this interpolation closes the position at Zero.
Other interpolation would close the position at WP 5 at profit.

If it was Optimistic method, it would chose the 2nd interpolation route since it goes to profit.


I thought with 1 min data the program would know exactly when the lowest price of the day was reached. In theory ambiguity can only happen inside the 1 min bar, not at higher time frames (in my view). So maybe that actually happened during the 1 minute bar (not impossible in this case).

Re: Swap rates - strange things happen

Popov, but the change you made to the code cannot in itself justify such a big change.
Basically you deduced half a point to enable the BE. In my case, I had the BE at 25 points, so it became 24,5 points. So what? This doesn't make any different. Before, with the old version, I was able to obtain a good strategy even if I placed the BE in 5 points.

Re: Swap rates - strange things happen

The half point doesn't reflect on the BE size but on the check if BE has to be placed.
The new code is correct. It places BE exactly on the right place.
In both versions BE is 25 points. The older, wrong code doesn't place BE when should on every cases. The corrected code does it on the right spot. That is!
Please do not compare both versions.
Check the new one. If you find a place with wrong BE, please report and I'll see for the eventual error.
What I see on your example, the code works well.
Backtester shows an ambiguous bar because there are two variants of interpolation.
If you don't believe that there are two variants, compare Optimistic and Pessimistic methods. If these methods show different result, it means that the bars were really ambiguous.

29 (edited by nquental 2014-04-14 21:39:40)

Re: Swap rates - strange things happen

I'm checking carefully the trades and compare the live account (available here http://www.forexfactory.com/nquental#04-list) with FSB pro. My first discovery: because futures are traded local NY time 17:00 Sun-15:59 Fri, FSB opens a trade Friday just before midnight, but such trade doesn't exist in reality because the market is closed. Very often such trades are closed with a loss.

So this is already a valid discovery in my view. The program needs to know when the market is open or closed smile

Re: Swap rates - strange things happen

FSB opens a trade Friday just before midnight

I'm sure the program does it because the strategy enters at bar closing.

You can try entering at Bar Opening.
Other option is to Set entry from Monday to Thursday.

Re: Swap rates - strange things happen

Sure, but I thought that for each instrument one could designate the market times. But I understand this would not be a priority thing to implement. Still it's important to take this into account when performing a backtest.

For my comparisons I would like to export the journal list of trades eg to Excel. Do you know how to do it?

Re: Swap rates - strange things happen

There is an Export function in FSB.

http://s7.postimg.org/7abtovr07/screenshot_303.jpg

Re: Swap rates - strange things happen

Ok I'll switch back to FSB.
I attach two snapshots: one without intrabar scanning, and the other with intrabar scanning. The graphs are equal. Strange, for this strategy. I'll post other findings if I came to any conclusion.

Post's attachments

1.zip 302.29 kb, 2 downloads since 2014-04-15 

You don't have the permssions to download the attachments of this post.

Re: Swap rates - strange things happen

Use Comparator when you have ambiguous bars.

Re: Swap rates - strange things happen

So I finally carried out a more careful assessment of one ambiguous bar. The result makes me now sure that interpolation is not working completely well. Let's see:

- I have 1 min data in FSB (and the same data in MT4)
- Long position opens at 1841,3
- BE is set to 1841,3 when the price reaches 1843,8 (BE=25 points)
- As you can see through the real 1 min data bars, the price never reaches again 1841,3, so the BE is in reality never activated

In fact, the bar should not be considered ambiguous. It is not ambiguous. If the program is able to check 1 min data, the price action leaves no room for doubts, and the position would be closed with a profit.

Post's attachments

Ambiguous bar.zip 144.2 kb, 1 downloads since 2014-05-02 

You don't have the permssions to download the attachments of this post.

Re: Swap rates - strange things happen

It's not clear in this screenshots for what bar you speak. What is the strategy time frame?

Please post full data info (Time OHLC)  for bar No 223 for the strategy time frame.
Post 1min data (Time OHLC) for the time span of bar 223

Re: Swap rates - strange things happen

Popov wrote:

It's not clear in this screenshots for what bar you speak. What is the strategy time frame?

Please post full data info (Time OHLC)  for bar No 223 for the strategy time frame.
Post 1min data (Time OHLC) for the time span of bar 223

Sorry I'll explain better. The MT4 screenshot is 1 min data for 31/12/2013. It corresponds to intraday data for bar 223, which is a 1 day bar (the strategy runs on daily data). Do you still need the original data?

Re: Swap rates - strange things happen

It looks like you are right.
Please post you 1 Day data, 1M data and the strategy. I'll check it.

Re: Swap rates - strange things happen

Data attached, thanks.

Post's attachments

Data 2013 SP500.zip 380.74 kb, 1 downloads since 2014-05-02 

You don't have the permssions to download the attachments of this post.

Re: Swap rates - strange things happen

For me it works fine.

http://s7.postimg.org/oj0qd18jb/screenshot_351.jpg

Please check if there is no broken or missing data for 31.12.2013 for other time frame - 4H, 1H... or for fast test remove all intermediate data files

Re: Swap rates - strange things happen

Popov wrote:

For me it works fine.

http://s7.postimg.org/oj0qd18jb/screenshot_351.jpg

Please check if there is no broken or missing data for 31.12.2013 for other time frame - 4H, 1H... or for fast test remove all intermediate data files

I think you are right, it had to do with the date. Thanks to that I've just discovered that MT4 places all history in a new folder...

I've found however another ambiguous bar with the same problem. I'll do the same thing, but I wonder if that fact that only 100000 intrabars are allowed could also explain the issue?

Re: Swap rates - strange things happen

nquental wrote:
Popov wrote:

For me it works fine.

http://s7.postimg.org/oj0qd18jb/screenshot_351.jpg

Please check if there is no broken or missing data for 31.12.2013 for other time frame - 4H, 1H... or for fast test remove all intermediate data files

I think you are right, it had to do with the date. Thanks to that I've just discovered that MT4 places all history in a new folder...

I've found however another ambiguous bar with the same problem. I'll do the same thing, but I wonder if that fact that only 100000 intrabars are allowed could also explain the issue?

I confirm, it seems to be related to the number of intrabars allowed. I have data since 2000 for SP500, so I would need this number to be increased?

Re: Swap rates - strange things happen

Dear Popov
Can you tell me how to increase the number of intrabars allowed? I'm pretty sure that it is the cause of the misbehavior I've spotted.
Thanks
nuno

Re: Swap rates - strange things happen

Open your DataSource config file.
It is in C:\Program Files\Forex Strategy Builder Pro\User Files\System folder.

Change values of the following settings:
"MaximumBarsLimit": 50000,
"MaxIntrabarBarsLimit": 100000,

45 (edited by nquental 2014-05-06 22:44:43)

Re: Swap rates - strange things happen

Dear Popov

After having increased the max number of intrabars, the problem maintains. For the same strategy, I get:

- lots of ambiguous bars with a large file with 1min data since 2000 (even if data horizon starts in 2013). This file and the strategy can be downloaded here: https://www.wetransfer.com/downloads/70 … 018/c93338;
- only one ambiguous bar with the small file with 1min data since 2013. This small file was obtained from the large one. I've sent in a previous post these data.

When I use the large file, FSB Pro tells me that around 50000 1min bars were loaded (I have more than 1 million).

I am only using 1min and 1440min data now, not the other timeframes.

What is happening here? Now I'm lost!!

I attach the bar explorer of both backtests (see two tabs).

Thanks again
Nuno

Post's attachments

SPX500 backtests.xlsx 25.67 kb, file has never been downloaded. 

You don't have the permssions to download the attachments of this post.

Re: Swap rates - strange things happen

I disabled "cut off bad data" and now I don't have many ambiguous bars. Bingo smile

Re: Swap rates - strange things happen

Wow, that's quite a journey you've been on, Nuno! I admire your persistency, many would have quit long time ago. Thumb up.

48 (edited by nquental 2014-05-08 21:46:34)

Re: Swap rates - strange things happen

footon wrote:

Wow, that's quite a journey you've been on, Nuno! I admire your persistency, many would have quit long time ago. Thumb up.

Thanks footon smile
I would say: this thing was starting to put my trust into question, so I made an effort because I like the program a lot. The balance was very positive for everyone as well, because an important bug was found out.