Create and Test Forex Strategies
You are not logged in. Please login or register.
Forex Software → Forex Strategy Builder Professional → OOS question
Hey guys!
I set 20% OOS in the generator but it does not draw that green zone on the charts:
Also it's not shown on a strategy page.
However if I stop the generator right after start it draws the green zone:
Is it normal or a bug? Maybe I'm doing something wrong 
It is a normal behavior. The green zone appears only after the Generator or Optimizer are stopped.
When you set OOS and start Generator or Optimizer, the program loads only data up to the OOS date. For example if you use OOS = 20%, the program will load the first 80%. The Generator and Optimizer use these 80% of the data without having access to the OOS part. That makes the OOS calculations "real". After the calculations are finished, the program loads the full data set and this is the only moment the program can compare the performance from the partial data (first 80%) with the performance with the complete data set.
In reality there is no such thing as OOS. However this topic is difficult for understanding and the popular writings do not help much.
It is a normal behavior. The green zone appears only after the Generator or Optimizer are stopped.
When you set OOS and start Generator or Optimizer, the program loads only data up to the OOS date. For example if you use OOS = 20%, the program will load the first 80%. The Generator and Optimizer use these 80% of the data without having access to the OOS part. That makes the OOS calculations "real". After the calculations are finished, the program loads the full data set and this is the only moment the program can compare the performance from the partial data (first 80%) with the performance with the complete data set.
In reality there is no such thing as OOS. However this topic is difficult for understanding and the popular writings do not help much.
So... there is no way to see OOS zone on the other tabs like repository or strategy tab?
No way. OOS data and stats are out of sample. If you see them, they are not OOS 
It is simple. There is absolutely no way to calculate stats on OOS (missing / not yet existing) data.
Once the data is available, it is not more OOS. Therefore we always calculate in sample.
Probably you want to calculate stats with less data. Later to calculate stats with more data and the program to mark the zone from where the new data starts.
It is possible only if the program makes two calculations every time when calculates the strategy. Such feature is not available in the current version.
No way. OOS data and stats are out of sample. If you see them, they are not OOS
It is simple. There is absolutely no way to calculate stats on OOS (missing / not yet existing) data.
Once the data is available, it is not more OOS. Therefore we always calculate in sample.Probably you want to calculate stats with less data. Later to calculate stats with more data and the program to mark the zone from where the new data starts.
It is possible only if the program makes two calculations every time when calculates the strategy. Such feature is not available in the current version.
Well... looks like the only way to do what I want is to generate a strategy using cut data horizon and then recalculate it on extended data horizon..
Oh.. actually looks like I got what you mean.. So the generator just uses 80% of the data set in data horizon and then I see the chart with 100% data?
Oh.. actually looks like I got what you mean.. So the generator just uses 80% of the data set in data horizon and then I see the chart with 100% data?
Yes. It is correct.
It uses the partial data only during the generation. After finishing, it loads all data and calculates the full stats that you see on the Editor tab hence no green zone in the Editor (full data calculation).
During the generation, it sends stats to the Collections repo. The collections has only the partial data (first 80%). This is the reason why the collection doesn't show the green zone.
dedok3000 wrote:Oh.. actually looks like I got what you mean.. So the generator just uses 80% of the data set in data horizon and then I see the chart with 100% data?
Yes. It is correct.
It uses the partial data only during the generation. After finishing, it loads all data and calculates the full stats that you see on the Editor tab hence no green zone in the Editor (full data calculation).
During the generation, it sends stats to the Collections repo. The collections has only the partial data (first 80%). This is the reason why the collection doesn't show the green zone.
Oh... but my idea was to show this green zone in the editor because it's useful to check when the OOS period starts on the chart...
And one more question - what about acceptance criteria? The generator applies it to the 80% part of data only?
I mean.. that would be a nice option in generator to generate strategies that fulfill the acceptance criteria INCLUDING the OOS part.
generator to generate strategies that fulfill the acceptance criteria INCLUDING the OOS part.
Switch off OOS and you'll get it.
Actually, when the generator finishes it shows the stats calculated on the full data set.
generator to generate strategies that fulfill the acceptance criteria INCLUDING the OOS part.
Switch off OOS and you'll get it.
No it's different.. I mean it should find a strategy that fulfills the acceptance criteria on 80% of data, then add the missing 20% and check for acceptance criteria AGAIN. And push the strategy to repo only after that.
There are very few cases where a strategy that fulfills the criteria on full data set doesn't fulfill the criteria on the partial data. So you can easily safe the time for the first calculation.
Imagine you have criteria for max DD and max consecutive losses.
If you have max DD of $200 for the full data set, it is not possible to have greater DD for the partial data. The same is for the max consecutive losses. If you have 6 consecutive losses for the full set, it is not possible to have more for the smaller set.
However the opposite is fully possible. For example to have 6 consecutive losses for the partial set, but after loading the full data, the losses to become 8.
There are very few cases where a strategy that fulfills the criteria on full data set doesn't fulfill the criteria on the partial data. So you can easily safe the time for the first calculation.
Imagine you have criteria for max DD and max consecutive losses.
If you have max DD of $200 for the full data set, it is not possible to have greater DD for the partial data. The same is for the max consecutive losses. If you have 6 consecutive losses for the full set, it is not possible to have more for the smaller set.
However the opposite is fully possible. For example to have 6 consecutive losses for the partial set, but after loading the full data, the losses to become 8.
I mean that on the 80% of data the strategy may be extremely over-optimized so when you check it on the 100% data the balance chart drops down at those last 20% and this is actually a very often case... until you check "use default indicators values" of course.
So I have to go to the Editor of each strategy and check if the chart drops down at the end (over-optimized) or not (probably good strategy) and my idea was to save time by making a new feature like "double acceptance criteria check" option Hope you understand what I mean.
Yes, I understand. You can achieve that when you do not use OOS. It will work on 99% of the cases.
Summary:
If you want the Acceptance Criteria to be fulfilled for the partial (first 80%) of the data, use OOS.
If you want the Criteria to be fulfilled for the partial data (80%) and full data (100%) with one calculation, do not use OOS.
Yes, I understand. You can achieve that when you do not use OOS. It will work on 99% of the cases.
Summary:
If you want the Acceptance Criteria to be fulfilled for the partial (first 80%) of the data, use OOS.
If you want the Criteria to be fulfilled for the partial data (80%) and full data (100%) with one calculation, do not use OOS.
With the second case we will more likely get an over-optimized strategy anyway.. thanks for replying
In a way, everything is over optimized..... best test is the real market or having the expert perform on other instruments.
Forex Software → Forex Strategy Builder Professional → OOS question
Powered by PunBB, supported by Informer Technologies, Inc.