FSB and openoffice!

I'll do my best! It's actually quite simple and not much resourceful. But the main point for now is to have results which have real value. Aronson in his book tested like 7000 rules (strategies) and he didn't find none, which were statistically significant. I'm not getting my hopes up, that's for sure. But I'll keep reporting of my findings.

Keep it up Footon!!!

So proud of you and it's such an encouraging news...seeing some bit of the fruit of your labour.  Wishing the very best that you will have a clarity of mind to sieved through all the information to find the essence of what you need.

Hang on there

Keep it up Footon!

Thanks for everything

Bloody hell, some technicalities turned out to be rather fundamental, but the numbers arrived finally. Unfortunately no one offered opinion whether the strat presented above should be regarded as potentially good or a total curvefit.

First discovery: bootstrapping results are not zero centered, I wonder why.

P-value is over 3, so this suggests the strat is a waste-bin material. But we have the opportunity to test whether this inference holds true or not, equity curve for the remaining 20k bars below.

The statistical significance test on this strat told us that it is not a good strategy, and it really isn't!

Frustration? Why? I'm getting the results that this statistical significance test works! It has merit. Certainly there is still a lot of work,  but this approach seems to offer remarkable ability in assessing a trading strategy. And it proves how big pitfalls there are in datamining.

I see now that there might be a slight error in p-value, I'll check the numbers later.

Corrected p-value is 0.54, pretty awful, just like the out of sample proves.

Dave, I'm experiencing a great statistical method to assess trading strats, I'm not banging my head against the wall or something. What does Thomas ask in his first post? He asks for (automated) methods to aid the process of picking strats, which have actual ability to operate profitably. We can call it anti curve-fit test or robustness test or whatever, it doesn't change the objective.

How this might come into play? Think of a man, a trader, who wants to trade but lacks resources. For instance, he lives up North, every spare moment he spends to make firewood to be able to live through the cold winter (he uses a handsaw and an axe). Therefore, he just doesn't have that capacity to develop strats manually. What can he do? He can use machines to do the work for him.

Like Thomas, he uses FSB to generate strats. At one point he has 15000 strats. Now he wants to build a portfolio of 10 strats. He knows that data-mining has its pitfalls, you can have great returns on a backtest, but these won't translate into profits on a live account with a flick of a switch. What next? Demo trade them all? Take a guess? No, he determines the statistical significance of the backtested strats and from that point he'll build his portfolio.

I hope you understand the project now.
-----------------------------------------

I increased the bootstrap cycle from 5000 to 15000. The p-value for the last strat is the same, 0.54! Or like Ahmed says - 0.54!!!

Now the most difficult part - I need to have a strat with a sufficiently low p-value, which I then put on unseen data. Given the theory of the concept, a strat with a sufficiently low p-value has to maintain its profitability. If I succeed in this, I have proof this works and implementation to FSB can begin!

I tried H's 03 strat. I got only 100k data, which is only a third of what I deem minimum. As it really trades seldom I think maybe 600k might show something meaningful, we can't forget the law of large numbers after all. I searched for export function in pro for quite some time, after I found the button it turned out csv is not supported, so I turned to old guns and my stress level lowered significantly. Another issue was that backtest result was similar but not exactly the same, fewer trades and some discrepancies in stats. My conclusion is that due to these factors my results are inconclusive.

Returning to the theoretical side of things, again the bootstrapped distribution is not zero-centered, which means that something is not right. I'm thinking it is an indication of too few original samples (trades). This makes me think how many samples is enough? Definitely more than 1000... Another thing I noted - higher TF results look more "normal", maybe I'll ditch M5 and work with higher TFs after all.

Now I have some meaningful results. It's a generated strat on m5 EU, its p-value is 0.0014! Highly significant in other words, it should retain its profitability in the following OOS test, but the cold hard truth is - total failure! Equity charts below...