Topic: Backtests over 10 years without crashing & the longer backtest debate
Hi there
I have searched here on this subject of backtests over long time periods and the consensus opinions seem to be that 1 -3 year backtests are good enough. I have seen Metatrader backtests of systems over 10 years with backtest at 99% modelling quality on 'every tick' - that show a smooth equity curve and great results - and perform long-term very well and in line with the backtests expectations in live trading. I know the opinions on here and elsewhere on the quality of Metarader backtests is mixed - but that's a separate issue I have asked about in another post.
It seems to me that it's logical that a system that survives 10 years backtest (of varying market conditions) with a smooth equity curve is much better than one that survived shorter backtests of 1 - 3 years - and if you test a shorter backtest on 'out-of-sample data it usually fails.
I noted the instructions and warning here about manually increasing the Max Bars in a historical data set causing problems with the computer. I have tested this and experienced it. After a few adjustments to the strategy it takes longer and longer (minutes) to calculate and quickly hangs the program and computer in a very nasty way requiring a reboot that goes very slowly. This worries me about the risks to my computer of this repeated occurrence.
This on an HP EliteBook i7 with 4gb Ram and Nvidia 2gb graphics card - not a monster but a decent machine that handles processor intensive tasks like rendering video very well.
The FSB system I'm working on has 3-4 opening and one closing logic slot and uses 2 x indicators and 2-3 timeframes (5 min, 1 hour and 4 hour).
I tried working with 2 x sets of 5 years (setting the data horizon) and the machine handles it better - but the problem is - I get it working for the 1st 5 years then test it on the next 5 years and the results are poor - and vice versa.
Questions:
1. So, my 1st question is how people can logically assert that a backtest over 1 - 3 years will yield a strategy likely to be as robust as one that shows a smooth equity curve over 10 years. Most systems fail on out-of sample data.
2. Is there's any trick or technique I can try to work with a full 10 years price data without hanging my computer in 5 minutes.
Request:
Perhaps you can consider finding a way to make it possible to change FSB so it can handle a 10 year backtest.
I think both the results and the FSB Platform will gain much credibility from this improvement.
Clearly there's some amazing programming and very hard work over many years that's gone into the development of this excellent platform; and it seems that being able to handle large data sets would be a big advance in it's ability and credibility.
Any input much appreciated.