Topic: Optimize using recent data
I feel it is important to use recent data when optimizing strategies -- I hope the above image helps to explain why.
A strategy combines various indicators to create trading signals. Indicators are simple formulas -- very similar to the algebraic formula for a straight line: y = mx + b, where m, b are constants. Furthermore, most strategies have one or more constants -- similar to 'm' and 'b'. If you don't believe me, then take a look at the *.mq4 code.
Let's say you have a bunch of x,y data points and want to calculate the "optimal" line that best matches with the points. Performing a linear regression will compute the best 'm' and 'b'. If the data points don't change, then you are done. On the other hand, if you continue to collect more data points then you will need to perform the linear regression again. In other words -- the 'm' and 'b' are only good for the data points used when performing the linear regression to calculate the "optimal" line. If the data changes then 'm' and 'b' that you previously calculated are no longer the optimal values.
Now -- take a look at the image above. The black line is the "optimal" line that was computed when using all the data points -- i.e. from 2013 through 2018. The red line is the "optimal" line that was computed when using only the data points from 2018. They have vastly different settings -- i.e. their 'm' and 'b' values are very different.
Switching over to forex -- when I add a strategy to a Real account all I really care about is how it will perform *now*. I do not care how it traded 5 years ago, 3 years ago, etc. That is why I think we should only use recent data when optimizing.
One last point -- I think some people have the impression that strategies "learn" and that by using more historical data they will, in some way, become "smarter". That's not the way it works. Optimization does not use Artificial Intelligence (AI) -- i.e. there is no database backing our strategies. In our case, optimization performs a one-time calculation for 'm' and 'b'. If we want our strategy's indicators to use the optimal 'm' and 'b' settings for the most recent data, then you will need to optimize using the most recent data. Using more historical data does not help -- it only skews the computation in favor of the old data.