By the way: does the program performs walk-forward analysis? Or only out-of-sample? If so, how to do it?

You mean walk-forward optimization? Easily doable.

A few months ago we were invited to submit suggestions for a revision which is just about complete at this time.

We asked for many things which we felt would benefit our projects and many of those were implemented and more will be implemented as FSB pro is released.

And we asked for a ton of stuff that would overload the developer...... he did his best to accommodate the list.

Backtesting is quite easy using the menus and it is really fast, almost instant. Walk forward is the same.. It does not need complexing, it works fine.... As fast as you move your mouse you can backtest, walk forward, whatever..  Take a couple weeks and learn how this thing works, you will be pleasantly surprised.

You will see as you explore that there are several outputs available for study and manipulation.

AND  all is FREE. no charge.....  this software is tremendous.!!

I'm not sure if FSB was costing $600 it would be better spread because of some extras used and understood by 2% of the users.

- Do we need so many indicators? I'm not sure about the need to include so many indicators. Would it be possible to make a pre-selection of the less correlated ones in order to increase the efficiency of strategy generation?

Thanks
Nuno

If I understand what you mean here that is already possible. You can ban indicator for both opening and closing position as well as opening logic and closing logic under the indicators menu in the generator.

- Backtesting with v-fold cross validation. Backtesting is one of the most important stages of a strategy development. So far I haven't found any program capable of performing a so-called v-fold cross-validation. FSB is already good in what it provides (OOS tests), but it's not enough to prevent over-fitting. I think the best way is to divide the entire period under analysis into x periods of equal size (the "folds") - for example, divide 1 year into 12 months - each period then being divided into a training and test samples. A separate optimization would be carried out for each of the folds. The best system can then be selected using different criteria. A good one is calculating a sort of risk factor such as average CAGR / [GSD of each period's equity growth], or average CAGR / average drawdawn. Systems with very poor performance in just one month could also be filtered out. This method is very powerful and is used is many statistical packages (but not those with applications related to the stock market).

- Work works well lately. Another interesting backtesting possibility would be this "WWWL". It is in my view a mistake to optimize systems with long datasets spanning through years and years. This is because structural economic conditions change (eg bull vs bear markets), or more simply because a currency or equity may move from a sideways movement into a trend period. When we optimize a large dataset we are basically optimizing our system for the most common price pattern. So the system will perform well under certain conditions and badly under others. My idea is to optimize the system every month and use the optimized parameters as inputs for the next month, and so on. The backtest would therefore be a combination of several small backtests, but all would be done in one go. This is contradictory with my previous idea; this system would clearly be over-fitted, but with some changes it could be a neat feature.

- Do we need so many indicators? I'm not sure about the need to include so many indicators. Would it be possible to make a pre-selection of the less correlated ones in order to increase the efficiency of strategy generation?

Thanks
Nuno