Topic: Proposals to improve the software

Dear friends
I'm really impressed by the quality of FSB!! I like it very much.
I have a few ideas to improve it, so I welcome any comments on them:

- Backtesting with v-fold cross validation. Backtesting is one of the most important stages of a strategy development. So far I haven't found any program capable of performing a so-called v-fold cross-validation. FSB is already good in what it provides (OOS tests), but it's not enough to prevent over-fitting. I think the best way is to divide the entire period under analysis into x periods of equal size (the "folds") - for example, divide 1 year into 12 months - each period then being divided into a training and test samples. A separate optimization would be carried out for each of the folds. The best system can then be selected using different criteria. A good one is calculating a sort of risk factor such as average CAGR / [GSD of each period's equity growth], or average CAGR / average drawdawn. Systems with very poor performance in just one month could also be filtered out. This method is very powerful and is used is many statistical packages (but not those with applications related to the stock market).

- Work works well lately. Another interesting backtesting possibility would be this "WWWL". It is in my view a mistake to optimize systems with long datasets spanning through years and years. This is because structural economic conditions change (eg bull vs bear markets), or more simply because a currency or equity may move from a sideways movement into a trend period. When we optimize a large dataset we are basically optimizing our system for the most common price pattern. So the system will perform well under certain conditions and badly under others. My idea is to optimize the system every month and use the optimized parameters as inputs for the next month, and so on. The backtest would therefore be a combination of several small backtests, but all would be done in one go. This is contradictory with my previous idea; this system would clearly be over-fitted, but with some changes it could be a neat feature.

- Do we need so many indicators? I'm not sure about the need to include so many indicators. Would it be possible to make a pre-selection of the less correlated ones in order to increase the efficiency of strategy generation?

Thanks
Nuno

Re: Proposals to improve the software

nquental wrote:

- Do we need so many indicators? I'm not sure about the need to include so many indicators. Would it be possible to make a pre-selection of the less correlated ones in order to increase the efficiency of strategy generation?

Thanks
Nuno

If I understand what you mean here that is already possible. You can ban indicator for both opening and closing position as well as opening logic and closing logic under the indicators menu in the generator.

smile

Re: Proposals to improve the software

Yes, but I don't have any easy way to see which indicators are more or less correlated. I have done it in a separate spreadsheet but for another time frame, so it's probably not usable in most cases.

Re: Proposals to improve the software

@nquental, we can add virtually every feature in the programs, but we do not have resources to do so.
If a copy of FSB was sold for $600, we would be able to higher 5,6 coders to do the job. In our case, I and other contributors work in our free time and provide the product for free.

I'm not sure if FSB was costing $600 it would be better spread because of some extras used and understood by 2% of the users.

Re: Proposals to improve the software

Perhaps you will use the software for a few months to learn of its capabilities.

A few months ago we were invited to submit suggestions for a revision which is just about complete at this time.

We asked for many things which we felt would benefit our projects and many of those were implemented and more will be implemented as FSB pro is released.

And we asked for a ton of stuff that would overload the developer...... he did his best to accommodate the list.

Backtesting is quite easy using the menus and it is really fast, almost instant. Walk forward is the same.. It does not need complexing, it works fine.... As fast as you move your mouse you can backtest, walk forward, whatever..  Take a couple weeks and learn how this thing works, you will be pleasantly surprised.

You will see as you explore that there are several outputs available for study and manipulation.

AND  all is FREE. no charge.....  this software is tremendous.!!

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....

Re: Proposals to improve the software

Thanks for the answers. I already use the software for some time. I liked it so much that I felt like sharing some thoughts. No need to justify this or that, I just gave some ideas that you might implement or not in the future.

Re: Proposals to improve the software

By the way: does the program performs walk-forward analysis? Or only out-of-sample? If so, how to do it?

Re: Proposals to improve the software

nquental wrote:

By the way: does the program performs walk-forward analysis? Or only out-of-sample? If so, how to do it?

You mean walk-forward optimization? Easily doable.

Re: Proposals to improve the software

use the data period, move the dates.... two weeks at a time, shows results immediately

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....