I am thinking that the best curve fitting prevention can be developed by an individual.

Use few indicators as these are what cause the curve fitting.
Use the original values for the indicators or else lock them during generation and optimization.
do a walk forward over several sets of data ie use ten sets of dates.....
use the monte carlo and multi markets on each result.
Be mindful of the traps in time frames less than 60 minutes.

expect that a strategy is not going to give 100 per cent profitable trades

I am thinking that any tests for curve fitting are going to be similar to monte carlo being done on several different time periods.....  with some little variations.

I actually do not think we are going to get better than what we have already.

For my own trading....... I work with a system for months, make little changes and test it and test it and test it.

That Multi Markets tool  I use all the time......

I may be wrong but it seems to me that Multi Markets is the moment of truth. If I can find a system that passes Multi Markets. I am pretty sure I have reduced the curve fitting.

Perhaps there is a couple things can be added to Monte Carlo, I do not know what else can be added, I am not that familiar with it. There seems to be a lot of options to play with.

Maybe Marc can develop the tests that you require.

I forgot something.

If you are working on M1 timeframe you should have the realtickdata from the broker who you want to trade.
It is not possible to take Duka data. Search EAs which this data and trade this found EAs on pepperstone. This don´t work.

At H1 or bigger timeframes you can use Dukadata.

this was my experience in the past.
thomas

Hii Thomas, i was using SQ also, currently waiting on the new version.., i got a portfolio from it already, working pretty well!,

although i want to expend my empire, mind & do not waste time in the meanwhile so i have started working with FSBPro also, both software are very unique in each way, and they are the best in the market right now, althogh i found FSBPro much more faster and flexible then SQ (well.. at least for the current time),

i have reed your post and noticed your way of working with SQ can be also implemented right now with FSBPro,
im working nearly the same way as you btw

1. Generate few lists of 100 algorithms each (using 30%~40% OOS), *Already suggested to Mr. Popov to rise this limit..
2. Checking all the lists i got on: 1 TF Above & 1 TF Below + different currencies + MonteCarlo AKA RobustnessTest.

This whole procedure of Section #2 can easily be done by that recently added feature 'Multi Markets' ,
the only limit is the procedure of Section #1 and thats the limit of the 100 algorithms per collection...

here are my suggestions to Mr. Popov including raising the limit of the collections list: http://forexsb.com/forum/topic/3995/fsb-pro-wish-list-requested-features/page/12/

hope it helped , good luck to you! and good luck for us all, i hope at least the limit would be raised though

Hi karish,
I hope that mr popov will expand the limit from 100 to 50000. I think this will be enough.

thomas

Dear all,

If we rely on  curve fitting EA, we let never learn the fundamental theory of trading.  A good and strong fundamental, theoretical EA beats most if not all curve fitting EA.

Start depending on yourself and less on the machine.  Start looking at various indicators and adjust the parameters till it fit into the price changes and you will find a consistent and dependable theory you can always rely on...no more curve fitting but solid and sound theoretical trading strategy which you have lots of confidence on.  That's the fruit of my labour and much prayers.

The main reason, I know what I'm doing and keep improving my EA results is because I know what I'm doing...I have a framework or theory in mind and I keep working and improving it and hence, my EA gets better and more accurate (now working on closing accuracy as I've so call "master" my opening conditions).

Because of my years of working on my EA, I have a better understanding when I look at a EA generated by FSB, I know which is theoretically sound and which aren't likely to be.  This is a wonderful skill and knowledge, FSB can't impact to you.  But FSB can give you a lead or a headstart by giving you some possible combination and you need to go back to your indicator and look at the settings to see whether the combination makes sense or not.

Trust me, if you send more time on looking and working out your parameters, you'll have better chance of nailing the EA of your dream than to rely on curve fitting EA, which you are clueless whether the EA is theoretically sound or not.

I'm glad I rely on due diligence to find a strong trading theory whereby I can have full confidence in.  As a result, it is a skill no one can rob me off and an advantage on how to use FSB to it's fullest potential.  Remember FSB is just a powerful tool.  And bad worker always blame his tools.  FSB is a brilliant and a genius work of Popov and his team...I fully respect him and his big and generous heart.  The rest is up to us to maximize it to our own knowledge and if you expect the machine to think for you...you will always end up clueless of what you are doing...you have to learn the art of trading and acquire the skills.

And it's a skill I'm going to impart to my children, a skill I'm so proud of and nobody can rob me off and a skill my children would benefit greatly and put them on a head start, a skill the university can't teach them and a perpetual skills that can earn them a living.  A tremendous rewards for those who labour to gain such skill which no nor FSB can give to you...you obtain yourself.

There are two types of traders, professional and gamblers.  One has the skill and the other depends on luck.  At the end of the day, you have to develop a strong trading theory for yourself, if you don't have, you aren't a trader, you are a gambler, you are still clueless on how to trade.  Without FSB, I can still look at the chart with indicators and know which direction to trade, do you?

@GD
ok I understand this "You have to remember that every time you decide a selection in your methodology, actually you make a kind of large fitting"
This is clear.
So I have to remove the filter M30, H4 and different currency-filter

New Filtering:
Make only tests with: Robustnesstest and 1+1 Filter
And add an additional Filter with different Tickdata from different Broker
I will check this.
thomas

@hannahis,
I don´t have a full concept to filter all curvefitted strategies. At the moment I work only with SQ (Strategy Quant). I thing the user "GD" has more knowledge about curvefitting.

To produce not so much curvefitted strategies a good setting of building blocks is very important. If a generator have a good building set with building blocks who make sense the strategy generation is fast.
The next Step is to test the found strategies on unseen data. For example.
You generate H1 Strategies from 1.1.2008-1.1.2012 you check the found strategies with data on
1.1.2012-1.1.2014

The second are the Robustnesstests. FSB have Montocarlo Analysis for RT.

The next check is to check the Strategies with different Broker data. For example with Data from Pepperstone and with Data from Tickmill. Please use realtick and use the same GMT.
The Trades should have nearly the same open and closetime.

If you use the Filter: M30, H4, and a different Currency pair we have one problem. You need Longer to find such Strategy. If you search longer you produrce more curvefitting.
I thing this was the statement from "GD"

@GD
Is this correct ?

I think curvefitting is very complex. I found some mathematical description and tests (bootstrap etc.) I don´t understand this all.

One of the best curvefittingtest is the human brain. A human with knowledge kann look and decide if this rules makes sense or not.
thomas

Tnickel

I have and another occupation, where I use a lot of curve fitting. I did not learn fitting by using fsb.
A minimum in trading is a complex situation as there are not standard rules (one theory) in trading.
The best is to remember that most of the traders use the most common ideas and they apply them in their trading.
Hannahis is right to try a strategy but how good it is only statistics can answer and research on existing data.
Experience and study and keep notes.
The question is "WHEN to stop fitting".

Footon quote
...(essentially FSB generator) comes with a number of biases, which result in non-fit strategies. The task is to use statistical evaluation methods to determine whether the strats have genuine predictive ability or the results are random (pure luck in in-sample, catastrophic in out-of-sample or live trading).

Footon, what do you mean by FSB "non-fit strategies", do you mean not genuine predictive ability?

If that's what you mean, I agree with you and that's the reason from the very start, I don't use generator and optimizer for my strategies at all.  However recently, I want to see whether I can Generate good strategies by the selection of certain indicators that are less "ambiguous", that would reduce the element of "unpredictive" strategies. 

I think the problem of having unpredictive strategies in Generator can minimized if we add in some simple "predictive" rules as I've mentioned before such as

Rule 1) faster period input must be smaller than longer period input of the same indicator or measurement i.e. MA with MA for long/buy position vice versa for short/sell position. This rule is for options such as

a) rising, higher than, cross over, change direction, convergence

Rule 2) Rule 1 need not necessary apply if use as divergence (because 2 negative makes 1 positive)
but we should rule out such redundant step.

Rule 1 can be apply to indicators such as MA, Oscillators, and any others using the same instrument/measurement for faster and longer time period.

As I've told Footon, my theoretical framework is actually very simple.  If I were to tell you guys, you probably going to slap yourself for not seeing it and you would realise it doesn't take a genius to come up with it.  Is the lacking of thinking.

I think in the olden days without calculator, people are "fascinated" with numbers.  They are curious to see how numbers "behaves" and hence searching for formulas to understand their behavioural patterns....with the invention of devices, we lost this "fascination" and want the machine to think and formulate for us and just give us the results.  We lost the art of understanding.

That's why smart device makes us stupid

Actually, if we apply just this simple rule(s) from above, we reduce tons of unnecessary and unpredictive strategies from Generator and wrongful "Optimization".

Of course there are others rules to add but I strongly believe, if Popov were to add this simple rules above, Generator can develop better and predictive strategies.  I've been using Generator recently, with this simple rules apply, I strongly believe Generator is capable of churning out profitable Ea.

One of the ways to improve Generator results is to add in these rules yourself.  How?  I input my own rules as opening conditions and lock these opening conditions. 

In order to focus on developing predictive strategies, we work on the opening conditions only.  Therefore, set Reverse as the next opposite direction signal.  The rational is to reduce "unpredictive" strategies.  Because predictive strategies can work in both ways better than unpredictive ones.  And Trailing Stop Limit as your only closing and lock that too.  Then in the Generator Common setting, choose the maximum number of opening conditions to use.  Count the ones you have added in your opening conditions and add in how many more opening conditions you would want Generator to add.  Bear in mind, the Generator may not necessary use all the maximum number so it's ok to put in more than less.

Next, select indicators that uses Trend, CCI, Price Volume  etc so narrow your search only for certain types of strategies...in my case, I aim for Breakout, I don't like to use Reversal from support and resistance though because you are betting on a reversal and you would be caught big if a trend break out of the resistance.

Hence, when I add in my own opening conditions to serve as a guide to the Generator, hoping to narrow or direct the Generator towards certain predictive direction, I have rather good results, the generated EA are more predictive and profitable in my demo test.

I agree with Footon, that re optimization makes it worse, it's like repeating Generator's work but this time just focus on the existing indicators.  I won't mind if the optimizer also has the added predictive rules apply to both Generator and Optimizer.

If only Popov emphasize optimization in an area which is more "useful" and predictive in my opinion.  I've mentioned it before and I called it as "mini optimizer"

If most of us are clueless about which parameter to use, do you think most user also know which base price or smoothing period to use too?  That's even more difficult for a layperson to know.  My ideal optimizer, is that each time I key in my input period, that FSB (with built in Optimizer, user has the option to enable or disable it) will automatically choose the best combination of smoothing and base price for me in consideration of all other opening conditions, i.e. FSB will adjust all the rest of the opening and closing condition's base and smoothing to optimize them. (I badly need it, otherwise it is very mentally draining and tedious work for me to manually adjust the base price and smoothing methods for all my opening and closing conditions, especially when I have about 12 or more opening conditions to work with...developing each EA is so exhausting for me)

This is what I call real optimization.  It doesn't goes against my theory or predictiveness but strengthen my input by searching for the best base or smoothing methods to use.  I'm just a lay person, I know which input to use but I'm not a mathematical genius to know for each input which smoothing method or base price is most suitable or fit into what I see in my chart.

FSB has tremendous potential...Popov need to add in some rules to create an "artificial intelligence", AI into FSB, transforming it from a smart calculating device into smart AI device.  I think at this stage, most people forgot that FSB is a smart calculating device and they were hoping for FSB to "think" for them.  Without any predictive rules input to FSB, it's not an AI as what we wish it to be.