• Registered: 2016-05-16
  • Posts: 190

Topic: What's your steps for robustness/avoiding curve fitting?

Hi,
I am very happy that I found FSB and this forum. Was way too long with mt4.... Hope that using FSB will make some significant changes in searching for working FX strategies smile

After using this great program for some time I am stuck with curve fitted systems, which do not pass simple OOS test. I know that this is probably the biggest systematic traders problem, how to find robust systems that works with unseen data. I honestly tried to find answer what steps to take to prevent curve fitting, but for now I have conflicting information.

Some authors use WFA (walk forward) for robustness, if it not pass it, leave strategy alone (why FSB still do not have this future?). I can remember recent pain doing it manually with mt4 for one week and was ready to pay big bucks for WFA software, but could not find it big_smile Some use ALL data and no out of sample, defending that using OOS or WFA is useless and it is creating data mining bias ... Other recommends to use no exits to see if there is an entry edge, and slight 1-3 variable optimization with much data as possible and for one variable at least 50 trades...

I am kinda lost here. Tried few approaches listed above but no success (did not found any WFA passing system). I am getting all the time PL curves like these:

http://s32.postimg.org/9s5hewy4x/in_sample.jpg

Maybe someone who already have working real time system portfolio can share your experience and say what steps you use? smile How do you create robust EA, which works real time? What test you make? Do you optimize (full or only few parameters)? Do you use OOS test? WFA? How much systems you must analyse before you find something working witch pass multi market and OOS tests?

Thanks in advance!

PS. Is it possible to work with more than 50k bars? In M5 it only near a year. And if a system gets rarer signals it is not enough data for slight optimization. While reading in this forum I saw that people used like 700 000 bars...

  • Registered: 2016-05-16
  • Posts: 190

Re: What's your steps for robustness/avoiding curve fitting?

No need to answer for last question for more bars for testing. Just found an answer smile

  • footon
  • footon
  • ◄≡≡≡►
  • Offline
  • Registered: 2009-08-17
  • Posts: 2,965

Re: What's your steps for robustness/avoiding curve fitting?

Irmantas, it is hard to avoid curvefitting, one way is to have a large number or strats, which then get run through Multi Markets and Monte Carlo tools to verify if they are worth something. Essentially what we do is datamining, which in turn can produce a lot of worthless strats and the task is to find those rare diamonds in the rough. That's my take on this smile

4 Reply by Irmantas (edited by Irmantas 2016-05-18 15:06:58)

  • Registered: 2016-05-16
  • Posts: 190

Re: What's your steps for robustness/avoiding curve fitting?

Thanks footon for answer smile

Well I think I will be trying to do like this and will see if something will work. Not sure how many systems will need to created like this to find pasing one, but I think a lot.

*use more data bars on <1H timefranes if needed to get at least 1-2k trades for couple of years
*develop strategies without exit, to avoid curve fitting. After that manual adding exit, and use slight optimization
*use OOS test, if pass then
*try multimarkets and montecarlo
*after that demo for some time

Maybe try to play with WFA manually, it will be fast with FSB, but not sure if this needed, or if I ever find passing systems smile

have a nice day

  • footon
  • footon
  • ◄≡≡≡►
  • Offline
  • Registered: 2009-08-17
  • Posts: 2,965

Re: What's your steps for robustness/avoiding curve fitting?

Irmantas wrote:

Thanks footon for answer smile

*develop strategies without exit, to avoid curve fitting. After that manual adding exit, and use slight optimization

No problems.

You cannot develop strats without exits, otherwise you'd have strats with one open position from start until the end.

  • Registered: 2016-05-16
  • Posts: 190

Re: What's your steps for robustness/avoiding curve fitting?

footon wrote:
Irmantas wrote:

Thanks footon for answer smile

*develop strategies without exit, to avoid curve fitting. After that manual adding exit, and use slight optimization

No problems.

You cannot develop strats without exits, otherwise you'd have strats with one open position from start until the end.

I meant "close and reverse" when opposite signal develops.