• Registered: 2016-05-16
  • Posts: 190

Topic: k-fold cross validation

Hi,
Maybe someone heard of "k-fold cross validation" for trading system (model) robustness test? What do you think of possibility to implement it to FSB? It was said that this method is the most advanced test to check your models predictive power. How I understand it is like advanced walk forward tool. If I understood correctly it could help finding best/robust parameters without curving data too.

If that testing model would help to catch some curve fitted / over fitted systems objectively I believe that a lot of users will be very happy to have it in FSB. Moreover if there are some users like me interested in this, we can pay mr. Popov hourly fee for programming it. I believe there should be some libraries already written, at least in R/Python there is, maybe mr. Popov would not need to do it from zero. For me it would be ok to have this tool on different/exclusive version of FSB, if mr. Popov do not want to pollute current version with not so sure testing methods...

Some links for more info:
Heard in this interview - https://www.youtube.com/watch?v=cqEncH4_5ls  , go to time 42:50 about this topic
https://www.analyticsvidhya.com/blog/20 … -python-r/
https://www.quantstart.com/articles/Usi … on-Setting
.. I believe everyone here know how to use google for more info if needed smile

  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 7,995

Re: k-fold cross validation

Hello Irmantas,

I have plans for implementing Walk Forward for long time and I think to implement it in the next major release of FSB (v4). I'll examine the links you have posted (thank you for that) and will decide how I can use this methods to improve FSB Pro.

Popov's Website