Topic: machine learning and curvefitting

Hi I found this webseite about machine learning and curvefitting.

I don´t understood this all, but it can be interested for someone.

FSB need more curvefitting tests.

https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!forum/machine-learning-for-trading-system-development

thomas

https://monitortool.jimdofree.com/

Re: machine learning and curvefitting

What curve fitting tests do you think are required..?

It could be that such tests are beyond what this software was created for.

What curve fitting tests have you applied in the past with this software or other software.

We have to give Popov some idea as to what you are looking for so he can make a decision.

A lot of the systems on the site you mention are related to data mining which will not fit into FSBPro.

I think, you may have to purchase some other softwares to accomplish your goal.

I wonder why you feel that the 'Multi Markets' does not help with curve fitting.

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....

Re: machine learning and curvefitting

I am thinking that the best curve fitting prevention can be developed by an individual.

Use few indicators as these are what cause the curve fitting.
Use the original values for the indicators or else lock them during generation and optimization.
do a walk forward over several sets of data ie use ten sets of dates.....
use the monte carlo and multi markets on each result.
Be mindful of the traps in time frames less than 60 minutes.

expect that a strategy is not going to give 100 per cent profitable trades

I am thinking that any tests for curve fitting are going to be similar to monte carlo being done on several different time periods.....  with some little variations.

I actually do not think we are going to get better than what we have already.

For my own trading....... I work with a system for months, make little changes and test it and test it and test it.

That Multi Markets tool  I use all the time......

I may be wrong but it seems to me that Multi Markets is the moment of truth. If I can find a system that passes Multi Markets. I am pretty sure I have reduced the curve fitting.

Perhaps there is a couple things can be added to Monte Carlo, I do not know what else can be added, I am not that familiar with it. There seems to be a lot of options to play with.

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....

Re: machine learning and curvefitting

I am using StrategyQuant.

For curvefitting I am using the following things.

If  I am searching H1 Strategies I do the following tests

a) Check with timeframe M30
b) Check with timeframe H4
c) Check with corresponding currency pair
d) Robustnesstest
e) Check with unseen Data
f) Sometimes WFA Analysis
g) I Look into the Strategies and decide. I look if its made sense or not.
h) Build a portfolio of non correlated (different Strategies) Strategies

a) till e) can be done automated.

If you do this I will get a portfolio of this.
http://www.myfxbook.com/members/tnickel/mt4-1111576038-jfd8/1342972 
(I don´t know how long this portfolio works)

a) till e) can be done by hand with the FSB pro. But this will be an endless task. I will never find such portfolio.

What I need is the following thing.

a) Generation Process who generate me a list of 15000 Strategies
b) I will filter all Strategies with M30 filter (for example 10000 left)
c) I will filter rest Strategies with H4 filter (for example 8000 left)
d) I will filter rest Strategies with corresponding currency pair (4000 left for example)
e) I will filter rest Strategies with a Robustnesstest (200 Strategies are robust)
f) Check with unseen data (50 Strategies are left)

In the next Step I look in every strategy and decide.

For example 30 Strategies are left.

In the next Step I use Quantanalyser to build a Portolio (http://www.strategyquant.com
/quantanalyzer/)

I choose 4 Strategies out of this.

This is my current task to find strategies. I think for curvefitting we need more. I don´t know excactly. Some matematical Algorithms or something simmilar.

It will be good If I can do the same generation and filtering process with the FSBpro. At the moment it is not possible because the fsb pro have an different userinterface to handle the strategies.
thomas

https://monitortool.jimdofree.com/

Re: machine learning and curvefitting

Maybe Marc can develop the tests that you require.

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....

Re: machine learning and curvefitting

Hi,
I found out that the best curvefitting test is to make backtest with unseen data.
The equitycurve should be nearly the same as with the seen data.

This test is very easy and better than many theroretical solutions.

For example.

H1 Timeframe
a) Calculate 1000 Strategies from 2008-2013
b) make a backtest with every strategy vom 2013 till now

This test is different than the build in oos test.

If I search  1000 Strategies. I have to do 1.000.000 Calculations.
With every test the internal OOS has been done.
-> This is curvefitting.
I do the OOS check 1 Million times.

It is better to check with unseen not so much times.

So I take only the result of 1000 Strategies and check this strategies with unseen data.

------------
For M1 you can do this.

Generation: 1 Month
Check: 2 years

This works with M1 strategies.

thomas

https://monitortool.jimdofree.com/

Re: machine learning and curvefitting

I forgot something.

If you are working on M1 timeframe you should have the realtickdata from the broker who you want to trade.
It is not possible to take Duka data. Search EAs which this data and trade this found EAs on pepperstone. This don´t work.

At H1 or bigger timeframes you can use Dukadata.

this was my experience in the past.
thomas

https://monitortool.jimdofree.com/

Re: machine learning and curvefitting

I make my own walk forward using the dates in market data, I just change the periods and it is very fast. even faster than walk forward in other softwares.

I check the journal each time to see where the bad trades are......

and I check to see on the chart where the stagnation is and find the cause..

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....

Re: machine learning and curvefitting

tnickel wrote:

I am using StrategyQuant.

For curvefitting I am using the following things.

If  I am searching H1 Strategies I do the following tests

a) Check with timeframe M30
b) Check with timeframe H4
c) Check with corresponding currency pair
d) Robustnesstest
e) Check with unseen Data
f) Sometimes WFA Analysis
g) I Look into the Strategies and decide. I look if its made sense or not.
h) Build a portfolio of non correlated (different Strategies) Strategies

a) till e) can be done automated.

If you do this I will get a portfolio of this.
http://www.myfxbook.com/members/tnickel/mt4-1111576038-jfd8/1342972 
(I don´t know how long this portfolio works)

a) till e) can be done by hand with the FSB pro. But this will be an endless task. I will never find such portfolio.

What I need is the following thing.

a) Generation Process who generate me a list of 15000 Strategies
b) I will filter all Strategies with M30 filter (for example 10000 left)
c) I will filter rest Strategies with H4 filter (for example 8000 left)
d) I will filter rest Strategies with corresponding currency pair (4000 left for example)
e) I will filter rest Strategies with a Robustnesstest (200 Strategies are robust)
f) Check with unseen data (50 Strategies are left)

In the next Step I look in every strategy and decide.

For example 30 Strategies are left.

In the next Step I use Quantanalyser to build a Portolio (http://www.strategyquant.com
/quantanalyzer/)

I choose 4 Strategies out of this.

This is my current task to find strategies. I think for curvefitting we need more. I don´t know excactly. Some matematical Algorithms or something simmilar.

It will be good If I can do the same generation and filtering process with the FSBpro. At the moment it is not possible because the fsb pro have an different userinterface to handle the strategies.
thomas


Hii Thomas, i was using SQ also, currently waiting on the new version.., i got a portfolio from it already, working pretty well!,

although i want to expend my empire, mind & do not waste time in the meanwhile so i have started working with FSBPro also, both software are very unique in each way, and they are the best in the market right now, althogh i found FSBPro much more faster and flexible then SQ (well.. at least for the current time),

i have reed your post and noticed your way of working with SQ can be also implemented right now with FSBPro,
im working nearly the same way as you btw smile

1. Generate few lists of 100 algorithms each (using 30%~40% OOS), *Already suggested to Mr. Popov to rise this limit..
2. Checking all the lists i got on: 1 TF Above & 1 TF Below + different currencies + MonteCarlo AKA RobustnessTest.

This whole procedure of Section #2 can easily be done by that recently added feature 'Multi Markets' smile,
the only limit is the procedure of Section #1 and thats the limit of the 100 algorithms per collection...

here are my suggestions to Mr. Popov including raising the limit of the collections list: http://forexsb.com/forum/topic/3995/fsb-pro-wish-list-requested-features/page/12/

hope it helped smile, good luck to you! and good luck for us all, i hope at least the limit would be raised though smile

10 (edited by GD 2016-01-20 02:09:17)

Re: machine learning and curvefitting

Dear Tnickel
I read your experience from StrategyQuant.

You have to remember that every time you decide a selection in your methodology, actually you make a kind of large fitting.
In your methodology you have made a lot of it.

The second is that StrategyQuant custom indicators cannot install by a programming language but by data. I do not have it.

The third is that StrategyQuant is very expensive for the job it does. It started by using WFA. Statisitcs is the result of rules and can be the same for different ones. Do we use appropriate statistics?

My self I have not so much experience in FSB Pro but I have experience in fitting (you cannot imagine) and Prediction Theory and Applications.

I see FSB Pro to have tremendous power if you know what to do and Mr.Popov has a philosophy to share and listen. It is a kind of my style also.

FSB pro has more problem in presentation of its large abilities... Mr.Popov is busy to develop and he forgets some things to do. I had same problem also in my job when I was younger.

Practically I found 3 EAs (one per day) in short time following basically an idea of another member Hannahis. They work nicely and they win 1000 JPY using 0.01 lots.

FSB pro can be used at first for any further EA development. I am sure on that.

Anyway everybody is free to keep his style and methodology as much he wins and he feels comfortable.
I like to listen new ideas and keep the best of them.

Thank you for share

Re: machine learning and curvefitting

Hi karish,
I hope that mr popov will expand the limit from 100 to 50000. I think this will be enough.

thomas

https://monitortool.jimdofree.com/

Re: machine learning and curvefitting

GD wrote:

Dear Tnickel
I read your experience from StrategyQuant.

You have to remember that every time you decide a selection in your methodology, actually you make a kind of large fitting.
In your methodology you have made a lot of it.

The second is that StrategyQuant custom indicators cannot install by a programming language but by data. I do not have it.

The third is that StrategyQuant is very expensive for the job it does. It started by using WFA. Statisitcs is the result of rules and can be the same for different ones. Do we use appropriate statistics?

My self I have not so much experience in FSB Pro but I have experience in fitting (you cannot imagine) and Prediction Theory and Applications.

I see FSB Pro to have tremendous power if you know what to do and Mr.Popov has a philosophy to share and listen. It is a kind of my style also.

FSB pro has more problem in presentation of its large abilities... Mr.Popov is busy to develop and he forgets some things to do. I had same problem also in my job when I was younger.

Practically I found 3 EAs (one per day) in short time following basically an idea of another member Hannahis. They work nicely and they win 1000 JPY using 0.01 lots.

FSB pro can be used at first for any further EA development. I am sure on that.

Anyway everybody is free to keep his style and methodology as much he wins and he feels comfortable.
I like to listen new ideas and keep the best of them.

Thank you for share

Hi GD,
I don´t have so much knowledge about curvefitting.
I don´t understand "You have to remember that every time you decide a selection in your methodology, actually you make a kind of large fitting.
In your methodology you have made a lot of it."

What do you mean with that ?
How can I improve my filtering?

I think SQ is not too expensive. It is cheap. There is a much work in this Software.

You found 3 Eas. How long are the 3 eas trading ?
I found many EAs in the past. But they work most time only a short time. 1,2 or 6 Months.
Most EAs were curvefitted in the past.

I think curvefitting is the biggest problem. If we solve this we can earn much money.
thomas

https://monitortool.jimdofree.com/

13 (edited by hannahis 2016-01-22 16:52:23)

Re: machine learning and curvefitting

Dear all,

If we rely on  curve fitting EA, we let never learn the fundamental theory of trading.  A good and strong fundamental, theoretical EA beats most if not all curve fitting EA.

Start depending on yourself and less on the machine.  Start looking at various indicators and adjust the parameters till it fit into the price changes and you will find a consistent and dependable theory you can always rely on...no more curve fitting but solid and sound theoretical trading strategy which you have lots of confidence on.  That's the fruit of my labour and much prayers.

The main reason, I know what I'm doing and keep improving my EA results is because I know what I'm doing...I have a framework or theory in mind and I keep working and improving it and hence, my EA gets better and more accurate (now working on closing accuracy as I've so call "master" my opening conditions).

Because of my years of working on my EA, I have a better understanding when I look at a EA generated by FSB, I know which is theoretically sound and which aren't likely to be.  This is a wonderful skill and knowledge, FSB can't impact to you.  But FSB can give you a lead or a headstart by giving you some possible combination and you need to go back to your indicator and look at the settings to see whether the combination makes sense or not.

Trust me, if you send more time on looking and working out your parameters, you'll have better chance of nailing the EA of your dream than to rely on curve fitting EA, which you are clueless whether the EA is theoretically sound or not.

I'm glad I rely on due diligence to find a strong trading theory whereby I can have full confidence in.  As a result, it is a skill no one can rob me off and an advantage on how to use FSB to it's fullest potential.  Remember FSB is just a powerful tool.  And bad worker always blame his tools.  FSB is a brilliant and a genius work of Popov and his team...I fully respect him and his big and generous heart.  The rest is up to us to maximize it to our own knowledge and if you expect the machine to think for you...you will always end up clueless of what you are doing...you have to learn the art of trading and acquire the skills.

And it's a skill I'm going to impart to my children, a skill I'm so proud of and nobody can rob me off and a skill my children would benefit greatly and put them on a head start, a skill the university can't teach them and a perpetual skills that can earn them a living.  A tremendous rewards for those who labour to gain such skill which no nor FSB can give to you...you obtain yourself.

There are two types of traders, professional and gamblers.  One has the skill and the other depends on luck.  At the end of the day, you have to develop a strong trading theory for yourself, if you don't have, you aren't a trader, you are a gambler, you are still clueless on how to trade.  Without FSB, I can still look at the chart with indicators and know which direction to trade, do you?

Re: machine learning and curvefitting

@hannahis
I see this so too. But the problem is. I don´t have so much knowledge at forex ea development at the moment. I have to learn it and it takes 5 till 10 years.
If I have a good mechanis to filter all curvefitted strategies I can do this automaticaly.

https://monitortool.jimdofree.com/

Re: machine learning and curvefitting

@GD
ok I understand this "You have to remember that every time you decide a selection in your methodology, actually you make a kind of large fitting"
This is clear.
So I have to remove the filter M30, H4 and different currency-filter

New Filtering:
Make only tests with: Robustnesstest and 1+1 Filter
And add an additional Filter with different Tickdata from different Broker
I will check this.
thomas

https://monitortool.jimdofree.com/

16 (edited by hannahis 2016-01-25 13:12:23)

Re: machine learning and curvefitting

tnickel wrote:

@hannahis
I see this so too. But the problem is. I don´t have so much knowledge at forex ea development at the moment. I have to learn it and it takes 5 till 10 years.
If I have a good mechanis to filter all curvefitted strategies I can do this automaticaly.

Hi Tnickel,

I'm not so talented like you in using mechanics.  How do you filter out curve fitted strategies?

By using Generator or Optimizer, aren't we creating curve fitting strategies in the 1st place?

How can we filter out curve fitting strategies unless we introduce certain trading rules to the machine to filter out "unsound theoretical" strategies...so isn't the art of filtering depends on one's knowledge of trading theory/rules?

Guess I've much to learn from you...do share your ideas in filtering out such strategies...would be very interested to hear.

As for EA development...don't need so many years...We just need adopt one fundamental concept which you think stood the test of market volatility and develop EA based on this concept.

As for my case, my EA development wouldn't have taken such a long time.  First my theoretical framework is the most important aspect took me only 3 to 6 mths to formulate.  The subsequent years, I was just waiting for FSB to introduce LTF and then waiting for the logical conditions to increase from 8 to 26.  Basically, waiting for technology to "catch" up with my vision.  Thank God for FSB, otherwise, I'm stuck with my theory and no way to carry out into practice, cos computer programming is definitely no my forte.

Re: machine learning and curvefitting

@hannahis,
I don´t have a full concept to filter all curvefitted strategies. At the moment I work only with SQ (Strategy Quant). I thing the user "GD" has more knowledge about curvefitting.

To produce not so much curvefitted strategies a good setting of building blocks is very important. If a generator have a good building set with building blocks who make sense the strategy generation is fast.
The next Step is to test the found strategies on unseen data. For example.
You generate H1 Strategies from 1.1.2008-1.1.2012 you check the found strategies with data on
1.1.2012-1.1.2014

The second are the Robustnesstests. FSB have Montocarlo Analysis for RT.

The next check is to check the Strategies with different Broker data. For example with Data from Pepperstone and with Data from Tickmill. Please use realtick and use the same GMT.
The Trades should have nearly the same open and closetime.

If you use the Filter: M30, H4, and a different Currency pair we have one problem. You need Longer to find such Strategy. If you search longer you produrce more curvefitting.
I thing this was the statement from "GD"

@GD
Is this correct ?

I think curvefitting is very complex. I found some mathematical description and tests (bootstrap etc.) I don´t understand this all.

One of the best curvefittingtest is the human brain. A human with knowledge kann look and decide if this rules makes sense or not.
thomas

https://monitortool.jimdofree.com/

Re: machine learning and curvefitting

On other thing I found out about curvefitting.

You need at least 100000 Bars to prevent curvefitting, I made some Tests with random generation.
If you search in H1 Timeframe your get
24*365=8760 Bars per Year. So you need a generation period of 11 years.

If you use smaller timeframes you need shorter time periods.
thomas

https://monitortool.jimdofree.com/

19 (edited by GD 2016-01-25 17:04:56)

Re: machine learning and curvefitting

Tnickel

I have and another occupation, where I use a lot of curve fitting. I did not learn fitting by using fsb.
A minimum in trading is a complex situation as there are not standard rules (one theory) in trading.
The best is to remember that most of the traders use the most common ideas and they apply them in their trading.
Hannahis is right to try a strategy but how good it is only statistics can answer and research on existing data.
Experience and study and keep notes.
The question is "WHEN to stop fitting".

Re: machine learning and curvefitting

tnickel wrote:

I think curvefitting is very complex. I found some mathematical description and tests (bootstrap etc.) I don´t understand this all.

I'm currently into researching it. Bootstrapping and other tests are not about curvefitting so much, but a statistical evaluation. Basically it revolves around randomness and probability, and the main point being that data-mining (essentially FSB generator) comes with a number of biases, which result in non-fit strategies. The task is to use statistical evaluation methods to determine whether the strats have genuine predictive ability or the results are random (pure luck in in-sample, catastrophic in out-of-sample or live trading).

21 (edited by hannahis 2016-01-26 05:00:49)

Re: machine learning and curvefitting

Footon quote
...(essentially FSB generator) comes with a number of biases, which result in non-fit strategies. The task is to use statistical evaluation methods to determine whether the strats have genuine predictive ability or the results are random (pure luck in in-sample, catastrophic in out-of-sample or live trading).

Footon, what do you mean by FSB "non-fit strategies", do you mean not genuine predictive ability?

If that's what you mean, I agree with you and that's the reason from the very start, I don't use generator and optimizer for my strategies at all.  However recently, I want to see whether I can Generate good strategies by the selection of certain indicators that are less "ambiguous", that would reduce the element of "unpredictive" strategies. 

I think the problem of having unpredictive strategies in Generator can minimized if we add in some simple "predictive" rules as I've mentioned before such as

Rule 1) faster period input must be smaller than longer period input of the same indicator or measurement i.e. MA with MA for long/buy position vice versa for short/sell position. This rule is for options such as

a) rising, higher than, cross over, change direction, convergence

Rule 2) Rule 1 need not necessary apply if use as divergence (because 2 negative makes 1 positive)
but we should rule out such redundant step.

Rule 1 can be apply to indicators such as MA, Oscillators, and any others using the same instrument/measurement for faster and longer time period.

As I've told Footon, my theoretical framework is actually very simple.  If I were to tell you guys, you probably going to slap yourself for not seeing it and you would realise it doesn't take a genius to come up with it.  Is the lacking of thinking.

I think in the olden days without calculator, people are "fascinated" with numbers.  They are curious to see how numbers "behaves" and hence searching for formulas to understand their behavioural patterns....with the invention of devices, we lost this "fascination" and want the machine to think and formulate for us and just give us the results.  We lost the art of understanding.

That's why smart device makes us stupid smile

Actually, if we apply just this simple rule(s) from above, we reduce tons of unnecessary and unpredictive strategies from Generator and wrongful "Optimization".

Of course there are others rules to add but I strongly believe, if Popov were to add this simple rules above, Generator can develop better and predictive strategies.  I've been using Generator recently, with this simple rules apply, I strongly believe Generator is capable of churning out profitable Ea.

One of the ways to improve Generator results is to add in these rules yourself.  How?  I input my own rules as opening conditions and lock these opening conditions. 

In order to focus on developing predictive strategies, we work on the opening conditions only.  Therefore, set Reverse as the next opposite direction signal.  The rational is to reduce "unpredictive" strategies.  Because predictive strategies can work in both ways better than unpredictive ones.  And Trailing Stop Limit as your only closing and lock that too.  Then in the Generator Common setting, choose the maximum number of opening conditions to use.  Count the ones you have added in your opening conditions and add in how many more opening conditions you would want Generator to add.  Bear in mind, the Generator may not necessary use all the maximum number so it's ok to put in more than less.

Next, select indicators that uses Trend, CCI, Price Volume  etc so narrow your search only for certain types of strategies...in my case, I aim for Breakout, I don't like to use Reversal from support and resistance though because you are betting on a reversal and you would be caught big if a trend break out of the resistance.

Hence, when I add in my own opening conditions to serve as a guide to the Generator, hoping to narrow or direct the Generator towards certain predictive direction, I have rather good results, the generated EA are more predictive and profitable in my demo test.

Re: machine learning and curvefitting

In a way, the market repeats itself over and over.

Sometimes the worries about curve fitting can be appeased by examining carefully what happened 10 or 15 years ago, or even 20 years ago.

A large enough sample, examined for moves that are not minute, will give results that are not going to be bothered by curve fitting.

For example, a couple days ago H posted about support and resistance, the same areas from 1990 will be support and resistance in 2050.

However, the oscillators may not work the same way on 2050 as they did in 1990. and that is where the curve fitting enters.

A trend this year will work as well as a trend  from 1999, if there are higher highs and higher lows there is a bullish trend, and it will stop at a historical area of support or resistance.

I do not have statistics or software to work with for that purpose so I can not give results of research.

Just my experience to combine trends with S&R to get results.

H is correct re optimization, it spreads the disease of curve fitting.

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....

23 (edited by hannahis 2016-01-26 05:29:21)

Re: machine learning and curvefitting

Blaiserboy wrote:

H is correct re optimization, it spreads the disease of curve fitting.

I agree with Footon, that re optimization makes it worse, it's like repeating Generator's work but this time just focus on the existing indicators.  I won't mind if the optimizer also has the added predictive rules apply to both Generator and Optimizer.

If only Popov emphasize optimization in an area which is more "useful" and predictive in my opinion.  I've mentioned it before and I called it as "mini optimizer"

If most of us are clueless about which parameter to use, do you think most user also know which base price or smoothing period to use too?  That's even more difficult for a layperson to know.  My ideal optimizer, is that each time I key in my input period, that FSB (with built in Optimizer, user has the option to enable or disable it) will automatically choose the best combination of smoothing and base price for me in consideration of all other opening conditions, i.e. FSB will adjust all the rest of the opening and closing condition's base and smoothing to optimize them. (I badly need it, otherwise it is very mentally draining and tedious work for me to manually adjust the base price and smoothing methods for all my opening and closing conditions, especially when I have about 12 or more opening conditions to work with...developing each EA is so exhausting for me)

This is what I call real optimization.  It doesn't goes against my theory or predictiveness but strengthen my input by searching for the best base or smoothing methods to use.  I'm just a lay person, I know which input to use but I'm not a mathematical genius to know for each input which smoothing method or base price is most suitable or fit into what I see in my chart.

FSB has tremendous potential...Popov need to add in some rules to create an "artificial intelligence", AI into FSB, transforming it from a smart calculating device into smart AI device.  I think at this stage, most people forgot that FSB is a smart calculating device and they were hoping for FSB to "think" for them.  Without any predictive rules input to FSB, it's not an AI as what we wish it to be.

Re: machine learning and curvefitting

do you mean that you want the optimizer to be able to work on only one part of the indicator...../

Like you will preset a couple parts and then just have only, say, the number of periods optimized..?

Like you would not allow the comparison or shift parts to change. is that what you need...?

Like for CCI convergence divergence, you want to preset the condition and allow only the '14' to be optimized?

That may require a complete rewrite of the program.. or could be really simple, I have no idea.

I wonder if you could ask Footon to lock a couple indies for you, except for the part that you want flexible, I am sure is simple to remove the options.

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....

25 (edited by hannahis 2016-01-26 05:44:56)

Re: machine learning and curvefitting

Yes Dave,

It's really that simple and focused, it just "optimize" the smoothing and base price but don't change your rules, be it shift or repeat.  So that you understand how each period behaves and thus gain a true knowledge of what works and what doesn't.

So that you can purely focus on getting your input right by looking at the results and adjust it yourself the input till you "landed" on the correct input.

Why most of us doesn't know what input to use because there are just too many factors to change (i.e. if you look at the number of probabilities from the combination of base price and smoothing periods) and we don't know which input to use.  Even if you get the input right but the combination of smoothing method and base price wrong...your EA is not going to work and hence, people jump from one input to another input, when they should be adjusting the smoothing and base price instead.

If these probabilities are "taken care of by optimization", we can then keep adjusting our input till we see which is the best period to use.

Frankly, if I tell you my input but you use the wrong combination of smoothing and base price, your EA is still unprofitable.  Every time, when I key in my input (they are always consistent with my theory), I kept adjusting the smoothing and base price till I turn my EA from unprofitable results to profitable results...trust me, it's a whole lot of difference.  And sometimes when I add in new opening condition, I have to adjust this new addition for the best results and then go back again, 1 by 1 to the previous opening conditions and adjust them again to get the best combination...mind you, it is manual optimization and tremendously tedious.