Syndata is not a new idea, some years ago, maybe even more than 5 years ago there was similar effort done in this forum. I have to dig deeper to find that topic. To my recollection it was not a particular success (I'm talking about combining multiple pair data into one), but it doesn't mean it's a dead idea.

Having data "chopped" by taking into account the data itself is a very interesting idea! I tried to get this thought across in the other thread, but rather cryptically I guess. I gather we all see dealing with multiple collections that the consistency is not there and one of the reasons has to be arbitrary data horizon selection process. If we use moving windows, which are equal in length or follow exact distribution in training and test data (75/25 etc), we essentially get "varying" data. It can be defined in many manners. And, even more importantly, given the data nature of fractality, be defined and looked at a different level or depth. I have been thinking about this for some time and hope to soon start testing this idea. Hopefully we can share our findings?

A few more thoughts. When we talk about market change ("my strat failed because of market changed its behaviour"), then is it really so? One could dig up a chart from 2000 and other one from 2017 and look at them to note the differences. They look alike, don't they? Given this observation I state that it is actually quantifiable if there this so-called market change and what it is about. And if there is one, it can be exploited. To widen this thought - our biggest advantage is having the data itself! it is literally packed with information, only "problem" is to ask the right questions. And it will take a whole lot of work to get the answers.

This is really interesting to me.  I'm still formulating some thoughts -- so the post below probably is more rambling than usual.

I've read elsewhere it is common practice to "refresh" a strategy from time-to-time -- and I assume that means importing it back into FSBPro, adjusting the DataHorizon to use more recent data, and then re-optimizing the indicator settings (i.e. curve-fitting to more recent data).  What I don't understand is how a strategy could go bad -- it's just an equation.  And then after reading Hannah's post things sort of clicked -- the EA doesn't go bad, it's the input data that has changed.

I had a couple of questions I'd like to ask to see if we are on the same page...
1. When we apply optimization to get the best indicator settings for a particular DataHorizon, then that is equivalent to curve-fitting, right?  And, if so, then why does curve-fitting have such a bad name?
2. If a strategy is optimized (or curve-fitted) using EURUSD data, then why would we want to use Multi-Market to judge whether it is good enough to trade?  That seems unfair -- sort of like teaching someone how to speak Spanish but then giving them a test in French and failing them if they don't pass.

Footon -- you bring up an interesting issue of whether or not changes in input data are quantifiable and, if so, whether they could be exploited.  As you point out this would be very difficult, but by asking the right questions it might be possible to eke out certain patterns or events that provide a clue as to the type of market we are in and then use that either to turn on or off certain EAs or use those clues to help an EA better navigate the incoming data.

Popov -- I like the idea of using indicators to monitor various aspects of a changing market.  I wonder how these could be used by an EA to adapt to changing data.

I've written a simple program that allows me to merge 2 or more *.csv files into one (e.g. H1 EURUSD and USDCHF from the same DataHorizon) and normalize the prices to take into account the difference between EURUSD and USDCHF.   In the short term I plan to take a few strategies and optimize them against EURUSD, USDCHF and the hybrid EURUSD-USDCHF -- and then compare them in MM.  I'm curious whether a strategy optimized against the hybrid data yields better statistics than the same strategy optimized against just EURUSD or USDCHF.  Does that make sense?

Brilliant idea!

This is probably the most promising idea on the forum as far as generation goes. I imagine we can brainstorm many opposing components that cause our EAs to "average out" and muddled values, and separate them into meaningful groups of data, then recombine them as a whole to run them through generation.

When recombining the elements, I don't think it would hurt to include certain data history more than once if it is "important" within two separate components. (?) Does this make sense? (Sorry, I've had little sleep lately)

This is yet another type of filtering and this is the type of thinking that we need to make a revolutionary step.

Indicators, by their very nature, lag behind the market. It seems logical to me that if indicators do indeed work as intended, that identifying trending/ranging markets (even though our information is lagging) can only boost our efforts significantly.

Thanks for digging this up -- interesting reading.  In the end it seemed like that thread just died -- never came to any conclusion.  I also noticed that they really got hung-up on the details of how to "stitch" together the data -- like worrying about leapyear -- seriously?

Stitching together data from multiple *.csv files is relatively straightforward, but I did have to come up with a way to make the date and timestamps continuous -- otherwise FSBPro wasn't happy.  C# has a few DateTime methods that make that easy.  I didn't worry about using the original timestamps -- I started at 2010 and just worked forward.

I stitched together data from EURUSD and USDCHF.  At the transition point (i.e. at the stitch) there is a HUGE drop.  Interestingly, some of my strategies handled that gracefully, while others dropped off the planet.  I don't think it is necessarily important to have a smooth transition -- it is only one point -- but then I found a way to make the transition smooth by simply taking the "delta" between the last EURUSD and first USDCHF and adding it to all the USDCHF prices.

For now I'm just curious to see whether a strategy optimized using hybrid EURUSD-USDCHF data yields better Multi-Market results than a strategy optimized using only EURUSD data.  I'll let you know what I find.

So if I understand correctly, if you use 50 variants, you would patch together data from all 50 variants? (EDIT: Sorry, I'm thinking about this all wrong; running on little sleep this week!)

To avoid "splice points", you could simply add or subtract value to the entire data segment you are splicing to give a smooth transition, perhaps using the data point immediately previously to where it was spliced as a reference. In other words, if you are splicing EURUSD, and the last point of a data segment is 1.19 and the next starts at 1.41, simply add 0.22 to every bar in the next data segment.

EDIT: Yes of course that's adding the delta value the sleytus has already referred to. I'm glad one idea I had is valid today, even if it's already been espoused.

I have some preliminary results that I would like to share.  In general I'm not prone to hyperbole, but these results are pretty amazing.  Please keep in mind this was a single, quick and dirty experiment -- so, tread carefully.

I created two data sets -- the first one was the most recent 6 months of EURUSD H1 that was duplicated (a total of around 7600 bars).  The second data set was a hybrid of the most recent 6 months of EURUSD H1 plus the most recent 6 months of USDCHF H1.

Using FSBPro I cloned one of my strategies -- I'll call them clone A and clone B.  Clone A was optimized using the duplicated EURUSD data set, and clone B was optimized using the EURUSD-USDCHF hybrid data set.

I then used Multi-Market and tested clone A and clone B against both the duplicated EURUSD and hybrid EURUSD-USDCHF data sets.  I've attached an image that shows the results.

I'll cut to the chase -- the most striking result is that clone B's Multi-Market result is *much* better than clone A's.  Upon reflection this result is not unexpected -- but what it does show is input data used to optimize (curve-fit) a strategy can have a significant affect on a strategy's "robustness".   This is consistent with hannahis' earlier post that there is no such thing as a bad strategy -- I'm exaggerating but that was sort of the take home message.

Another point to observe is the statistics for clone A are better than clone B.  So, even though clone B is more "robust", clone A may be more profitable -- at least in the short term while it is trading in an environment that it is comfortable in.  So, I'm not sure yet how exactly I would take advantage of this observation.  If you accept the premise there is no bad EA -- then perhaps the best (and simplest) approach would be to do a better job of managing which EAs are running since they are sensitive to the current data environment.  A simple management policy might be to temporarily disable a strategy for a couple of weeks if it loses two trades in a row -- and then after a couple of weeks turn it back on to give it another chance.  That is just a very simple example -- of course, more complex policies could be applied.

Tomorrow I'm leaving the country for a couple of weeks.  I should have access to the Internet from time-to-time -- but I may be slow to respond to questions or comments about these results.  When I return I plan to delve into this more deeply.

I also wanted to add, again, this is a testimony to Popov's amazing software.  Without his software there is no way we could experiment like this.

Hannah,

I'm not sure the result I reported necessarily supports "alternative optimization".  Using the example you give of the MA crossover indicator -- yes, there are 3 additional options that each have more settings.  What you would like is for FSB Pro to also take these additional 112 combinations into account when it performs its optimization / curve-fitting / training -- whatever you wish to call it.  Fine -- I have no qualm with that -- but that is not the point of this thread.

I'll make a statement (based on something I learned from you) and it is an exaggeration -- but only to make a point:
"There is no such thing as a bad EA."

Every strategy that Popov's software generates has been optimized, curve-fitted, trained using a particular data set.  Actually, currently I think "trained" is a good description.  It will trade well when it is placed in a data environment similar to the one it trained for -- let's refer to that as its "comfort zone".  When it is forced to process data it wasn't trained for, then it will fail.  An analogy -- you train to speak French and in France you do perfectly fine.  And then you find yourself in Bulgaria -- and it doesn't go so well since you don't speak Bulgarian.  That doesn't mean there is anything wrong with you and you will do fine again once you return to France.  That's what my results would seem to indicate.

The way I see it now -- nothing needs to change in the way strategies are generated.  What I've learned from this simple experiment (and also reading your earlier post and taking into account some of the trading experiences you described) is that we should perhaps pay more attention to the current data environment.  You are way more familiar than I when it comes to identifying trends -- ranging, breakout, etc -- and you have a good idea as to which of your EAs perform better with different trends.  I'm not there yet.  But, I can do something almost as good.  And that is turn my EAs on / off based on some "policy" -- where a policy could be something as simple as "turn the EA off for two weeks if it loses two trades in a row, and then turn it back on".  Since we are now dealing with portfolios of many EAs then it means at any particular time some will be active and some will be in "time out".  Do you know what I mean?

Sorry -- another point occurred to me -- it may not go over so well.

I don't think an alternative optimization is necessary.  The FSB Pro generator creates millions of strategies by fiddling with various indicator settings.  Including the additional options you mention only increases the number of combinations that need to be checked -- it will not necessarily result in more or better EAs.  If there were a scarcity of good EAs then, yes, I would say it might be valuable to include additional combinations of indicator settings to see if more, good EAs could be generated.  But since there is no scarcity, then I don't see the benefit.

I will grant you there might be some good EAs that are missed because the generator doesn't take into account some of the options you mentioned -- but other EAs take their place. 

I am now of the belief that once you have an EA that exhibits good statistics then to get the most out of it we should take into account whether the current data environment matches with the one the EA trained for.

Sleytus keep in mind that automated trading is not about getting more better looking backtest, but it is about making money with unseen data. So all that robustness what we are looking for beats single over optimized better looking system statistic

For me your results is as expected. You get better results with what data you optimized. Real test would be to check on unseen data chunk or at least with different multi market pairs.

1. Optimize on hybrid data -> test it on unseen EURUSD and unseen USDCHF
2. Optimize same strategy separately on EURUSD and USDCHF data -> test it on unseen data with the same pairs

If unseen data 1 beats 2 then you have success

Thanks for the link -- it is interesting reading.  Though I'm not sure exactly how I would incorporate it.

Back to Synthetic Data -- the result I posted, though preliminary, has some very interesting implications.  I started with one of my better strategies and cloned it.  So, I began with two *identical* strategies.  I then optimized/curve-fitted/trained each of those against two different data sets.  The result was two different EAs -- one that was very robust, and one that was less robust but more profitable.

Take this a few steps further.  I'm going to create 100 different data sets -- stitching together data from different pairs, different trends, etc -- whatever I can think of.  And then I'll take my favorite strategy and make 100 *identical* clones.  And then I'll optimize/curve-fit/train each clone against a different data set.  In the end I will have a portfolio of 100 strategies -- each using the same combination of indicators -- but trained against a different data set.

Using carefully designed data sets -- rather than arbitrary DataHorizons -- can be used to generate a portfolio of EAs starting from one (your favorite) EA.  Think about it...

Or -- another approach.  You have a library of 100 data sets.  Then you cycle through your EAs and pick the ones that do well against the most data sets.

No other software -- only Popov's FSBPro -- could allow me to even consider such a wild approach...

I'm not sure we're talking about the same thing -- maybe we are.

I don't know what you mean by "unseen dataset".  Also, I don't understand 26% versus 36% winners.  It's not about the number of winners.   I created a hybrid EURUSD+USDCHF data set and used it to optimize one of my favorite strategies.  I then used that optimized strategy in a MM test -- and the results were very compelling.  Did you look at the images I attached a few posts back?  I don't know how else you could explain it other than the data set used to optimize/ curve fit / train (whatever one chooses to call it) can make a huge difference in the MM test.  If the MM test doesn't mean anything then, okay, it is not worth pursuing.  I haven't been a big fan of the MM test mostly because so few strategies pass with flying colors.  But if one can create more strategies that pass the MM test, I think it is worth pursuing. 

I really haven't thought this through completely -- so I'm not going to push it further.  The last thing I'll say about it is I think the test I described above demonstrated that the data set plays an important role in "training" a strategy -- at least in terms of its ability to survive the MM test.  I'll let others decide for themselves what, if any, the implications are.   And as to how one would exploit this to gain an "edge" -- I'm not sure.  I'm still experimenting...