Re: Using Synthetic DataHorizons For Generating Strategies
1. Training your EA with Synthetic Data
First of all, I think it's a great idea to use synthetic data to "train" our EA for Specific Market conditions.
It's like Artificial neural networks (ANNs), whereby you "condition" your EA to be good (curve fit) in certain market situations.
If I were to go about with this project, this is what I'll do...
a) Cut and Paste as many ranging/breakout market conditions as I can to create a long series of similar market conditions so that my EA can Identify some Key trading concepts from these similar data.
b) Next train my EA to Differentiate non similar trading conditions (improving on it's filtering process to avoid false signals). In order to do this I'll start adding opposing/mix signals. For example, If I want to develop Breakout Strategies, my 1st synthetic data will have mostly breakout trends cut and pasted together. And my 2nd training data set will have Ranging data period add in between 2 breakout trends.
In order to make sure my Breakout strategies are good, I will use "prolong" ranging periods in between 2 breakout trends. The aim is to filter out poor Breakout Strategies that can't handle ranging periods and the end product will be good Breakout Strategies that can handle prolong ranging in between breakouts.
c) After the Strategies have pass these 2 training (a +b), I'll subject these EA with "unseen" data/OOS data of "a normal real" historical period or put it to demo testing.
2. Subjecting EA with "unseen" data (isn't that also mean OOS data?)
I agree that putting EA to demo testing is rather long process if you look at from 1 EA's point of view. However, imagine if I've hundreds of EA and I've to keep monitoring and setting up one by one EA for backtesting, I think the demo path seems a much easier task. I install hundreds of EA and I leave it there for 3mths, in between these 3mths, I develop and create another new batches. So it's not time consuming because I don't have to do anything during these 3mths for that particular batch of EA in demo testing. Whereas, if I were to backtest hundreds of EA, I'll be spending lots of hours walking to and fro my computer checking and changing another EA for backtesting, saving the results, putting them together for comparison etc.
So in terms of testing hundreds EA, I would rather just put them in demo and leave them be than to sit in front of the computer, backtesting 1 EA at a time.
3. Backtesting results vs Demo testing (which is more reliable?)
Personally, I trust Demo testing results more than any backtesting tools. Even when I got an excellent backtesting results, it still leave me with questions whether it can still be profitable in demo/live. With demo testing, there is no room for doubt, I know I can replicate the success from Demo to live and hence, I trust and rely on my demo statistic to assist me in my portfolio selection.
Lastly, as I mentioned before, even if you have a backtest results, you still won't know what types of EA you have (especially if you depend on FSB to search for you) and you won't know how the EA behave in different market situations. Thus, these are extra vital information you can get from demo testing. For example, when I know that my Ea will lose about $3000 during ranging, do I panic and stop the trading. No, cos since I know how my EA behave, I know that my EA is capable of earning back what it loss 2x or 3x more. A lot of people panic when their EA lose money and they shut it down prematurely cos they haven't observe their EA enough to have confidence in their EA, all they understand of their EA is just those backtesting statistic.
It is very important to understand how your EA behave so that you can have confidence whether to keep your EA running during those times when it is losing money. That's part of the psychology of trading with EA, do you ultimately trust your EA? And such trust often can only be built over time when you observe your EA's performance in a demo account, not just by staring at the statistical numbers.
4. Does MM really a good filter for robust EA?
I agree with Steve's analogy comparing different markets as different language. However, for EA, despite different markets, their language is still the same (calculation using those indicators formula to determine a rule is true/false.
So it's not so much about speaking in different language but I think it's more apt to use Dance as the analogy. Each market has their own unique rhythm/pattern/flow/tempo. Each has different ranging gaps/channels etc and hence to expect a Jazz dancer to excel well in a tango dance, it would be "off sync", not that the Jazz dancer can't tango, given his natural talent/EA's fundamental theory, he may pass off with a good dance but not an excellent one.
Thus I personally don't agree using MM at all to "train" my EA. You ended with "Jack of all trades, master of none"
Theoretically speaking, the trading concepts are applicable to all markets. I use my same trading concept in all different markets, it is sound and hence it is robust. However to work out the fine details of creating an EA for a particular market, one still need to use that particular market's data set to "understand" what are the reason and rhyme for that EA to click with the market. Different markets have different ranging duration, different Breakout heights, different government interventions/fundamentals issues...these are times, when technical analysis take on a passenger seat while the fundamental are driving the markets trends (as we have often observe for the past couple months).
During such fundamental market interventions, a lot of EA will fail especially when it uses laggy indicators/parameters because it would have created many "false" signals (which otherwise are real but because of government interventions, the technical or the normal peak and trough didn't complete it's full cycle, changed "prematurely").
Yes, I believe it is good to train our EA with synthetic EA but we don't have to subject it to MM and like what Footon said, ultimately, whatever method it is, the one that helps you yield the best and consistent results is the one most ideal (be it backtesting or demo). The key is "are you able to repeat your success"?, if you can, you have found an excellent workflow to replicate your success.