I'd like something about the losing streaks, because when the strategy goes live and it loses 5 times in a row, it is a big question if that is within expected results of the backtest, or if it indicates the strategy no longer works.
Moreso than losing streaks, I think it is important to identify a non-subjective way to know when a live-trading bot has become "out of spec" per the FSB testing.
According to some other reading I have done (actually per your suggestion elsewhere) -- I now consider strategies that "flat line" for a period of OOS to be acceptable whereas ones that decline too much during an OOS test are not-acceptable to me.
I am currently tracking new-found bots using 50,000 bars of M1 data with 30% OOS enabled, but rolled back 2 weeks into the past via Data Horizon. This allows me to evaluate strategies from a 2-weeks-ago point of view. I use the 30% OOS as a pre-screen only. If the strategy looks GREAT, including the 30% OOS portion, I then move to the next phase.
In this next phase, I am still working with 2-weeks-old data. I run Optimizer for the indicators and also find the optimal Permanent SL/TP/BE.
Lastly, I take this developed strategy and use Data Horizon to see how it would handle OOS data in the next week and also next-next week. These two tests are the most important factors in my decision of whether or not to go live. During this phase I record all stats and also screenshot the equity curve. In the stats, I'm looking for these specific and non-subjective things:
max balance DD -- want to see this the same
max equity DD -- want to see this the same
max equity DD% -- want to see this the same
reward ratio -- bonus if it has improved
efficiency -- bonus if it has improved
After the non-subjective checks, if all still looks good, I evaluate (subjectively) the change in the equity curve plot. I'm looking for a flat or preferably rising equity line. Declining line is ok ONLY if its decline was muted such that it did not increase those hotspots listed from the stats.
This method gives me about 3.5 weeks of OOS analysis. I'm not sure if it's "enough" for trading M1 or not.
Anyway... back to the original concept here --- what would be great is if FST could automatically trigger an alert when certain metrics fall out of spec. These could be self-programmed for example. In some ways, I already do this by enabling the minimum account balance feature of the expert. I key the bots to trade at a level where the tested-max-equity-DD% equals my comfort level (say 10%), using the formula I've posted elsewhere. Then, I set the expert to stop trading if it exceeds this level (say 12% for cushion). In that way, I know if a bot stops trading then it has exceeded a critical level (max equity DD%) and has fallen out-of-spec.
I also analyze every live-trading bot on weekends to determine whether or not they are still in-spec or have fallen out-of-spec. I rotate the live-traders weekly.