1 Topic by algotrader21 (edited by algotrader21 2026-04-02 18:05:49)

  • Registered: 2026-03-22
  • Posts: 27

Topic: The Extra OOS Trick in EA Studio With Real Examples

This is something I don’t see many people talk about, but for me this was one of the biggest shifts in my whole process. And you can literally see it in the screenshots.


What I did here is actually very simple, but the impact is huge.


In the #2 screenshot, you see the strategy built on data from 2009 until 2020.

This is close to how I was actually doing it back then in early 2024.

Back then I was using slightly different starting ranges as well, sometimes more like 2007–2019 and then extending forward, but the idea itself was exactly the same.

That is the important part.



The principle never changed:

build first, then push the same strategy into extra unseen data.

This is the normal build phase. This is where the strategy is generated and optimized. At that stage, the strategy already looks solid. The equity curve is good, the net profit is there, the trade count is decent, and the SQN is already 3.45.


For many people, that would already be enough to say the strategy looks good.

But this is where I started thinking differently.


Because for me, it is not enough that a strategy looks good only on the period where it was built and optimized.

That is exactly where many people stop too early.

They build, they check the normal out of sample inside EA Studio, and then they move on.

But I wanted to push the strategy further.

I wanted to see what really happens when you expose it to data it has never seen.



So what I started doing back then was this:

after generating and optimizing the strategy, I do not touch the strategy itself anymore.

I do not change the rules
I do not reoptimize
I do not tweak anything

I only change one thing:

the data end.


So the strategy stays exactly the same, but now I extend the end date forward.

And this is exactly what you see in the #1 screenshot.

The same strategy is now extended from 2009–2020 to 2009–2024.

And that is the key point.


That extra period was never optimized.

So what you are looking at there is pure unseen data.

Real out of sample.

No fitting
no tweaking
no hidden optimization

Just the strategy being pushed forward into new market conditions.

And this is exactly where weak strategies usually break.

That is why this became such an important step in my workflow.


Because if a strategy looks good until 2020 but then falls apart when I extend it to 2024, then I already know enough.

I don’t care how nice it looked before.

It is not stable.


But if I extend it and it keeps going, stays stable, and even improves, then now we are talking about something completely different.

And that is exactly what you see here.

The strategy holds.

The equity curve keeps moving.

The structure stays intact.

That is what you want to see.



For me, this is one of the clearest ways to separate:

strategies that look good
from strategies that are actually stronger

Because now you are not looking at in-sample anymore.

You are forcing the strategy through a completely new period it has never seen.

That is real testing.


That is why I always say:

strategies break or survive on out of sample.

And this is just one more layer of that idea.

Simple concept.

Huge difference.



One important thing I want to add here about the build process itself.

When I build strategies, I use full data optimization with 40 steps.

And there is a reason for that.


A lot of people are afraid of heavy optimization because of overfitting.

But that is exactly where Monte Carlo comes in.

Monte Carlo is not there to replace optimization.

It is there to test it.


So I actually push the optimization harder on purpose.

I give the strategy room to explore the data.

And then I use Monte Carlo to check if that structure is still stable under randomness.

This strategy you see here was built with 40 optimization steps.

And then you can clearly see what happens when it goes through heavy Monte Carlo testing.

That is the balance.


Strong optimization → then strong validation.

And on top of that, the extra out of sample period is not optimized at all.

So that is pure OOS.


Also during the build itself, I already use 50% out of sample in EA Studio.

So from the start there is already a separation between in-sample and out-of-sample data.

But the extra OOS trick is what really pushes it further.

That is where things get exposed.


After that, I move to the next step.


Monte Carlo.

Normally, like I explained before, I use:

50 runs
then 100 runs as confirmation


But here I pushed it much further.

I ran 1500 simulations.

And this is where things become very clear.

Because now you are not looking at one equity curve anymore.


You are stress testing the same strategy across a massive number of variations.

Randomized history
randomized spread
randomized slippage
different starting points

Everything gets pushed.

And if a strategy is weak, this is where it shows immediately.



Now look at the confidence table.

Even under heavy randomization, the structure holds.

The performance stays consistent
the SQN remains solid
the degradation is controlled

Even at higher confidence levels, the strategy is still standing.

That is not normal for weak strategies.


And if you look at the editor, you can see the magic number.

This is EA 165.


So again, this is not a random example.

This is a real strategy.

And this strategy has been running live for around a year.


That is also why I was confident enough to push it to 1500 Monte Carlo simulations.

Because by that point, I already had real live data behind it.


But the important part is this:

back then, when I first built it, I was not doing 1500 runs.

I was doing 50 → then 100 runs as confirmation.

Exactly like I explained before.

The 1500 runs you see here are just an extra layer on top of something that was already proven step by step.



So what you are really seeing here is this:

a strategy built on historical data
then extended into real unseen data (extra OOS)
then stress tested through massive Monte Carlo

And it still holds.

That is what I call confirmation.

Not one good backtest.

Not one lucky curve.

But multiple layers, all pointing in the same direction.


That is the difference.

That is also why for me this is not optional.

This is a filter.

If a strategy fails here → it is out.

If it survives → it earns the right to move forward.


That is the whole idea.

Data
out of sample
extra out of sample
Monte Carlo

Stack those layers.

And you will see very quickly what is real and what is not.

Simple idea.

Huge difference.


For the live results of EA 165, you can check here:
https://forexsb.com/forum/topic/10057/building-robust-eas-on-xauusd-a-structured-workflow-that-works/



One last thing to make this even more clear.

After everything you just saw, I extended the same strategy even further.

From 2009 all the way to 2026.


Again:

no changes
no reoptimization
same logic

Just more unseen data.

And this is what matters.


Because at this point, you are not just looking at a strategy that survived a small extension.

You are looking at a strategy that continues to hold its structure over an even longer period.

The equity curve keeps moving
the behavior stays consistent
the structure remains intact

That is not something you fake.


And if you compare this with the previous topic where I shared the live performance, you can start connecting the dots.

This is exactly the point.

Not one test.
Not one phase.


But consistency across:

build
extra out of sample
extended data
Monte Carlo
demo
and live performance

That is where real confidence comes from.

Everything has to align.

And when it does, this is the result.

Post's attachments

EA 165 after build proces extr oos.png 306.98 kb, file has never been downloaded. 

EA 165 build proces and optimization .png 252.48 kb, file has never been downloaded. 

EA 165 FULL DATA 2026.png 270.37 kb, file has never been downloaded. 

EA 165 MC 1500 runs .png 577.03 kb, file has never been downloaded. 

EA165 MC 1500 Confidence.png 102.54 kb, file has never been downloaded. 

You don't have the permssions to download the attachments of this post.

algotrader21's Website

2 Reply by algotrader21 (edited by algotrader21 2026-04-02 18:01:24)

  • Registered: 2026-03-22
  • Posts: 27

Re: The Extra OOS Trick in EA Studio With Real Examples

Because this EA has already been running live for about a year, I decided to push the validation much further.

So this is not a random test.


This is a proven strategy being pushed to its limits.


I increased the stress deliberately:

history data heavily randomized (80% changed bars)
ATR variation increased to 40%
slippage increased up to 40 points
and 1500 Monte Carlo simulations

Not because I needed to “prove” it.

But because I already knew this strategy could handle it.


Why?

Because this is not a fresh build.

This is an EA that has already:

passed the build process
survived extra out-of-sample
and proven itself in live conditions

So at this stage, the goal is different.

Not discovery.

Confirmation under pressure.

Results



Even under heavy stress:

the structure holds
performance remains consistent
degradation stays controlled
SQN remains solid across confidence levels

This is exactly what you want to see when you push a proven system.



Key point

No changes were made to the strategy.

No reoptimization.
No adjustments.

Just pure stress testing.

At this level, you are no longer looking at a backtest.

You are looking at behavior under extreme conditions.

And it still holds.

Post's attachments

MC CHART .png 516.25 kb, file has never been downloaded. 

MC CONFIDENCE.png 105.15 kb, file has never been downloaded. 

settings MC (1).png 100.9 kb, file has never been downloaded. 

You don't have the permssions to download the attachments of this post.

algotrader21's Website

  • From: Germany
  • Registered: 2010-12-18
  • Posts: 173

Re: The Extra OOS Trick in EA Studio With Real Examples

The “extra OOS” idea is, from a practical standpoint, one of the most solid approaches I’ve seen for reducing curve fitting.

Many traders rely on the standard OOS in EA Studio and assume it’s sufficient, but in reality the generator still “sees” the full data context. By isolating a completely unseen period and testing without any re-optimization, this method gets much closer to real live trading conditions.

What I find particularly valuable about this approach:

it enforces discipline (no tweaking after seeing results)
it filters out strategies that are “too good to be true”
it highlights true robustness rather than just performance

is probably the closest we can get to real-market conditions in a retail environment.

Great contribution,  this is the kind of method that actually improves how strategies are filtered, not just how they are generated.

  • Registered: 2026-03-22
  • Posts: 27

Re: The Extra OOS Trick in EA Studio With Real Examples

mentosan wrote:

The “extra OOS” idea is, from a practical standpoint, one of the most solid approaches I’ve seen for reducing curve fitting.

Many traders rely on the standard OOS in EA Studio and assume it’s sufficient, but in reality the generator still “sees” the full data context. By isolating a completely unseen period and testing without any re-optimization, this method gets much closer to real live trading conditions.

What I find particularly valuable about this approach:

it enforces discipline (no tweaking after seeing results)
it filters out strategies that are “too good to be true”
it highlights true robustness rather than just performance

is probably the closest we can get to real-market conditions in a retail environment.

Great contribution,  this is the kind of method that actually improves how strategies are filtered, not just how they are generated.


Thanks Mentosan, appreciate it.

Good to see this perspective being recognized, not many people approach it this way.

Once the strategy is built and optimized, extending the end date without touching anything basically turns it into pure, untouched out-of-sample data.

That’s usually the point where the difference shows between strategies that only looked good during the build phase and those that actually hold up when pushed into completely unseen conditions.

A lot of strategies simply don’t survive that extension, even if they passed initial filtering and Monte Carlo.

At the same time, you’ll never filter everything out completely, but working with longer datasets and adding that extra OOS layer makes a big difference in what remains.

Like you said, it’s probably one of the closest things we have to real market conditions in a retail environment, especially when that oos extension covers multiple market regimes instead of just a short period.

That’s where the real filtering happens.

algotrader21's Website

  • Registered: 2020-07-02
  • Posts: 66

Re: The Extra OOS Trick in EA Studio With Real Examples

Do you test multiple timeframes, multiple symbols, or multiple broker data? I ask because I'm using express generator and mm.js (multimarket) does not support the same flags that gen.js does (the generator) for OOS and data-end, etc.

  • Registered: 2026-03-22
  • Posts: 27

Re: The Extra OOS Trick in EA Studio With Real Examples

aaronpriest wrote:

Do you test multiple timeframes, multiple symbols, or multiple broker data? I ask because I'm using express generator and mm.js (multimarket) does not support the same flags that gen.js does (the generator) for OOS and data-end, etc.

Hi Aaron,

Thanks, good question.

I don’t test across multiple timeframes, multiple symbols, or different brokers, and I’ll explain why.

For me, a strategy does not need to work across everything to be considered good. What matters most is that it performs well on the specific market, timeframe, and data it was built for. I build strategies for a specific symbol and timeframe, and I validate them within that context.


Regarding timeframes, I don’t build on lower timeframes at all. There is too much noise there and, in my experience, a much lower chance of getting something truly robust. I mainly work from M30 and higher.



For symbols, I’m not a big believer in cross-pair validation as a main measure of robustness. Some strategies can work on multiple pairs, yes, but in my view that is rare. For me, the main question is not whether a robot can run on many pairs, but whether it is truly robust on the market and timeframe it was built for. And once a robot reaches demo or live, that is really what I focus on: the actual metrics and whether it is making money on the pair and timeframe it was designed for.


Another reason I don’t put much weight on multi-market or multi-timeframe validation during the build phase is that, in my view, we still cannot clearly tell what we are looking at.. Even if a robot shows good results across multiple markets or timeframes inside EA Studio, that still does not tell me clearly whether I’m looking at something truly robust or just something that is broadly overfit.

And if we then start putting that robot on demo across different pairs and timeframes, it can easily become a waste of time and resources, because the main question is still unanswered: is the robot actually robust or not?

For me, it makes much more sense to first go deeper on one market, one timeframe, and one environment, and only later, if the robot has already proven itself through forward or live results over a meaningful period, then maybe check whether it can be expanded to other pairs or timeframes.

The difference is big. Then we are not expanding with a robot that EA Studio simply approved. We are expanding with a robot that has already proven itself in real conditions.


Same idea for brokers.

I’m not building or selling EAs for general distribution, so I don’t test across different brokers. I focus on the broker I actually use for building, demo, and live trading.

In my view, if you are not selling robots and you already use one solid broker, then multi-broker testing does not add much value. I would rather use that time and energy to run more strategies on demo, expand within the same broker, and push more of them through real forward testing.

That gives me much more useful information than checking whether the same robot behaves slightly differently across several brokers.

If I ever want to expand beyond the maximum number of accounts with the same broker, I can always connect another broker through copy trading. I already do that with one account. Sometimes I see a difference of a few seconds in execution, not always, but that is normal. It can even happen between different live accounts at the same broker. That is simply part of trading reality and execution. But for me, that does not say much about the underlying robustness of the strategy.
The main point is that the robot needs to work properly on the broker and environment it was actually built for.

So for me, testing across multiple brokers is more about distribution and compatibility than about robustness. It can make sense if you sell robots and want to know whether they behave similarly everywhere, but for my own workflow it does not add much.

algotrader21's Website

7 Reply by aaronpriest (edited by aaronpriest 2026-04-16 04:38:31)

  • Registered: 2020-07-02
  • Posts: 66

Re: The Extra OOS Trick in EA Studio With Real Examples

Thanks for your detailed response! It mostly mirrors my own experience. I've had many good strategies fail multi timeframe and multi symbol that still ran well for what it was designed for. I stick with H1 and higher, and occasionally M30. Multi broker gets a bit more nuanced, particularly if you can't get good data from a broker (missing candles, etc.), or enough history. This is less of a problem with MT5 if you have a good broker though.

If you don't mind me picking your brain some more... what is your opinion and experience on date range? I used to generate from 2010 (or as far back as my broker could go) with about 60/40 IS/OOS and heavy Monte Carlo, and run strategies that passed until they weren't profitable for three months in a row or so or broke a previous pattern. It was quite tedious to keep track of them. Lately I've been experimenting with generating on the last year of data (excluding the last 3 months for OOS) with optimizer and Monte Carlo, validating the last 3 months to see how it would have traded (also with Monte Carlo, but no optimizing), and then just throwing it straight on a demo account. I generate all week, then validate and swap out new EAs on weekends, so I'm always running new strategies for a week on recent data. I've got it automated to a point where I don't spend much time on a weekend setting up the next week. I don't have enough data yet to say if it's more profitable or not, but curious to hear other people's experience on this before I waste much more of my time testing the theory... Is it better to analyze many years of various markets that won't ever repeat the same anyway, or analyze and tune for more recent data? Either way, tomorrow won't continue the same as yesterday. LOL

I think I'm going back to a larger date range though because it's hard to find a strategy with enough trades on only 3 months of OOS data to be truly valid.

  • Registered: 2024-02-23
  • Posts: 119

Re: The Extra OOS Trick in EA Studio With Real Examples

Is this EA 165 the account you are referring to as outcome  ?




algotrader21 wrote:

Because this EA has already been running live for about a year, I decided to push the validation much further.

So this is not a random test.


This is a proven strategy being pushed to its limits.


I increased the stress deliberately:

history data heavily randomized (80% changed bars)
ATR variation increased to 40%
slippage increased up to 40 points
and 1500 Monte Carlo simulations

Not because I needed to “prove” it.

But because I already knew this strategy could handle it.


Why?

Because this is not a fresh build.

This is an EA that has already:

passed the build process
survived extra out-of-sample
and proven itself in live conditions

So at this stage, the goal is different.

Not discovery.

Confirmation under pressure.

Results



Even under heavy stress:

the structure holds
performance remains consistent
degradation stays controlled
SQN remains solid across confidence levels

This is exactly what you want to see when you push a proven system.



Key point

No changes were made to the strategy.

No reoptimization.
No adjustments.

Just pure stress testing.

At this level, you are no longer looking at a backtest.

You are looking at behavior under extreme conditions.

And it still holds.

Post's attachments

expert advisor 165 overview main account.png 80 kb, file has never been downloaded. 

You don't have the permssions to download the attachments of this post.

9 Reply by algotrader21 (edited by algotrader21 2026-04-16 12:41:42)

  • Registered: 2026-03-22
  • Posts: 27

Re: The Extra OOS Trick in EA Studio With Real Examples

aaronpriest wrote:

Thanks for your detailed response! It mostly mirrors my own experience. I've had many good strategies fail multi timeframe and multi symbol that still ran well for what it was designed for. I stick with H1 and higher, and occasionally M30. Multi broker gets a bit more nuanced, particularly if you can't get good data from a broker (missing candles, etc.), or enough history. This is less of a problem with MT5 if you have a good broker though.

If you don't mind me picking your brain some more... what is your opinion and experience on date range? I used to generate from 2010 (or as far back as my broker could go) with about 60/40 IS/OOS and heavy Monte Carlo, and run strategies that passed until they weren't profitable for three months in a row or so or broke a previous pattern. It was quite tedious to keep track of them. Lately I've been experimenting with generating on the last year of data (excluding the last 3 months for OOS) with optimizer and Monte Carlo, validating the last 3 months to see how it would have traded (also with Monte Carlo, but no optimizing), and then just throwing it straight on a demo account. I generate all week, then validate and swap out new EAs on weekends, so I'm always running new strategies for a week on recent data. I've got it automated to a point where I don't spend much time on a weekend setting up the next week. I don't have enough data yet to say if it's more profitable or not, but curious to hear other people's experience on this before I waste much more of my time testing the theory... Is it better to analyze many years of various markets that won't ever repeat the same anyway, or analyze and tune for more recent data? Either way, tomorrow won't continue the same as yesterday. LOL

I think I'm going back to a larger date range though because it's hard to find a strategy with enough trades on only 3 months of OOS data to be truly valid.


Thanks, and also respect for sharing your way of working so openly.

Yeah, exactly. I’m honestly glad to see someone who thinks like this, not just by theory, but by actually testing, by trial and error. That’s what this is really about in the end.


EA Studio offers a lot of functions, and in principle that makes sense because it belongs to algo building. More tools and more testing can give more answers in a certain way. But in my eyes, not every function adds real value when it comes to robustness testing.

Because like you also experienced yourself, robots can fail multi-pair and multi-timeframe tests, and still run well live on the exact pair and timeframe they were built for.

And the opposite also happens. Some strategies can pass those multi-pair or multi-timeframe checks inside EA Studio, and still fall apart later on demo or eventually live. Not all of them, of course, but you see it happen often enough.

That is also why, for me, EA Studio, Monte Carlo, and all those tools are still reference points, not final proof of robustness. As long as a robot has not proven itself through a long enough demo period and then a long enough live period, I do not really see it as a robust system yet.

For me, real robustness only starts to become clear after 6 months live, 1 year live, and ideally even longer. The more time and data a system survives, the more we actually know. I always come back to that same principle.

To me, that already says a lot.

It tells us that multi-pair and multi-timeframe are extra answers, maybe interesting later, but not something I see as core proof of robustness in the build phase.

If a system has already proven itself live, then yes, putting it on another pair or timeframe can make sense and can add diversification. But that is something else. Then you are expanding a proven system, not validating an unproven one.


About data, I’m on the long-data side, with a lot of OOS.

For me, data and OOS are key, plus Monte Carlo. Honestly, that is already enough.

That is also why I build from 2009 to 2022. Inside that build period I already use 50% OOS. The data I use is the premium data from Popov, and I’ve been using that from the beginning. I adjust that data inside EA Studio to match my broker specs, and because of that I get a correct match between EA Studio, MT4 backtests, demo, and live.

Good robots show the same structure everywhere.


So my process is like this:

I build from 2009 to 2022
with 50% OOS
and inside Reactor I optimize the full build period with 40 steps, also with 50% OOS

Then when that is done, I extend the end date forward to 2026.

That creates 4 extra years of pure, raw, unseen, non-optimized OOS data.

A strategy that survives that gets Monte Carlo tested.

After that I run an MT4 tick-data backtest from 2003 to 2026, mainly to see what the robot does on the years before my build period started. Since my build period starts in 2009, I want to see how it behaves on that extra OOS before 2009 as well, so roughly 2003 to 2009.


Can I be honest?

I can clearly see you are dedicated, and I respect that. I also see that you are not biased for long data or short data. You are clearly learning from your experiments and noticing patterns, and for that, real respect.

At the same time, I do think there is a downside to the shorter data approach you are currently testing.

And I’ll explain what I mean.


Right now you are building robots on one year of data, where three months of that is OOS inside the building process.

So basically, you are giving the robot around nine months to train on, plus three months as an OOS check.

That means the robot has only seen a relatively small portion of the market.

Even with OOS included, it is still trained on a limited amount of data.


For me personally, that is a bit too little to build strong confidence.

It is the same logic everywhere.


If someone wants to build an AI model, do they train it on as little data as possible, or on as much relevant data as possible?


Or if you choose someone to manage your capital, do you pick the one with 20 years of experience, or someone with only a short track record?

For me, it translates directly to algo building.

The more historical data a strategy has seen, the more market conditions it has been exposed to.

The more OOS inside the build phase, the better.

And on top of that, I like to add a wide extra OOS layer after the build.


If that is not possible inside Express generator, then building on 2009–2022 and testing in MT4 on 2009–2026 can give a similar effect, just a bit slower.

Of course, I don’t know exactly how Express Generator works, since I don’t use it.


And yes, what you are doing can definitely work. Shorter data can work.

But one thing I’ve noticed with that approach is that it tends to push the process into a much faster and more rotational workflow.

You need to keep building new strategies regularly, because you need fresh ones for the next cycle.

And because of that constant rotation, everything becomes more reactive.

I also think one of the risks there is that some potentially good strategies might be rotated out too early, simply because they did not have enough time yet to show their behavior.

Strategies often need time to play out.


That is one of the biggest differences for me.

With longer-data robust strategies, the process is slower, yes. It takes longer in building, longer in demo, and longer in live.

But it is much less rushed.

I don’t need to switch every week. I don’t need to build every day.


At this point I have around 12 robots with good metrics, plus maybe another 5 that are still profitable but where the metrics are not really top level. So those are not my best systems, but they still do their job.

And honestly, that’s also part of my view on this. We do not need a portfolio of 50 systems. Even in manual trading, one person with a good strategy and the right psychology can make serious money. For me it is the same with robots.

One strong, robust system that works for a long time can already make you serious money. So if you have a stack of around 10 robust robots that have already been running well live for a year, in my opinion you really do not need much more than that.

And yes, there are enough builders who can make shorter-data constant rotation work.

I’m not denying that.

It can work.

But for me personally, the goal is different.

I prefer building systems that are robust and can last over longer periods of time.

That is also where I see the main difference:

Long data is slower, but generally leads to more stable structures over time.

Shorter data is faster, but relies much more on constant rotation and replacement.

So yes, it moves faster.

You get robots faster.
You can move them faster from demo to live.

But you also tend to remove them faster again, because they depend more on staying in rotation.

That is why I personally prefer the long-data approach.

algotrader21's Website

  • Registered: 2026-03-22
  • Posts: 27

Re: The Extra OOS Trick in EA Studio With Real Examples

Vincenzo wrote:

Is this EA 165 the account you are referring to as outcome  ?




algotrader21 wrote:

Because this EA has already been running live for about a year, I decided to push the validation much further.

So this is not a random test.


This is a proven strategy being pushed to its limits.


I increased the stress deliberately:

history data heavily randomized (80% changed bars)
ATR variation increased to 40%
slippage increased up to 40 points
and 1500 Monte Carlo simulations

Not because I needed to “prove” it.

But because I already knew this strategy could handle it.


Why?

Because this is not a fresh build.

This is an EA that has already:

passed the build process
survived extra out-of-sample
and proven itself in live conditions

So at this stage, the goal is different.

Not discovery.

Confirmation under pressure.

Results



Even under heavy stress:

the structure holds
performance remains consistent
degradation stays controlled
SQN remains solid across confidence levels

This is exactly what you want to see when you push a proven system.



Key point

No changes were made to the strategy.

No reoptimization.
No adjustments.

Just pure stress testing.

At this level, you are no longer looking at a backtest.

You are looking at behavior under extreme conditions.

And it still holds.


Yes, that is EA 165 from my main live account.

The symbol and magic number are visible in the screenshot as well.

algotrader21's Website

  • Registered: 2020-07-02
  • Posts: 66

Re: The Extra OOS Trick in EA Studio With Real Examples

algotrader21 wrote:

And yes, what you are doing can definitely work. Shorter data can work.

But one thing I’ve noticed with that approach is that it tends to push the process into a much faster and more rotational workflow.

You need to keep building new strategies regularly, because you need fresh ones for the next cycle.

And because of that constant rotation, everything becomes more reactive.

I also think one of the risks there is that some potentially good strategies might be rotated out too early, simply because they did not have enough time yet to show their behavior.

Strategies often need time to play out.

Yup, you nailed it. They are very short lived and have to get replaced often, that is a huge downside. And quite likely some good strategies are replaced prematurely, but most short term ones are bad strategies in the long run. I've been running a mix of both short and long term strategies on demo accounts, and keeping track of their performance via magic numbers, swapping out short term ones frequently and only swapping out long term ones when they clearly break their pattern--that part takes more effort to track, but it's easy to see if one has been unprofitable for three months in a row for example to flag it for testing.

  • Registered: 2026-03-22
  • Posts: 27

Re: The Extra OOS Trick in EA Studio With Real Examples

aaronpriest wrote:

Yup, you nailed it. They are very short lived and have to get replaced often, that is a huge downside. And quite likely some good strategies are replaced prematurely, but most short term ones are bad strategies in the long run. I've been running a mix of both short and long term strategies on demo accounts, and keeping track of their performance via magic numbers, swapping out short term ones frequently and only swapping out long term ones when they clearly break their pattern--that part takes more effort to track, but it's easy to see if one has been unprofitable for three months in a row for example to flag it for testing.





Yeah, honestly I think what you’re doing right now is probably one of the best ways to really understand what works and what doesn’t.

That hybrid approach combining short-term strategies with longer-data builds gives you a lot of insight through actual experience, not just theory.

For me personally, I lean more towards the long-data side. I build on larger datasets with 50% OOS, allow full optimization during the build phase (even quite heavy, like 40 steps), and then rely on Monte Carlo to validate whether the structure holds or breaks under stress. That’s exactly why I’m comfortable pushing optimization  because MC is there to test if it’s real or not.

But at the same time, I completely agree with you that experimenting like you’re doing is key. In the end, everyone has to go through that process and figure out what fits their own mindset and workflow.

Short-term strategies can definitely work, but like you said, they tend to require constant rotation and replacement. It becomes a much more active process. The longer-data approach is slower and honestly a bit more “boring”, but for me it fits better because it’s more focused on long-term behavior rather than constant switching.

For example, at this point I have a core group of around 12 strong systems that have been running live well for over a year, plus a few others that are still profitable but not top-tier. Because of that, my building frequency has dropped a lot. I mostly focus now on managing, validating, and slowly expanding what already works, rather than constantly generating new strategies.

On the execution side, I also try to keep things structured. Ideally, I want to see around 6 months on demo before moving a robot to live. But if a robot already reaches around 60 trades within 4 months and the metrics still look strong, that can also be enough for me. In the end, it always depends on the overall behavior and structure.

At the same time, I do agree with your point that after enough experience, you start recognizing patterns faster. You can often see earlier when something is off. But even then, I still try to balance that with giving strategies enough room, because sometimes it really depends on the market phase they start in.

That’s also something I’ve noticed a lot certain strategies just need the right conditions. For example, breakout systems can struggle in ranging periods at the start, even if they perform well over time.

So yeah, overall I think both approaches can work. It really comes down to what kind of process you want to run and what fits your mindset best.

Curious to see how your results evolve over time with that mix.

algotrader21's Website

  • Registered: 2024-02-23
  • Posts: 119

Re: The Extra OOS Trick in EA Studio With Real Examples

And are you satisfied with this outcome ?
Does it cover your expectations?


algotrader21 wrote:
Vincenzo wrote:

Is this EA 165 the account you are referring to as outcome  ?




algotrader21 wrote:

Because this EA has already been running live for about a year, I decided to push the validation much further.

So this is not a random test.


This is a proven strategy being pushed to its limits.


I increased the stress deliberately:

history data heavily randomized (80% changed bars)
ATR variation increased to 40%
slippage increased up to 40 points
and 1500 Monte Carlo simulations

Not because I needed to “prove” it.

But because I already knew this strategy could handle it.


Why?

Because this is not a fresh build.

This is an EA that has already:

passed the build process
survived extra out-of-sample
and proven itself in live conditions

So at this stage, the goal is different.

Not discovery.

Confirmation under pressure.

Results



Even under heavy stress:

the structure holds
performance remains consistent
degradation stays controlled
SQN remains solid across confidence levels

This is exactly what you want to see when you push a proven system.



Key point

No changes were made to the strategy.

No reoptimization.
No adjustments.

Just pure stress testing.

At this level, you are no longer looking at a backtest.

You are looking at behavior under extreme conditions.

And it still holds.


Yes, that is EA 165 from my main live account.

The symbol and magic number are visible in the screenshot as well.

  • Registered: 2026-03-22
  • Posts: 27

Re: The Extra OOS Trick in EA Studio With Real Examples

Vincenzo wrote:

And are you satisfied with this outcome ?
Does it cover your expectations?


algotrader21 wrote:
Vincenzo wrote:

Is this EA 165 the account you are referring to as outcome  ?


Yes, that is EA 165 from my main live account.

The symbol and magic number are visible in the screenshot as well.


Yes, I’m satisfied with it.

Not because of the numbers alone, but because of the consistency across all layers:

build phase 
extra out-of-sample extension 
Monte Carlo under heavy stress 
and most importantly, real live performance over time 

For me, that combination is what defines whether a system meets expectations or not.

At that point it’s no longer about a single result, but about consistent behavior across different validation stages and real conditions.

Also, considering the metrics and the fact that this EA has been running across multiple live accounts with the same behavior, I think the outcome speaks for itself.

algotrader21's Website

  • Registered: 2024-02-23
  • Posts: 119

Re: The Extra OOS Trick in EA Studio With Real Examples

Just to make sure I understood you correctly.

When you say you’re satisfied with the outcome, it’s not mainly because of the absolute metrics, but because the strategy shows consistent behavior across all validation stages, from build, to extra out-of-sample, to Monte Carlo, and then also in live.

So in your view, that alignment across layers is more important than the standalone numbers at any given stage.

Did I get that right?



algotrader21 wrote:
Vincenzo wrote:

And are you satisfied with this outcome ?
Does it cover your expectations?


algotrader21 wrote:

Yes, that is EA 165 from my main live account.

The symbol and magic number are visible in the screenshot as well.


Yes, I’m satisfied with it.

Not because of the numbers alone, but because of the consistency across all layers:

build phase 
extra out-of-sample extension 
Monte Carlo under heavy stress 
and most importantly, real live performance over time 

For me, that combination is what defines whether a system meets expectations or not.

At that point it’s no longer about a single result, but about consistent behavior across different validation stages and real conditions.

Also, considering the metrics and the fact that this EA has been running across multiple live accounts with the same behavior, I think the outcome speaks for itself.

  • Registered: 2026-03-22
  • Posts: 27

Re: The Extra OOS Trick in EA Studio With Real Examples

Vincenzo wrote:

Just to make sure I understood you correctly.

When you say you’re satisfied with the outcome, it’s not mainly because of the absolute metrics, but because the strategy shows consistent behavior across all validation stages, from build, to extra out-of-sample, to Monte Carlo, and then also in live.

So in your view, that alignment across layers is more important than the standalone numbers at any given stage.

Did I get that right?



algotrader21 wrote:
Vincenzo wrote:

And are you satisfied with this outcome ?
Does it cover your expectations?


Yes, I’m satisfied with it.

Not because of the numbers alone, but because of the consistency across all layers:

build phase 
extra out-of-sample extension 
Monte Carlo under heavy stress 
and most importantly, real live performance over time 

For me, that combination is what defines whether a system meets expectations or not.

At that point it’s no longer about a single result, but about consistent behavior across different validation stages and real conditions.

Also, considering the metrics and the fact that this EA has been running across multiple live accounts with the same behavior, I think the outcome speaks for itself.


I never said standalone metrics are not important. If metrics were not important, I would not be posting screenshots of them in the first place.

For me, everything has to be good as a whole. Metrics matter a lot, because they already tell me whether a system is strong, mediocre, or just weak. But when I talk about robust systems, I’m looking at more than just numbers at one stage.

What matters to me is that the same quality keeps showing up across all layers: build phase, extra out-of-sample, Monte Carlo, demo performance, and then live performance over time.

And when I say it all has to be good as a whole, or that I focus a lot on structure, then metrics are obviously important as well. Without metrics, I cannot even judge performance properly, and I also cannot see whether the equity curve, drawdown, and overall behavior are actually strong or not.

So for me, it is not metrics versus alignment. It is the combination. Good metrics make a system interesting, but consistency across all validation stages is what tells me whether it is actually robust or not.

algotrader21's Website

  • Registered: 2024-02-23
  • Posts: 119

Re: The Extra OOS Trick in EA Studio With Real Examples

Got it, that makes sense.

Out of curiosity, how do you actually define “good metrics” in your workflow?

For example, do you have specific thresholds or ranges you expect to hold from build (backtest) to live (like SQN, drawdown, etc.), or is it more based on overall behavior and experience?


algotrader21 wrote:
Vincenzo wrote:

Just to make sure I understood you correctly.

When you say you’re satisfied with the outcome, it’s not mainly because of the absolute metrics, but because the strategy shows consistent behavior across all validation stages, from build, to extra out-of-sample, to Monte Carlo, and then also in live.

So in your view, that alignment across layers is more important than the standalone numbers at any given stage.

Did I get that right?



algotrader21 wrote:

Yes, I’m satisfied with it.

Not because of the numbers alone, but because of the consistency across all layers:

build phase 
extra out-of-sample extension 
Monte Carlo under heavy stress 
and most importantly, real live performance over time 

For me, that combination is what defines whether a system meets expectations or not.

At that point it’s no longer about a single result, but about consistent behavior across different validation stages and real conditions.

Also, considering the metrics and the fact that this EA has been running across multiple live accounts with the same behavior, I think the outcome speaks for itself.


I never said standalone metrics are not important. If metrics were not important, I would not be posting screenshots of them in the first place.

For me, everything has to be good as a whole. Metrics matter a lot, because they already tell me whether a system is strong, mediocre, or just weak. But when I talk about robust systems, I’m looking at more than just numbers at one stage.

What matters to me is that the same quality keeps showing up across all layers: build phase, extra out-of-sample, Monte Carlo, demo performance, and then live performance over time.

And when I say it all has to be good as a whole, or that I focus a lot on structure, then metrics are obviously important as well. Without metrics, I cannot even judge performance properly, and I also cannot see whether the equity curve, drawdown, and overall behavior are actually strong or not.

So for me, it is not metrics versus alignment. It is the combination. Good metrics make a system interesting, but consistency across all validation stages is what tells me whether it is actually robust or not.