As for the 2024–2026 data, I didn't filter it again because I treated it as unseen data. I simply recalculated the portfolio performance on this data range to check if my filtering method during the generation step was effective.

If you don't mind, could you please share the strategy collection you created, Ridwan?"

Hi Ridwan,

I'm currently using a similar approach. I use free data from histdata.com spanning from 2000 to 2025.

I use the 2000–2024 data to generate strategies, retest them on 2024–2025 data, and then run a final test on Premium data from 2024 to the present.

I strictly use the H1 timeframe because I found higher timeframes are quite sensitive to GMT offsets, while lower timeframes are vulnerable to spread and commission spreads and commissions.

I optimize the strategies and sort the collection based on stagnation.

Most importantly, I use Monte Carlo validation with the "Randomize indicator parameter" feature enabled and the "indicator change probability %" set to 100% (my settings are attached). Since applying this feature, I've seen a significant improvement in my validated strategies.

Hello yonkuro,

Thank you for sharing that rigorous Monte Carlo workflow and setup. I am very interested in your approach, so I immediately tried it out in EAS to test it using a "Blind Holdout" framework (pure testing on unseen future data).

I want to see if your data-slicing method can truly bridge the gap between backtesting and forward testing. Here are the steps I implemented based on your logic:

*using your settings file*

Step 1 (Creation): I created a strategy using historical data from 2000 to 2024 on the H1 timeframe, incorporating your rigorous Monte Carlo analysis.

Step 2 (Validation): I recalculated and filtered the qualifying strategies using the 2024–2025 dataset, applying strict criteria (including sorting based on *Stagnation*).

Step 3 (Establishing a Baseline): Since we are currently in August 2026, I wanted to isolate May 2026 as "pure" future data that had never been seen before. So, instead of testing "from 2024 to the present," I recalculated the qualifying strategies only up to April 2026 to establish a clean final baseline.

Step 4 (Blind Holdout): After obtaining several highly promising strategies that passed all stages up to April, I moved them into the *Blind Holdout* phase. I ran a *forward test* covering May 2026 to evaluate whether the method consistently produced robust strategies.

The Results:
Unfortunately, the results were highly inconsistent:

1. High Failure Rate: Although a few strategies held up, the majority failed significantly, and their performance plummeted.

2. No Clear Differentiator: I have yet to statistically determine what distinguishes the past performance (steps 1, 2, 3) of strategies that passed the *Blind Holdout* stage from those that failed during the *Blind Holdout* step.

3. Unpredictable Lifespan: For the few strategies that successfully passed the *Blind Holdout* stage, there is no measurable way to determine when they will eventually fail or how long their *edge* will remain valid.

Paradigm Shift (Conclusion However, these failures gave me a valuable insight. I believe many of us on this forum are pursuing the wrong goal. We are often obsessed with finding a "great" strategy that can last forever.

What if we shifted our focus? Instead of looking for a permanent strategy, we should seek a repeatable Generation Pattern or Workflow.

If we can find a specific framework that consistently produces strategies capable of surviving for a short yet predictable timeframe (e.g., 1 to 3 months), then the EA itself doesn't need to last forever. The workflow becomes the "Holy Grail" (the ultimate solution), not the EA.

The cycle would look like this:

1. Execute a proven strategy-generation workflow using current market data.

2. Deploy the resulting strategy on a *Live* account, fully aware that it has a "lifespan" (an expiration date).

3. Capture profits while the current market regime remains aligned with the strategy.

Once the strategy's performance declines or hits a certain threshold, stop using it. Then, re-run the exact same generation workflow using the latest data to capture the next market regime.

In this approach, the strategies themselves are entirely disposable, yet the method used to generate them is permanent and continuously adapts to current market conditions.

Has anyone here successfully implemented a "Continuous Regeneration" cycle like this? If so, what metrics do you use to determine when a strategy is officially considered "expired" and needs to be pulled from the *Live* account?

I'm currently using a similar approach. I use free data from histdata.com spanning from 2000 to 2025.

I use the 2000–2024 data to generate strategies, retest them on 2024–2025 data, and then run a final test on Premium data from 2024 to the present.

I strictly use the H1 timeframe because I found higher timeframes are quite sensitive to GMT offsets, while lower timeframes are vulnerable to spread and commission spreads and commissions.

I optimize the strategies and sort the collection based on stagnation.

Most importantly, I use Monte Carlo validation with the "Randomize indicator parameter" feature enabled and the "indicator change probability %" set to 100% (my settings are attached). Since applying this feature, I've seen a significant improvement in my validated strategies.

Nice initiative!
Few points: "the blind holdout" is still a form of turning OOS data sample into IS data sample, therefore it alone might not be enough for "confident assumption" for positive results as it is just another form of playing with IS/OOS.
The second point is WFA. In its current state, when one uses it in the reactor, it is rather useless. The idea of WFA is to use "unseen" or OOS data for walk-forward validation, but if it is used in the reactor it goes like this: strategies are calculated on the whole dataset and in WFA part that same dataset is cut into pieces for walk-forward validation. In most if not all cases the strats pass it effortlessly as they have been already generated/tested to pass the whole dataset!

But Ridwan, what do you think of my idea that in order to find "the golden settings" we should look at the data itself? Essentially the data "characteristics" or a change in them make or break a strategy or a whole collection. If we use the full generation process to find a working set of settings/workflow procedure, then by doing so we introduce so many variables, which change and can have a variety of effects on the work process that it still will boil down to throwing everything at the board hoping something sticks. And even then we can't be sure why something sticked. Do you get my idea?

Thank you for the fantastic insights! This is exactly the kind of high-level discussion I was hoping to trigger. You hit the nail on the head on all three points, and I completely agree.

To address your points:

1. The Blind Holdout / Meta-Overfitting:
You caught me there, and you are mathematically correct. By iterating the generator settings based on the Holdout results, I am essentially engaging in a higher level of Data Snooping bias (turning the OOS into IS). My initial thought was to limit the iterations to find a structural "baseline" rather than tweaking it 100 times, but your point stands. The bias is still there.

2. The WFA Flaw in the Reactor:
Spot on. The look-ahead bias in EAS's Reactor WFA is a known structural issue. Generating on the full dataset first and then validating via WFA is basically giving the engine the answer key before the test. It defeats the entire purpose of Walk-Forward validation.

3. Looking at the Data Characteristics:
This is the most critical part of your reply, and I love this idea. You are talking about Market Regimes (e.g., shifts in volatility, trending vs. mean-reverting phases). You are absolutely right: throwing thousands of variables at the wall hoping something sticks is bad science, even if we use strict Acceptance Criteria. The "Golden Settings" shouldn't be static numbers, but rather a dynamic approach based on what the data is doing.

This leads me to a follow-up question for you, as I'm very interested in your workflow:

How do you practically identify or quantify these "data characteristics" before running EAS?

Or do you have a specific way to filter data characteristics within EAS itself?

I'd love to hear how you integrate this data-driven approach into your daily generation workflow!

But Ridwan, what do you think of my idea that in order to find "the golden settings" we should look at the data itself? Essentially the data "characteristics" or a change in them make or break a strategy or a whole collection. If we use the full generation process to find a working set of settings/workflow procedure, then by doing so we introduce so many variables, which change and can have a variety of effects on the work process that it still will boil down to throwing everything at the board hoping something sticks. And even then we can't be sure why something sticked. Do you get my idea?

I’ve noticed a recurring pattern, both from personal experience and various discussions in this forum. We all know how easy it is to generate a strategy with a visually perfect equity curve in Expert Advisor Studio (EAS). We go out of our way to use Out of Sample (OOS), Walk Forward Analysis (WFA), and various strict filters within the software.

However, the main issue often remains the same: Not all strategies that pass these filters and make it into the Collection manage to survive (print a positive profit) when actually tested on a Demo or Live account.

Many people suggest testing the strategy on a Demo account for 3 months (as an example timeframe) before moving to a real account. But let's be more pragmatic: Why should we waste 3 months of real-world time purely just to wait for Demo results?

Wouldn't it be much more efficient if we "extend" the historical data and simulate the forward test instantly using a method I call "Blind Holdout" ?

The Critical Difference: EAS Built-in OOS vs. Blind Holdout

Many of us fail because we assume the OOS in EAS is the final test. In reality, when we enable the OOS feature (e.g., 30%), EAS still uses that OOS data as a selection criterion. If a strategy fails in the OOS zone, the engine discards it. This means the strategies in the Collection have indirectly been "optimized" to coincidentally fit that specific OOS data (Data Snooping Bias).

To overcome this, we need Blind Holdout Data—a segment of data that we completely hide and entirely exclude from EAS during the generation process.

The 3-Stage Master Plan:

Here is the logic on how we can use a Blind Holdout not just to find strategies, but to validate the EAS Settings themselves. Let’s assume today is July 1, 2026, and we have data from January 2025 to June 2026.

Stage 1 (Inside EAS - Finding the Logic): We restrict the data loaded into EAS only up to March 31, 2026.
- In-Sample: Jan 1, 2025 – Dec 31, 2025 (The engine builds the logic).
- EAS OOS: Jan 1, 2026 – Mar 31, 2026 (The engine filters passing strategies).

(Note: In Stage 1, you can fully utilize the Reactor with all its analysis tools like Multi Market, WFA, etc.).

Stage 2 (Blind Holdout - Validating the Settings): April 2026 – June 2026 (3 months).
- We manually re-backtest the Stage 1 strategies over this last 3-month range (data the engine has never seen) by adjusting the Data Horizon and using the Recalculate button in EAS. If the strategy breaks down, we change our EAS Generator Settings and repeat Stage 1. We keep doing this until we find the EAS Settings that consistently produce strategies capable of surviving this Stage 2 Blind Holdout.

Stage 3 (The Ultimate Goal - Live Deployment):
- Once we find those "Golden Settings", we no longer need the Blind Holdout. We simply run EAS using these proven settings on the forward-shifted dataset (April 2025 up to yesterday, June 30, 2026).
- Assumption: Because the settings themselves have been rigorously validated through the Stage 1 & 2 holdout process, we can confidently assume that any strategy generated today will print positive profits for the next 3 months on a Live Account.

The Goal of This Discussion:

Based on this premise, I want to open a discussion:
"We need to figure out what kind of EAS Settings ensure that when the engine generates a strategy, that strategy consistently prints a positive profit when exposed to a Blind Holdout period."

My question focuses purely on EAS Settings. For those of you who have successfully formulated a robust engine setup—where your generated strategies rarely break down during forward testing—I truly hope you’d be willing to share them. Specifically regarding:

- What Acceptance Criteria do you use?
- What Timeframe do you use?
- How many Data Bars or what Date Range do you use?
- What preset indicators do you allow the engine to use initially?
- What specific "Strategy Properties" or "Generator Settings" do you use?
(Or you can directly upload your EAS Settings .json file).

Hopefully, by sharing these settings, we can formulate the "Meta Settings" for EAS that are truly reliable and massively save our research time.