1 Topic by Vincenzo (edited by Vincenzo 2025-10-11 21:11:16)

  • Registered: 2024-02-23
  • Posts: 72

Topic: Follow-up 2: Phase 3 & Live Validation

Following the demo validation phase (Phase 2), this stage moves the same SQN-filtered generation process into real trading conditions to observe how survival probability and performance behave under execution constraints.
All EAs were generated and validated before live deployment and not re-optimized afterwards.

(Post #1 link: https://forexsb.com/forum/post/82865/#p82865 )
(Post #2 link: https://forexsb.com/forum/post/82906/#p82906 )

Test set up & Parameter Description

- Start Date 4 April 2025
- Active EAs 24 (3rd generation – SQN-filtered)
- Brokers / Accounts: 2 brokers (BlackBull Markets, RoboForex) / 4 accounts
- Symbols 21 symbols, ≥ 2 timeframes per symbol
- Trade Sample 477 trades (≈ 190 days / 6 months)

Performance Metrics:
- Portfolio PF = 1.35 (aggregate)
- Trade Win Rate = 72.3 % (aggregate)
- Total Return = +19.9 %
- Max DD = –7.1 %
- EA Status 13 EAs in profit (54 %), 11 in loss (46 %)
- Data Source Real execution with live spreads, slippage, and commissions

Interpretation

This first live dataset shows consistent survival behavior compared with the Phase 2 demo validation, with 54 % of EAs currently in profit (vs. 55 % in Phase 2).
Preliminary data suggest a balanced profit distribution across symbols and timeframes, though this will require further confirmation as the live sample expands.

All results are based on real execution across two brokers (BlackBull Markets and RoboForex) and include all EAs originally deployed — no re-optimization or exclusion has been applied. Underperforming EAs remain counted in the statistics to avoid survivorship bias.

Next Phase — Workflow Maintenance (Live Phase 4) see the attachement.

The next phase will introduce dynamic rotation across the full live portfolio:

    •    Monthly rollover — apply the classification rules to update each EA’s status (New, Stable, Watchlist, Critical).
    •    Rotation logic — after ≈ 50 trades per EA, suspend the weaker ones (Max Losses ≥ 5 or PF < 1.1) and replace them with new SQN-filtered candidates from the same pool.
    •    Cross-broker extension — extend validation to additional brokers for feed diversity and execution resilience.

Conclusion

The SQN-based filtering maintains a comparable survival probability between demo and live conditions.
Preliminary performance remains aligned with prior phases, and the dataset now provides the foundation for the upcoming maintenance-rotation workflow.

This next stage will focus on the systematic exclusion and replacement of underperforming EAs to gradually improve aggregate portfolio efficiency — with the operational target of stabilizing the EA-level Win Rate above 70 % over time, subject to ongoing monitoring and live execution validation.

I’m sharing this work — the result of several months of testing, data collection, and refinement — because I believe that open, data-driven discussion helps all of us learn faster.

(Only) Constructive feedback and alternative viewpoints are very welcome.

Vincenzo

Post's attachments

IMG_3572.jpeg 335.55 kb, 13 downloads since 2025-10-11 

You don't have the permssions to download the attachments of this post.
  • Blaiserboy
  • Blaiserboy
  • Junior Part-Time Aspiring Space Cadet
  • Offline
  • From: Toronto
  • Registered: 2010-09-06
  • Posts: 2,867

Re: Follow-up 2: Phase 3 & Live Validation

I certainly appreciate your sharing.

One question I have...... what is your guideline for your testing?

Once I know a bit more about what you are doing I may have some comment.

Blaiserboy's Website

3 Reply by Vincenzo (edited by Vincenzo 2025-10-12 07:30:26)

  • Registered: 2024-02-23
  • Posts: 72

Re: Follow-up 2: Phase 3 & Live Validation

Thanks, Blaiserboy — appreciate your interest.

A quick background:
I started trading about 14 years ago — first discretionary (volume profile), then about five years ago I switched fully to algorithmic trading, mainly with third-party grid and martingale EAs.
They still run and make money, but I’ve modified all sets of them to manage risk better after some blown up account.

Today, I still buy & test a few external bots (very special things like news scalpers), but my focus is on building my own EAs with EA Studio.

After almost two years using it, I’ve learned that easy generation doesn’t mean reliable profitability — it requires a systematic, repeatable process.

Here’s the simplified workflow I use now:

    1.    Generation – Data 2016–2024, 30 % OOS, filters as acceptance criteria (PF > 1.1, Win % > 60, SQN > 2).

    2.    Validation – WFA on 2025 data, selecting top performers (older collections re-filtered).

    3.    Incubation – Demo for at least 3 months or 50 trades across brokers and timeframes.

    4.    Live – No re-optimization, monitored by PF, Win %, Recovery, and Max Losses.

    5.    Monthly maintenance – Rotate EAs based on rule-based performance thresholds.

Goal: reach 60–70 % success rate in live portfolios and maintain 4–5 efficient, rotating sets over time.

Happy to compare notes — especially on data spans, filters, or live validation setups.

Vincenzo

Blaiserboy wrote:

I certainly appreciate your sharing.

One question I have...... what is your guideline for your testing?

Once I know a bit more about what you are doing I may have some comment.

  • Blaiserboy
  • Blaiserboy
  • Junior Part-Time Aspiring Space Cadet
  • Offline
  • From: Toronto
  • Registered: 2010-09-06
  • Posts: 2,867

Re: Follow-up 2: Phase 3 & Live Validation

I think that you do not use very many indicators in the generation stage, for example I use Stagnation, Backtest Quality, Profit factor, Sqn, Win/Loss, Consec Loss, Max Equity Drawdown, Min R - Squared.

This doesn't give me long living ea's, so I am interested in your research to determine what I can improve.

I don't do the incubation phases, I will start that, and I have been using a much shorter time period, 5 years.

Some grid ea's give me great results, but I feel obligated to watch them closely.

Blaiserboy's Website

5 Reply by Vincenzo (edited by Vincenzo 2025-10-12 19:47:54)

  • Registered: 2024-02-23
  • Posts: 72

Re: Follow-up 2: Phase 3 & Live Validation

Okay, let’s go deeper.
I will snapshot my process, step by step and publishing it here so we can discuss small bites each time. I can also share the 91 combinations I used to filter the 233 strategy of the phase 1.

Will come back soon…


Blaiserboy wrote:

I think that you do not use very many indicators in the generation stage, for example I use Stagnation, Backtest Quality, Profit factor, Sqn, Win/Loss, Consec Loss, Max Equity Drawdown, Min R - Squared.

This doesn't give me long living ea's, so I am interested in your research to determine what I can improve.

I don't do the incubation phases, I will start that, and I have been using a much shorter time period, 5 years.

Some grid ea's give me great results, but I feel obligated to watch them closely.

  • From: United States
  • Registered: 2012-02-06
  • Posts: 125

Re: Follow-up 2: Phase 3 & Live Validation

Vincenzo this looks great and is somewhat similar to the routine I'm following. In my case, I basically have envisioned this routine for years but after seeing one of Petko's videos with nearly the same process, I figure I'm on the right track.

With EA Studio I am basically doing this:

  • Use a documented Reactor routine (let me know if interested in the specifics)

  • Daily/periodically process the yield from Generator that also passed my strict validations

  • Load all strategies into my $100k demo group testing account; all are set to 0.1 lots at this stage

  • Based on FX Blue statistics, I'm taking the top 5 from each pair and promoting them to my (live) account

  • At some (yet to be determined) interval, swap out poor performers and replace from the incubator account

I think it's interesting that you mentioned ~50 trades in the rotation phase. I hadn't considered that but might rather do the rotations like this instead of waiting some arbitrary time period like 30 days. It makes sense that you could consistently compare performance by seeing how a given bot performs under trading conditions - and I might like the trade count better.

I am running all my Reactors on data from Baazex-Demo, which is the broker that iFunds uses. I plan to start with an iFunds account hopefully. I'm paranoid to use other data sources because they seem to vary greatly -- or at least strategies perform quite differently between datafeeds. Also I should note that Baazex only gives me 100,000 bars, which works out to about 4 years on M15. On one hand, I like it because the past 4 years seems sufficient and relevant for current markets. On the other hand, it would also be great if I had 10+ years to work with. But if my theory is correct - that recency of markets should carry some weighting - then it should work out fine with a consistent rotation logic similar to what you suggested.

So far the gems in my promotion account are doing well.

  • Registered: 2024-02-23
  • Posts: 72

Re: Follow-up 2: Phase 3 & Live Validation

Hi,

About the settings and metrics I use (not indicators) — I keep it simple with just three acceptance criteria.
The idea is to cut out the weak strategies and keep a clean list of potentially good ones, without over-filtering and losing too many in the process.

I decided to go with a set of three, including SQN, because it consistently appeared in the better results and, from what I’ve seen, it’s one of the few metrics that professionals actually use for statistical system testing.
These three — SQN, Profit Factor, and Trade Win % — come from Experiment #1, where the setups achieved over 50 % EA win rate.

I’m adding here the snapshot from the 91 combinations I tested — all the ones that reached a win rate above 50 %.

Hope that makes clear why I use just these three.

Regards
Vincenzo


Blaiserboy wrote:

I think that you do not use very many indicators in the generation stage, for example I use Stagnation, Backtest Quality, Profit factor, Sqn, Win/Loss, Consec Loss, Max Equity Drawdown, Min R - Squared.

This doesn't give me long living ea's, so I am interested in your research to determine what I can improve.

I don't do the incubation phases, I will start that, and I have been using a much shorter time period, 5 years.

Some grid ea's give me great results, but I feel obligated to watch them closely.

Post's attachments

IMG_3574.jpeg 346.77 kb, 10 downloads since 2025-10-14 

You don't have the permssions to download the attachments of this post.
  • Registered: 2024-02-23
  • Posts: 72

Re: Follow-up 2: Phase 3 & Live Validation

Hi Dusktrader,

That’s a really solid routine — very close to what I’ve been refining as well.

On the data horizon, I’ve settled on using around 8–9 years (2016–2024).
For me, that’s the right balance — long enough to cover multiple volatility regimes (COVID, inflation waves, wars and geopolitical crises, low-volatility periods, rate-hike cycles, and the rise of online trading through new technologies), yet recent enough to reflect today’s market structure.
It gives a realistic snapshot of how each logic behaves across diverse but still relevant market conditions. Going back 10+ years often introduces noise from structurally different markets, while using only 3–4 years gives too narrow a view.

Regarding the 50-trade rule, I use it as a rotation threshold because it’s a replicable and market-independent metric.
It removes distortions from slow or hyperactive markets and aligns with the concept behind the SQN metric, where statistical significance depends on the number of trades rather than elapsed time.
This makes portfolio rotation decisions more consistent and comparable across symbols and timeframes.

Your use of Baazex data from iFunds makes perfect sense — keeping data aligned with your execution environment ensures consistency.
In my case, I test across multiple brokers (BlackBull and Roboforex) to stress-test feed stability before going live. It’s more resource-intensive, but it helps avoid overfitting to a single feed.

Would be great to compare outcomes at some point — seeing how different data lengths and brokers influence stability could be really valuable.

Vincenzo


dusktrader wrote:

Vincenzo this looks great and is somewhat similar to the routine I'm following. In my case, I basically have envisioned this routine for years but after seeing one of Petko's videos with nearly the same process, I figure I'm on the right track.

With EA Studio I am basically doing this:

  • Use a documented Reactor routine (let me know if interested in the specifics)

  • Daily/periodically process the yield from Generator that also passed my strict validations

  • Load all strategies into my $100k demo group testing account; all are set to 0.1 lots at this stage

  • Based on FX Blue statistics, I'm taking the top 5 from each pair and promoting them to my (live) account

  • At some (yet to be determined) interval, swap out poor performers and replace from the incubator account

I think it's interesting that you mentioned ~50 trades in the rotation phase. I hadn't considered that but might rather do the rotations like this instead of waiting some arbitrary time period like 30 days. It makes sense that you could consistently compare performance by seeing how a given bot performs under trading conditions - and I might like the trade count better.

I am running all my Reactors on data from Baazex-Demo, which is the broker that iFunds uses. I plan to start with an iFunds account hopefully. I'm paranoid to use other data sources because they seem to vary greatly -- or at least strategies perform quite differently between datafeeds. Also I should note that Baazex only gives me 100,000 bars, which works out to about 4 years on M15. On one hand, I like it because the past 4 years seems sufficient and relevant for current markets. On the other hand, it would also be great if I had 10+ years to work with. But if my theory is correct - that recency of markets should carry some weighting - then it should work out fine with a consistent rotation logic similar to what you suggested.

So far the gems in my promotion account are doing well.

Post's attachments

IMG_3575.jpeg 1.32 mb, 10 downloads since 2025-10-14 

You don't have the permssions to download the attachments of this post.