Hi,

Theory question for you. We all know that we "should" be finding strategies based on the same broker data that we intend to use. It seems intuitive, firstly, but my question is why.

I know for one that this is important to compare results of the backtest in the strategy software with the execution platform, but all you are effectively doing here is checking you have the same settings. If they match and we get the same results, this seems comforting, and when they dont match we flip out and write a message to the forum. But I dont necessarily see any extra value in getting two systems to match. Here's why.

A good trading system isnt just about the backtest due to overfitting, as we know. It's about how robust that system will be in the future and on unknown data.

We test the robustness with Monte Carlo and how good our little boat will fare in unknown seas and the MC looks to scramble things up as much as possible and simulate the unknown future - if the strategy passes the MC, we say its a good strategy. If your strategy is a boat (my analogy) then the MC test is a wave machine designed to upset your vessel in the test pool, creating different conditions to make your strategy capsize.

But "different" data - i.e. non broker data (different currency & same currency and different broker) is also a kind of MC test. Different data is also testing the strategy on a different currency pair and if the strategy does well there, we say its a good one. Surely the best test is to push your strategy onto very different markets and expose it to as wide a range of conditions as possible. Would it be better to have 1) a strategy that worked very well on ONE currency or 2) a strategy that worked averagely on multiple markets/datasets - which is the better one?

My thoughts are that it could be interesting to find strategies that work on "multiple" currencies and data sources and also to trust the numbers rather than get caught up on how the balance chart looks. Im trying to find strategies that have the right numbers on different data, rather than solely having a nice comforting equity graph (although this is useful to prune strategies that seem to have losing streaks). For some of my tests, Im blocking out the equity graphs and then checking these only at the end of the process, so I am trying to rely on the filters more and more.

I guess finding a good strategy is a blend of approaches. Im just interested (as usual) to hear others experiences on whether they always use same broker data or not.