forex software

Create and Test Forex Strategies

forex software

Skip to forum content

Forex Software

Create and Test Forex Strategies

You are not logged in. Please login or register.


Forex Software → Developers Forum → Feature Proposal: Histogram of Trades

Pages 1

You must login or register to post a reply

RSS topic feed

Posts: 22

Topic: Feature Proposal: Histogram of Trades

I've found it very useful in researching strategies to include a histogram of the trades. I've got a prototype working on my local copy of FSB, and wanted to discuss if users think it is useful, and how it should be added to FSB to submit.

Here's a screenshot:

http://s15.postimage.org/ylrkk3gs7/histogram_By_Trade_Result.jpg

This is a random strategy, I just took the first couple of indicators, and bar closing to show how they plot. The equity curve shows the trades over time, and the histogram gives an idea of the distribution of trades.

The mouse over sends the detailed info to the status bar at the bottom. In the screenshot, I have my mouse on the trades that resulted in 0, so on the bottom you can see "Result: 0  Count: 169  Total: 0".
"Result" is trades that resulted in that amount; "Count" is how many, my strategy had 169 trades that resulted in $0; "Total" is the total amount from all trades at the result (or Result * Count).

In the upper left, it has a toggle button. I wanted to put it on the right, but had a bug with redrawing and Invalidate() I could not figure out, so left it on the left. It toggles to cumulative amount at that trade result, like in this screenshot:

http://s16.postimage.org/b3hwz0gxt/histogram_By_Amount.jpg

Reds are losers and greens are winners. This view gives a better example of "Total" -- I have mouse over trades that resulted in profit of 10, and there 15 of them, so the Total is 150. The goal is to get a visual sense of the area under the curve of the losers vs winners, and a better idea of how the outliers affect the end result.

Questions on feature:

1) Where should the panel go? I have it over the Equity Chart on the Main work area, because I like to see it as I adjust parameters. Would it be better to have as a separate pop up panel, like the Comparator / Scanner / Analyzer features? Or to have it toggle out of sight like the Journal by Positions and Journal by Bars?

2) Are there better labels than "Result", "Count", "Total"?

3) How to export histogram data? For now, I have it as Right Click -- pops up Message Box, then I can copy and paste to somewhere. That was the simplest way to do it for the prototype.

4) Is there other useful info to add to the feature?

Bugs:
1) I modeled off the Small_Balance_Chart, but could not get the right axis correct. After minimizing / restoring, the right axis scale gets larger and larger numbers.

2) The toggle button is on left. If I resize FSB, it redraws the button several times across the title bar without removing the previous ones. I couldn't figure out how C# does that.  tongue

3) Rounding -- the trade results are rounded to the nearest integer. This means for a permanent stop set to 30, the stopped trades will plot as 32 and 33, unfortunately removing the visual sense of how many got stopped. It makes it look shorter since they are separated over 2-3 bars:

http://s18.postimage.org/i728l9o7p/stops_limits_Split.jpg

Same happens on the right for the Permanent Take Profit at 90 -- it has 2 bars for the limit trades at 86 and 87.

I am also ok if this is Fact of Life, probably not a big deal or bug.

__

Please feel free to comment with your ideas, this is in the prototype stage so suggestions or changes can be put in fairly easily, and it will help to get everyone to agree on the how the final feature should look and work.

I think this feature is useful because you can see the distribution, it gives a visual sense of how random a strategy is, and where it is centered. It also gives an idea if your strategy has fat tails or not, and really helps show that sometimes just a few positive outliers account for all the profit of a strategy, the rest mostly balance out.

2

Re: Feature Proposal: Histogram of Trades

This feature looks really useful, great suggestion.  My comments below:

Questions

1. Possibly the panel could be overlayed in the area where the Indicator Chart is displayed, with the user having the option to toggle between Indicator and Histogram displays.  I (personally) would probably not be interested in the looking at the Indicator Chart and the Histogram Chart at the same time, but would definately be interested in seeing the Histogram Chart and Equity Chart together.

2. The naming make sense to me.

3. I am working on a way to export the results of a generator run to a file that would include many strategies, along performance details for each strategy.  I'd be happy to take a look at this for you if you like, and may have some existing code in my prototype that can be used for this.

Bugs

3. You may be able to work around this by toggling through more than 2 display modes.  I guess that you currently have a 'Total' and a 'Long/Short' mode, what if you added a 'Stop/Target' chart to this toggle (so there is now 3 modes) that showed only those trades that hit the SL or TP - assuming that this data is available in the backend of course.  This may allow the SL and TP trades to be removed from the other displays, so that the data is more meaningful in the 'Total' and 'Long/Short' displays.

Re: Feature Proposal: Histogram of Trades

Hello,
I have a minute to comment, so first, this is great addon to FSB.
Ab is right to display the chart on the place of the indicator chart. We can also utilize the market statistics with additional backtesting info.

One potential problem is the data for the charts and info on the main screen. Each display (indicator chart, journal, balance chart...) has to use buffer data instead directly data from Data.Strategy and Backtester. This is because the strategy and backtester data are dynamically changed when Generator or Optimizer work.
I'll develop such buffer with all stats data. It will be updated once the strategy is calculated.
I did something similar for the indicator chart that crashes when the display is resizing during generating. The same is the situation for the journal.

I'll make a class StatsBuffer or something like that and functions: SetUpWorkspace and RepaintWorkSpace.  We'll put links for updating charts, stats windows and journals there.

WE have to also find better toggle buttons for the charts and info windows. I think about something like sliding overlay buttons that appear on the upper right corner of the corresponding panel when mouse goes over it. I want to rid off this ugly toggle and close buttons that we are using at the moment.

I'll work these two three days on that buffer and refactoring the main screen fields.
Hope to find net to update github.

....
The carnival starts these days in Brazil...

Re: Feature Proposal: Histogram of Trades

Thank you ab and Mr Popov for your feedback, this helps a lot.

Placement:  Ok, for placement, it sounds like: over Small Indicator Chart, with a button to toggle between the 2. I will see if I can do this.

Exporting: currently it has a function to make a long string of csv values, basically "<result>,<count>,<total>" as numbers -- very easy to copy and paste into Excel. I'll try to make it with "Copy to Clipboard" and "Write to File" options. I may be able to use FSB's existing code to write out file, as in all of the items in the Export menu.

Stops/Limits over 2 Histogram Bars: on second thought after playing with this a bit, I've found most strategies have very high stop and limit counts (ex, 50 or 100), then the strategy closed trades in between have very small counts (ex, 1, 2, or 3); it actually is better for the high stop and limits to have shorter bars, or else due to the scale they overwhelm the strategy close trades, can't see anything.

Popov wrote:

....
The carnival starts these days in Brazil...

Please enjoy Carnaval, no need to rush for my feature. I always wanted to go, but closest I got was a local parade and youtube ....

Re: Feature Proposal: Histogram of Trades

Here is a screenshot of it on the left. But I think you meant to toggle between the Indicator and Histogram charts, not necessarily stacked on top of each other as I have it. Is there an existing control in FSB I can use for that?

http://s8.postimage.org/tw1hx0tf5/hist_On_Left.jpg

Re: Feature Proposal: Histogram of Trades

Would it be helpful to make all strings as Language.T("string"), or would it be better to skip the Language.T?

Re: Feature Proposal: Histogram of Trades

Hello Krog,
You have to make both charts with equal size and location and to toggle their visibility. See how it's done in the generator's chart.
You also have to use Language.T("MyString") in orther to make the text translatable. After that you have to add the new strings in the Bulgarian.xml file in the Resources folder of the project. Just put both strings in english. Latter, when you start the program FSB will tell you that the new string is not in the language file. This is because FSB compares the integrated Bulgarian.xml with every language file at loading. You don't need to add the new phrases  manually in all the lang files. Just use genlangfiles and repairlang commands from the console. FSB will inject the new phrases in the lang files. After that the phrases can be translated by using the Edit Translation tool.

Re: Feature Proposal: Histogram of Trades

Popov wrote:

Hello Krog,
You have to make both charts with equal size and location and to toggle their visibility. See how it's done in the generator's chart.
You also have to use Language.T("MyString") in orther to make the text translatable. After that you have to add the new strings in the Bulgarian.xml file in the Resources folder of the project. Just put both strings in english. Latter, when you start the program FSB will tell you that the new string is not in the language file. This is because FSB compares the integrated Bulgarian.xml with every language file at loading. You don't need to add the new phrases  manually in all the lang files. Just use genlangfiles and repairlang commands from the console. FSB will inject the new phrases in the lang files. After that the phrases can be translated by using the Edit Translation tool.

Ok, I'll try to do that before submitting to github.

Re: Feature Proposal: Histogram of Trades

Exporting Data:
I got 2 buttons, one will copy the CSV data to the clipboard, so user can paste into Excel sheet very quickly; the other exports and writes a csv file to the local machine, and I added a function to Exporter.cs to do it.

Re: Feature Proposal: Histogram of Trades

I hope someone can help me with this one, I'm not sure how to do this in C#:

how to make a public callable function for the class's private function with out parameters:

private void getHistogramDataInts (out int[] results, out int[] indexes, out int[] counts, out int[] cumulatives)

I've tried:

public void GetHistogramDataInts { get { getHistogramDataInts( ); } }
public void GetHistogramDataInts { get { getHistogramDataInts(out int[] results, out int[] indexes, out int[] counts, out int[] cumulatives ); } }
public int[][][][] GetHistogramDataInts { get { getHistogramDataInts(out int[] results, out int[] indexes, out int[] counts, out int[] cumulatives ); } }
public void GetHistogramDataInts { get { getHistogramDataInts(new int[], new int[], new int[], new int[] ); } }

and several other combinations like that, but I've had no luck finding one that works ...

11

Re: Feature Proposal: Histogram of Trades

This may be easier by just creating a class containing HistogramDataInt, then a List<HistogramDataInt> to pass data back, eg:

public class HistogramDataInt
{
  private int count;
  public int Count
  {
    get { return count; }
    set { count = value; }
  }

  // ... Other stuff follows
}

private List<HistogramDataInt> getHistogramDataInts()
{
  List<HistogramDataInt> thelist = new List<HistogramDataInt>();
  // Populate the List<>
  return thelist;
}

public List<HistogramDataInt> mylist = getHistogramDataInts();

Re: Feature Proposal: Histogram of Trades

I've posted the addition to my github repository, and you can get the .exe to try it out at:
https://github.com/krogKrog/Forex-Strat … uilder.exe
I think if you click Get Raw, it will download the exe for you.

Here is a screenshot, I was able to add to the left to toggle between the Indicator chart:

http://s15.postimage.org/56h4b83qv/hist_Overlaid.jpg

- ToolStrip to hold buttons for toggling between Indicator and Histogram
- Splitter under ToolStrip to resize charts
- 3 buttons on Histogram chart for
  * toggle between Count and Cumulative views
  * copy histogram data to Clipboard
  * Export histogram data to CSV file

This addition is minimally invasive, adding only Small Histogram Chart.cs. Modified files:
- Controls Market.cs
- Actions.cs
- Exporter.cs

To get the histogram data from somewhere else in the code, the calculating method is static and publicly available as:

Small_Histogram_Chart.GetHistogramDataInts(out int[] results, out int[] indexes, out int[] counts, out int[] cumulatives)

I think this will work, I didn't test it, let me know if it doesn't work, it will be easy to remove it.

These details, I wasn't able to do them:
- putting Toggle/Clipbard/Export buttons on the right
- generating the language files (but all strings are in Language.T('myString') form)
- making the Indicator and Histogram charts the same size all the time so when toggling between them the chart space does not jump around

I've also had a few Visual Studio crashes in debugging when using the Clipboard feature. I haven't seen it with the exe, so hopefully it's just something about Vis Stud.

Let me if any comments or questions, then if it's passes review, I'll send on the push request in github.

Re: Feature Proposal: Histogram of Trades

Krog,
Thank you for uploading the sources to github. I played a little bit with the chart and found out unusual behaviour.

The Histogram Chart shows only loosing trades for the included in the distribution Demo MACD Reversal strategy.

http://s14.postimage.org/6i11t3y7h/histogram_chart_bug.jpg

Another issue is that the histogram chart doesn't take the same size as the indicator chart. (Do you need help with this?)

Also the chart takes raw values from the Backtester class. This can lead to an issue when the chart is forced to recalculate during the work of Generator or Optimizer. Th problem comes from the fact that Generator uses the Backtester class for its calculations and puts temporary values there. If happens that the user interface charts or journals need recalculation at that time, they will receive the temp values from the Generator instead of the already fixed strategy on the main screen. That's why I started developing this StatsBuffer. My idea is the main screen panels to use data only from it. I'll expand the buffer with the necessary data for the Histogram Chart and we can redirect the sources to the buffer.

Re: Feature Proposal: Histogram of Trades

Hi Mr Popov, thanks for your help on this, FSB covers many areas, there are more details than I know ...

Popov wrote:

Krog,
Another issue is that the histogram chart doesn't take the same size as the indicator chart. (Do you need help with this?)

Yes, if possible -- I couldn't get the initialization and event handlers to work out right, I couldn't figure it out.

Popov wrote:

The Histogram Chart shows only loosing trades for the included in the distribution Demo MACD Reversal strategy.

I think it is what happens -- it considers a trade complete by searching for Transaction.Close, and the counts are based on transactions that are Transaction.Close. So, does it need to include Transaction.Reverse?
I'm not sure how to handle Transaction.Reduce, is the close lot considered a trade, or when all open lots are finally closed? I would prefer when all open lots are closed and there are no more open lots, it would make it easier to get the result (from Position.ProfitLoss).

Popov wrote:

Also the chart takes raw values from the Backtester class. This can lead to an issue when the chart is forced to recalculate during the work of Generator or Optimizer. Th problem comes from the fact that Generator uses the Backtester class for its calculations and puts temporary values there. If happens that the user interface charts or journals need recalculation at that time, they will receive the temp values from the Generator instead of the already fixed strategy on the main screen. That's why I started developing this StatsBuffer. My idea is the main screen panels to use data only from it. I'll expand the buffer with the necessary data for the Histogram Chart and we can redirect the sources to the buffer.

I agree, I will have to change it to use the StatsBuffer class.

Re: Feature Proposal: Histogram of Trades

Actually you have made the sizing, but you misspelled the name of the class smile

/// <summary>
/// Arrange the controls after resizing
/// </summary>
void pnlMarket_Resize(object sender, EventArgs e)
{
    smallIndicatorChart.Height = 2 * pnlMarket.ClientSize.Height / (Configs.ShowJournal ? 3 : 4);
    smallHistogramChart.Height = 2 * pnlMarket.ClientSize.Height / (Configs.ShowJournal ? 3 : 4);
}

versus

/// <summary>
/// Arrange the controls after resizing
/// </summary>
void pnlMarket_Resize(object sender, EventArgs e)
{
    smallIndicatorChart.Height = 2 * pnlMarket.ClientSize.Height / (Configs.ShowJournal ? 3 : 4);
    smallIndicatorChart.Height = 2 * pnlMarket.ClientSize.Height / (Configs.ShowJournal ? 3 : 4);
}

Re: Feature Proposal: Histogram of Trades

I think we have to rename the chart. "Histogram Chart" speaks about the chart type, but not about what it interprets. I think "Trade Distribution Chart" will be better.

17 (edited by Popov 2012-02-17 04:28:12)

Re: Feature Proposal: Histogram of Trades

The chart was updated in order to shows also the reversals and reducing trades.

for (int bar = 0; bar < Data.Bars; bar++)
{
    for (int pos = 0; pos < Backtester.Positions(bar); pos++)
    {
        Transaction transaction = Backtester.PosTransaction(bar, pos);
        if (transaction == Transaction.Close  ||
            transaction == Transaction.Reduce ||
            transaction == Transaction.Reverse)
        {
            results[ctr] = (int)Backtester.PosProfitLoss(bar, pos);
            ctr++;
        }
    }
}

Re: Feature Proposal: Histogram of Trades

Popov wrote:

The chart was updated in order to shows also the reversals and reducing trades.

for (int bar = 0; bar < Data.Bars; bar++)
{
    for (int pos = 0; pos < Backtester.Positions(bar); pos++)
    {
        Transaction transaction = Backtester.PosTransaction(bar, pos);
        if (transaction == Transaction.Close  ||
            transaction == Transaction.Reduce ||
            transaction == Transaction.Reverse)
        {
            results[ctr] = (int)Backtester.PosProfitLoss(bar, pos);
            ctr++;
        }
    }
}

On my local copy, I've added for Transaction.Close || Reduce || Reverse.
Small Histogram Chart:
- changed title bar name to "Trade Distribution Chart"
- added bar under title bar for buttons (Toggle, Clipboard and Export)

TBD -- use StatsBuffer instead of Backtest class directly

After I get StatBuffer in there, I will upload to my github repository.

Re: Feature Proposal: Histogram of Trades

StatsBuffer question -- I've updated my local copy to the latest on github, I only see StatBuffer used in Small Indicator Chart. Is it used in Small Balance Chart too? Should it be used in Small Histogram chart?

thanks

Re: Feature Proposal: Histogram of Trades

Should it be used in Small Histogram chart?

I made Small Balance Chart taking data from StatsBuffer, but there was a side effect. It worked perfectly on the main screen. But this chart is also included in the Generator, Optimizer ... What happened is that the chart in the Generator was always showing the balance of the strategy on the main screen smile
I have to rethink the solution. StatsBuffer has to be used only from panels on the main screen because it contains data only for the strategy shown on the main screen. Other panels on Generator or Optimizer has to use local copy of the raw data or to update from Backtester class.

Small Balance Chart and probably Histogram Chart has to use local copy of the raw data. This is because these charts can be shown on main screen and also in some tools like Generator.

Or maybe the charts has to use StatsBuffer when they are shown on main screen and Backtester, when they are shown in a dynamic tool. I'll make some tests with this logic and will update the codes.

Re: Feature Proposal: Histogram of Trades

Here's latest screenshot, I've got this up on github:


http://s7.postimage.org/ycn5f4wd3/hist_Trade_Distribution.jpg

- renamed as Trade Distribution Chart
- buttons on small background under title bar
- with tooltips

Not yet with StatsBuffer.

Re: Feature Proposal: Histogram of Trades

graphical repredentation of trades, that is a great feature Thanks Guys smile
.... I wish i could contribute but this is way too advanced

Posts: 22

Pages 1

You must login or register to post a reply

Forex Software → Developers Forum → Feature Proposal: Histogram of Trades

Similar topics in this forum