Re: FSB Wish List - Requested features

Hi is it possible to use C++ AMP in the application(i guess it is easier to use then OpenCL for some sort of)? At least in the loops for generator / optimizer.  Since it is .net based i guess it is relativly easy to adopt and take some advantage of GPU.

252

Re: FSB Wish List - Requested features

kukreknecmi wrote:

... is it possible to use C++ AMP in the application

I have some experience with CUDAfy.NET and investigated some ideas around integrating GPU processing into FSB a while ago.  My belief is that while it is probably possible, the time involved is unlikely to provide any outcomes that are better than what is possible with the changes that are currently planned for the application.

There is a big overhead in moving data in and out of the GPU in order to offload the calculations, and due to the number of calculations performed during a generator process it is likely that the processing would actually be slower.

As an example, working with 50,000 bars with only 100 FP calculations per bar in a generator run that produces 5,000 strategies would perform 25 billion FP calculations (and there would likely be a lot more than 100 FP calculations per bar).  In order to process these on the GPU, a massive amount of data would needed to be moved back and forward due to the differences in how a CPU -v- a GPU operates.

Assuming that the above process takes 4 hours on your hardware, it is likely that the overhead involved in moving data to and from the CPU to the GPU and back would increase this time, given that now we only need to perform the FP calculation and not shift data around.

My understanding is that popov has a v 2.8 in the works for Q1 next year, with one of the big changes being the ability to multi-thread the generator (and maybe/hopefully the optimizer) process.  In this event, the addition of more CPU cores would provide a more cost effective way of speeding up the analysis process, particularly given the amount of work that would be needed to integrate CUDYfy (or another library).

There is also the obvious issue that there would likely be a very small number of people who have access to GPU capabilities, so the amount of work involved wouldn't necessarily provide any benefit to the wide user community.

Just my thoughts on the topic.

ab

Re: FSB Wish List - Requested features

Since when then program is consuming less than 500mb of ram, doesnt a graphic card with 2 gb ram will suffice? Transfer will be mainly between local memory / L2 to  VRAM , cant the kernel handle this type of optimization / generator parallelization? Still lots of GPU to System RAM transfer needed?

Re: FSB Wish List - Requested features

ab wrote:

My understanding is that popov has a v 2.8 in the works for Q1 next year, with one of the big changes being the ability to multi-thread the generator (and maybe/hopefully the optimizer) process.

Multi-threading is great and very useful feature and I would like to propose to organize workers not as (local) threads but rather as network-accessible workers (including local workers of course). This will allow to use several remote computers - instead of limiting FSB using only one local computer.

255

Re: FSB Wish List - Requested features

A friend of mine wants FSB OS.
I proposed FSB Pad running with FSB OS, equipped with credit card slot for withdrawing profit.

Re: FSB Wish List - Requested features

Popov wrote:

A friend of mine wants FSB OS.
I proposed FSB Pad running with FSB OS, equipped with credit card slot for withdrawing profit.

Don't laugh ;-)
Simply I have access to several 24-way servers and would like to use all of them

257 (edited by ab 2012-12-27 22:43:18)

Re: FSB Wish List - Requested features

kukreknecmi wrote:

Since when then program is consuming less than 500mb of ram, doesnt a graphic card with 2 gb ram will suffice? Transfer will be mainly between local memory / L2 to  VRAM , cant the kernel handle this type of optimization / generator parallelization? Still lots of GPU to System RAM transfer needed?

A GPU doesn't function in the same way as a CPU - they are very fast for low-level floating point calculations but the calculations being performed in FSB can't just be passed to the GPU directly.  Even a basic calculation such as an addition which is written in C# as:

int a = b + c

Needs to be loaded into the GPU first (only once), eg:

[Cudafy]
public static void add(int a, int b, int[] c)
{
    c[0] = a + b;
}

And then the underlying FSB code would need to be modified to support GPU memory allocation, GPU calculation, data transfer, and memory release, eg:

int dev_c = gpu.allocate<int>();
int c;
gpu.Launch().add(5, 3, dev_c);
gpu.CopyFromDevice(dev_c, out c);
gpu.Free(dev_c);

Please keep in mind:

  • The above example only adds two numbers, and this type of work would need to be performed for every low-level calculation that needs to be performed (hundreds of times for some indicators), we are talking about a massive coding effort.

  • In order to take full advantage of the multi-core GPU we really need to be multi-threading the data in and out, so the work to migrate FSB into a mutli-threaded application would still be needed although now threads would need to be co-ordinated across both the CPU and GPU cores.

My previous post wasn't because I don't believe that implementing GPU functionality into FSB isn't possible, I just don't believe that it is practical given the massive amount of work required.

Note: Code examples are from CUDAfu.NET - http://cudafy.codeplex.com/

258

Re: FSB Wish List - Requested features

vlad123 wrote:

Multi-threading is great and very useful feature and I would like to propose to organize workers not as (local) threads but rather as network-accessible workers (including local workers of course). This will allow to use several remote computers - instead of limiting FSB using only one local computer.

Rather than network-accessible workers, what about a something that uses IP multicast to send the results (anything that hits 'Top 10' in the generator) onto the network for collection by a central 'listener' application?

This would allow multiple computers to be running the generator, with a central system collating all the results and allow for selection of the best from all generator processes.

259 (edited by kukreknecmi 2013-01-01 15:44:31)

Re: FSB Wish List - Requested features

I understand the main concepts of gpu coding, what makes me wonder is, once the data set is copied to Vram and after the  kernel started, how much copy back is needed? Ofc i dont know how FSB calculates each condition but, if the main code is mainly build on  calculate-copy-back-compare kinda stuff and it is the only way to do, then ofc it will be slow as hell. Yet restricting to cudafy is a obstructive imo, since it will only limit us to Nvidia (ofc i can buy an Nvidia card if a gPu versin speeds up thing in the order of magnitude).

260

Re: FSB Wish List - Requested features

ab wrote:

My previous post wasn't because I don't believe that implementing GPU functionality into FSB isn't possible, I just don't believe that it is practical given the massive amount of work required.

As mentioned, this is not an impossible task but it is a very large amount of work and then is only useful for those with GPU capabilities.  In terms of effort -v- return, my feeling is that moving FSB to a multi-threaded model would be something that is beneficial to all users, not just the select few with special purpose hardware.

If this is something that you have some experience with then perhaps you could have a go at it and see if the work is worth the return.

261

Re: FSB Wish List - Requested features

kukreknecmi wrote:

... Yet restricting to cudafy is a obstructive imo, since it will only limit us to Nvidia...

At no stage was I suggesting that IF any form of GPU work was undertaken we should use CUDA, I just used the code as an example of what would need to be done in the C# space (since FSB isn't a C++ project).

Re: FSB Wish List - Requested features

kk i see now, thx for clearance.

Re: FSB Wish List - Requested features

ab wrote:

Rather than network-accessible workers, what about a something that uses IP multicast to send the results (anything that hits 'Top 10' in the generator) onto the network for collection by a central 'listener' application?

This would allow multiple computers to be running the generator, with a central system collating all the results and allow for selection of the best from all generator processes.

Multicast can save some IP traffic but it's VERY questionable thing - nobody can predict if network administrator will allow/disallow multicast on the way between Generator and Workers.
Also - I see no benefits of using multicast in comparison with traditional Generator/Workers schema.

264

Re: FSB Wish List - Requested features

vlad123 wrote:

Multicast can save some IP traffic but it's VERY questionable thing - nobody can predict if network administrator will allow/disallow multicast on the way between Generator and Workers.

Agreed, although I wasn't trying and save bandwidth - just making a suggestions that could probably be built into the current code fairly quickly as there is no need for any work to be done on the core threading model (multicast would allow for anywhere from  n concurrent instances of FSB to be run).

In terms of the 'network admin issue', I am sure that is a business is using FSB for analysis then making multicast available to their analysts wouldn't be a problem. 

I was just suggesting an alternative that (I agree) is nowhere near as robust as a truly distributed application, but faster to develop.

265 (edited by ab 2013-02-01 12:44:50)

Re: FSB Wish List - Requested features

I would like to have a change made to the balance/equity chart to allow a visual display for when working with a loaded strategy file.

In the attached image, the top chart is as per the current software version, where the bottom chart has a vertical line (or possibly shaded area to the right of where the line is indicated).

This line would (ideally) be able to display the value contained stored in the 'No data newer than' variable that is now (as at version 2.74) stored in the strategy file.

This would allow for strategies developed in the past to be re-opened with FSB in the future and it would be easy to identify the data that this strategy has not 'seen' in the past.

ab

Post's attachments

chart2.png 42.78 kb, file has never been downloaded. 

You don't have the permssions to download the attachments of this post.

266

Re: FSB Wish List - Requested features

What I understand is that you want a mark of the last data used when the strategy was calculated and saved last time. In that way you want to see a true Out of Sample performance.

I'll try this in some of the next FSB Pro releases.

Re: FSB Wish List - Requested features

lots of time i find a profitable strategy but some of trades one or two trades still open for months , such strategy couldnot be used in real life

could popov add a new feature limits the maximum time for open trades (per trade)  lets say i have a strategy containing one trade ( stayed open for 10 days ) so this strategy will be excluded

Post's attachments

ScreenHunter_32 Feb. 09 09.53.jpg
ScreenHunter_32 Feb. 09 09.53.jpg 23.15 kb, 1 downloads since 2013-02-09 

You don't have the permssions to download the attachments of this post.

268

Re: FSB Wish List - Requested features

What i would like to see is new limitation in strategy generator/optimizer.
Now you can set up min number of trades and percentage of winning trades.
What i would like to get is minimum number of winning trades, and do not limit it to 1000 please wink

269

Re: FSB Wish List - Requested features

I'll try to make custom limitations addons for Generator as well as custom stats params in FSB  Pro.

Re: FSB Wish List - Requested features

Popov wrote:

I'll try to make custom limitations addons for Generator as well as custom stats params in FSB  Pro.

fantastic Thanks

Re: FSB Wish List - Requested features

ahmedalhoseny wrote:

lots of time i find a profitable strategy but some of trades one or two trades still open for months , such strategy couldnot be used in real life

could popov add a new feature limits the maximum time for open trades (per trade)  lets say i have a strategy containing one trade ( stayed open for 10 days ) so this strategy will be excluded

Inactivity Minimum Change stop loss
This stop closes an open position when the instrument price does not make a minimum  price change within a specified time
Regards

Re: FSB Wish List - Requested features

Take Profit

I think Take Profit is restricted to being greater than 4 pips.

Could that be modified so that there is no restriction.?

Thanks

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....

Re: FSB Wish List - Requested features

Popov wrote:
Blaiserboy wrote:

That would be great. I thank you for considering that.

I think that many times is necessery to tweak some of the integrated indicators. Probably overiding feature will be useful. I'll add it to the beta version next weak.
Probably Ill announce a new property of a custom indicator: OverideSameNameIndicator = true;


Has this feature 'OverrideSameNameIndicator' been implemented...... and, if so.. how can I access it...?

Thanks

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....

274 (edited by Popov 2013-02-14 04:04:06)

Re: FSB Wish List - Requested features

Blaiserboy wrote:

Has this feature 'OverrideSameNameIndicator' been implemented...... and, if so.. how can I access it...?

It is implemented in FSB Pro. It works like a charm. In FSB Pro you don't need to mark Custom indicators. The programs mark them at loading.
In order to replace an existing indicator with a custom one, you have to add OverrideMainIndicator = true; in constructor. That's all.

Edit: Actually this version is not publish yet. Custom indicators will be available in upcoming alpha 2.0

Re: FSB Wish List - Requested features

This new version is going to be terrific.!!

My 'secret' goal is to push EA Studio until I can net 3000 pips per day....