1 Topic by geektrader (edited by geektrader 2018-06-19 09:29:49)

  • From: Germany
  • Registered: 2010-04-05
  • Posts: 831

Topic: "Aw, Snap!" in Chrome / Chromium

If anyone wonders why they get a "Aw, Snap!" tab crash in Chrome / Chromium if (re-)validating / (re-)testing large collections of systems, this is not solely because of EA Studio, but because of Chrome FORCING a limit of 2GB RAM usage per tab instance. You can raise this is a bit by using the command:

chrome.exe --args --js-flags="--max_old_space_size=4096"

This raises the max allowed per-tab memory to 4GB, but that is the absolute limit (by design, because of "exploits" using buffer overflows as the developers say). If you revalidate large collections, this is still not enough, so I am using Firefox for these special instances, because it has no process limit per tab at all (had it using ~40GB here already) and the crashes will be gone.

@Mr. Popov: maybe there still is a way to stop the memory "leak" that EA Studio seems to create during revalidating a large number of strategies (>4000 on 32 years of H4 data)? Because the funny thing is, as soon as the re-validation is done, RAM drops to just 0.5GB right away. So this seems to be a leak that only happens during continuos retesting and has nothing to do with the strategies that end up in the collection (which are no more than 100 anyway).

geektrader's Website

  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 7,976

Re: "Aw, Snap!" in Chrome / Chromium

Hmm. This is very interesting. I have to examine the case.

Thank you for the report.

Popov's Website

3 Reply by geektrader (edited by geektrader 2018-06-19 10:06:23)

  • From: Germany
  • Registered: 2010-04-05
  • Posts: 831

Re: "Aw, Snap!" in Chrome / Chromium

Yes, just use ~6000 strategies in the Validator on around 49000 bars. Nothing special apart from that, the memory raises and raises up to several GB quickly (once 2GB / 4GB reached, Chrome will crash the tab, so use Firefox to see how much further up it goes - 40GB in my case on 6000 strats). Then, right when the test finished, it drops back to a hundred megabytes. Also when generating there is no issue like that, only in the Validator...

geektrader's Website

  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 7,976

Re: "Aw, Snap!" in Chrome / Chromium

Please make tests with the Validator without additional tools enables. In that way we will know if the problem is in the Validator core. If the memory usage is not so drastic, please try to use the different tools one by one to see which one accumulates memory.  I'll make the same test, however you may find the relation faster.

Popov's Website

  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 7,976

Re: "Aw, Snap!" in Chrome / Chromium

Yaaa, there is a third party commercial service that will solve your case.

https://s15.postimg.cc/5j90d8zbf/screenshot_244.png

Popov's Website

6 Reply by geektrader (edited by geektrader 2018-06-20 01:56:15)

  • From: Germany
  • Registered: 2010-04-05
  • Posts: 831

Re: "Aw, Snap!" in Chrome / Chromium

Haha :-) Good one!

"Tools" in terms of what exactly? Addons you mean? I am using stock Chromium / Firefox, no addons, the memory usage is the same on both, just that Chrome crashes >2GB / 4GB (if using the command line to extend it from 2 to 4GB) and Firefox just keeps on going up to 40GB usage at the end of the 6000 strategies test, then drops to just a few MB once done.

Have you tried calling the garbage collector in the validation loop in your code? Maybe that helps...

geektrader's Website

  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 7,976

Re: "Aw, Snap!" in Chrome / Chromium

A possible reason for the RAM overuse is that the Validator keeps the full strategy backtesting status of all the strategies during the calculation. I'll try to add some sort of cleanup after a calculation.  I'll make experiments and will report.

Popov's Website

  • Popov
  • Popov
  • Lead Developer
  • Offline
  • From: Bulgaria
  • Registered: 2006-11-30
  • Posts: 7,976

Re: "Aw, Snap!" in Chrome / Chromium

A new version is uploaded. Please reload the app and test.

Popov's Website

9 Reply by geektrader (edited by geektrader 2018-06-21 02:48:46)

  • From: Germany
  • Registered: 2010-04-05
  • Posts: 831

Re: "Aw, Snap!" in Chrome / Chromium

Perfect, no more leaks, even if validating 20000 strategies! No more need for Firefox, the most that was used during the validation was ~1GB in Chrome. Nice work, thank you!

geektrader's Website