Node runs out of memory running our builds

@hterkelsen

I've seen it crash on JS bundle tasks and build/pure-packages.dart. @hterkelsen reports it crashing on JS sourcemaps.

Workaround: node --max-old-space-size=2000 ./node_modules/.bin/gulp {TASK}

(2000 is 2GB)

fix the underlying issue
remove the heap config workaround from scripts/publish/npm_publish.sh

@IgorMinar

i merged this one yesterday to try mitigate the JS/source map end without getting too invasive, and it seemed to help somewhat, but might need a bit more work. #5208

See also #5223 as we're likely to be re-orging this

I was so happy that we can run this bundling in parallel :-) But yeh, if it gives trouble to our build infrastructure we can easily re-shape it to do things sequentially. Provided that we are sure that this is the root cause, of course.

it might be enough to just refactor the code bundle sequentially instead of using Q.all.

I have a pending PR of the new system builder, that allows separation
between the tracing step and the writing to disk step. It definitely is
much faster and lighter weight. Coming soon.
On Fri, Nov 13, 2015 at 12:11 Igor Minar [email protected] wrote:

it might be enough to just refactor the code bundle sequentially instead
of using Q.all.

—
Reply to this email directly or view it on GitHub
#5229 (comment).

build(npm_publish.sh): run gulp via node --max-old-space-size=1900 to…

1e74058 build(npm_publish.sh): run gulp via node --max-old-space-size…

Just in case it helps. I think this trace might be related to this issue. Running gulp build on OS X El Capitan with node v4.2.4 :

[12:38:10] Starting '!bundle.testing'...

<--- Last few GCs --->

  113851 ms: Mark-sweep 1380.6 (1456.9) -> 1383.2 (1456.9) MB, 1259.5 / 0 ms [allocation failure] [GC in old space requested].
  115110 ms: Mark-sweep 1383.2 (1456.9) -> 1383.2 (1456.9) MB, 1258.9 / 0 ms [allocation failure] [GC in old space requested].
  116382 ms: Mark-sweep 1383.2 (1456.9) -> 1378.1 (1456.9) MB, 1272.0 / 0 ms [last resort gc].
  117639 ms: Mark-sweep 1378.1 (1456.9) -> 1379.9 (1456.9) MB, 1256.3 / 0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0xf6261cb4629 <JS Object>
    2: _serializeMappings(aka SourceMapGenerator_serializeMappings) [/Users/alfonso/Workspace/angular/node_modules/systemjs-builder/node_modules/source-map/lib/source-map/source-map-generator.js:~289] [pc=0x111d3ae2115e] (this=0x1b111ff80b59 <a SourceMapGenerator with map 0x21a7c3960b41>)
    3: toJSON(aka SourceMapGenerator_toJSON) [/Users/alfonso/Workspace/angular/node_modules/systemjs-builder...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
Abort trap: 6

It doesn't happen if I only run gulp build.js

Contributing a failure for gulp build.js. On node v4.2.1 El Capitan 16GB RAM.

[01:06:42] Starting '!bundle.testing'...

<--- Last few GCs --->

  140814 ms: Scavenge 1374.7 (1455.7) -> 1374.7 (1455.7) MB, 3.5 / 0 ms (+ 1.7 ms in 1 steps since last GC) [allocation failure] [incremental marking delayin
g mark-sweep].
  142008 ms: Mark-sweep 1374.7 (1455.7) -> 1296.6 (1455.7) MB, 1194.1 / 0 ms (+ 3.1 ms in 2 steps since start of marking, biggest step 1.7 ms) [last resort g

c].
  143279 ms: Mark-sweep 1296.6 (1455.7) -> 1296.6 (1455.7) MB, 1271.4 / 0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x3e0e73637399 <JS Object>
    2: _serializeMappings(aka SourceMapGenerator_serializeMappings) [/Users/stephenbussey/personal/angular/node_modules/systemjs-builder/node_modules/source-
map/lib/source-map/source-map-generator.js:~289] [pc=0x13217c8fd240] (this=0x2b5a4f0e72d1 <a SourceMapGenerator with map 0x37c172fd5261>)
    3: toJSON(aka SourceMapGenerator_toJSON) [/Users/stephenbussey/personal/angular/node_modules/syste...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
[1]    95243 abort      $(npm bin)/gulp build.js

This script seems to do the trick, however.

node --max-old-space-size=2000 ./node_modules/.bin/gulp build.js

@pkozlowski-opensource

@pkozlowski-opensource is looking into this. I don't think that we'll be able to reduce the memory usage easily without impacting the total build time, so let's increase the heap size by default when calling gulp build.js (or more specifically the bundle task that causes the OOM).

This will require spawning a new process from within the gulp process but that should be a big deal. We just need to make sure that it's done a cross-platform compatible way.

I am also hitting this issue with gulp build and gulp build.js. Is there a mitigation I can use to get around this?

workaround is node --max_old_space_size=4000 ./node_modules/.bin/gulp ...

On Tue, Mar 8, 2016 at 4:25 PM Zack Hall [email protected] wrote:

I am also hitting this issue with gulp build and gulp build.js. Is there
a mitigation I can use to get around this?

—
Reply to this email directly or view it on GitHub
#5229 (comment).

@alexeagle

Thanks, @alexeagle. That worked for me!

Documentation should just cite npm run build as that takes care of it for you.

This one is obsolete with all the recent build changes.

on windows environment I needed:

"scripts": {
  "build": "node --max-old-space-size=2048 node_modules/gulp/bin/gulp.js"
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
}

run as usual:

npm run build

folks just a question

I've been running into process out of memory issues while running ngc.
ngc runs fine for the first time when things are in JIT mode in main.ts, after first run, I change main.ts to AoT mode.
Then ngc takes a while to compile and the process ends up out of memory.

Is there a way to increase --max-old-space-size while running ngc?

Please file a new issue as this is unrelated to the old issue.

On Fri, Oct 28, 2016, 9:44 PM Sukhdeep Singh [email protected]
wrote:

folks just a question

I've been running into process out of memory issues while running ngc.
ngc runs fine for the first time when things are in JIT mode in main.ts,
after first run, I change main.ts to AoT mode.
Then ngc takes a while to compile and the process ends up out of memory.

Is there a way to increase --max-old-space-size while running ngc?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#5229 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AANM6M3lErFQixAujxARi0aDbG-6goz9ks5q4lB1gaJpZM4Gf6Oo
.

there is issue #12184 which is also closed.
I couldn't find a solution anywhere to this

https://www.npmjs.com/package/increase-memory-limit

npm install -g increase-memory-limit

run inside project
increase-memory-limit
worked for me.

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}

Navigation Menu

Repository navigation

Node runs out of memory running our builds #5229

16 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Node runs out of memory running our builds #5229

Description

Activity

yjbanov commented on Nov 11, 2015

robwormald commented on Nov 11, 2015

pkozlowski-opensource commented on Nov 11, 2015

IgorMinar commented on Nov 13, 2015

robwormald commented on Nov 13, 2015

alfonso-presa commented on Jan 7, 2016

sb8244 commented on Jan 21, 2016

IgorMinar commented on Jan 22, 2016

16 remaining items

zackhall commented on Mar 9, 2016

alexeagle commented on Mar 9, 2016

zackhall commented on Mar 9, 2016

jocull commented on May 3, 2016

pkozlowski-opensource commented on Jun 6, 2016

jepetko commented on Jun 22, 2016

singhjusraj commented on Oct 28, 2016

IgorMinar commented on Oct 28, 2016

singhjusraj commented on Oct 28, 2016

AjayShelar commented on Sep 28, 2018

angular-automatic-lock-bot commented on Sep 14, 2019

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions