Hi @ericf as we discussed, there might be some case to cause a GC pause,
I'm guessing it's some V8 de-opt we're hitting because the replacer forces V8 to go from native-land to user-land and back. Whereas with JSON.stringify().replace() it can stay in native-land the whole time.
So it's probably causing a serialization/de-serialization of the massive string causing a GC pause.
Now we have data ~500KB to be serialized, and found that some case(typically when the data is complicated and big enough) str.replace takes a very long time after JSON.stringify with replacer
To reproduce it, here's the simple script with the data we have,
We will get rid of it with a gc() or run without replacer, it acts like what you mentioned.
https://drive.google.com/file/d/0B0ZB5_LvayjrRjV0U2hrMGVjMUE/view?usp=sharing
please run node --trace-gc --replacer --issue ./slowserialization.js to reproduce it and see gc solve it with node --trace-gc --expose-gc --replacer --issue --gc ./slowserialization.js
will see when slow serialization happens, there's a lot of sweep and cause time spend.
[41363] 3133 ms: Mark-sweep 331.0 (368.4) -> 327.2 (366.4) MB, 123 ms [last resort gc].
[41363] 3249 ms: Mark-sweep 327.2 (366.4) -> 327.2 (364.4) MB, 116 ms [last resort gc].
[41363] 3361 ms: Mark-sweep 327.2 (364.4) -> 327.2 (364.4) MB, 112 ms [last resort gc].
[41363] 3475 ms: Mark-sweep 327.2 (364.4) -> 327.2 (364.4) MB, 113 ms [last resort gc].
[41363] 3578 ms: Mark-sweep 327.2 (364.4) -> 327.2 (364.4) MB, 103 ms [last resort gc].
[41363] 3676 ms: Mark-sweep 327.2 (364.4) -> 327.2 (364.4) MB, 97 ms [last resort gc].
[41363] 3776 ms: Mark-sweep 327.2 (364.4) -> 327.2 (364.4) MB, 100 ms [last resort gc].
The slow serialization only happens with Node.js 10, 12 won't have that issue.
I don't think that makes sense to fix this only for Node.js 10, but let's discuss here, like can we try something else for handling the function/regexp or make that optional (for our main customer's case, we don't have function and regexp)?
@redonkulus @lingyan @mridgway @Vijar
Hi @ericf as we discussed, there might be some case to cause a
GC pause,Now we have data
~500KBto be serialized, and found that some case(typically when the data is complicated and big enough) str.replace takes a very long time after JSON.stringify with replacerTo reproduce it, here's the simple script with the data we have,
We will get rid of it with a
gc()or run withoutreplacer, it acts like what you mentioned.https://drive.google.com/file/d/0B0ZB5_LvayjrRjV0U2hrMGVjMUE/view?usp=sharing
please run
node --trace-gc --replacer --issue ./slowserialization.jsto reproduce it and see gc solve it withnode --trace-gc --expose-gc --replacer --issue --gc ./slowserialization.jswill see when slow serialization happens, there's a lot of sweep and cause time spend.
The slow serialization only happens with Node.js
10,12won't have that issue.I don't think that makes sense to fix this only for Node.js 10, but let's discuss here, like can we try something else for handling the
function/regexpor make that optional (for our main customer's case, we don't have function and regexp)?@redonkulus @lingyan @mridgway @Vijar