Regression Node.js Streams from 16 to 18

[RafaelGSS]

This pipe regression, however, was reduced in Node.js 20:

$ node-benchmark-compare compare-streams-18-20.csv
 confidence improvement accuracy (*) (**) (***)
streams/creation.js kind='duplex' n=50000000 *** 12.76 % ±4.30% ±5.73% ±7.47%
streams/creation.js kind='readable' n=50000000 *** 3.48 % ±1.16% ±1.55% ±2.05%
streams/creation.js kind='transform' n=50000000 ** -7.59 % ±5.27% ±7.02% ±9.16%
streams/creation.js kind='writable' n=50000000 *** 4.20 % ±0.87% ±1.16% ±1.53%
streams/destroy.js kind='duplex' n=1000000 *** -6.33 % ±1.08% ±1.43% ±1.87%
streams/destroy.js kind='readable' n=1000000 *** -1.94 % ±0.70% ±0.93% ±1.21%
streams/destroy.js kind='transform' n=1000000 *** -7.44 % ±0.93% ±1.24% ±1.62%
streams/destroy.js kind='writable' n=1000000 0.20 % ±1.89% ±2.52% ±3.29%
streams/pipe.js n=5000000 *** 87.18 % ±2.58% ±3.46% ±4.56%

---

I'll share more detail when I publish the "state of nodejs performance 2023" blog post.

[mcollina]

do you known what caused it?

[krassx]

Is this regression still relevant? I am planning migration from 16.x to 20.x and stumbled upon this issue.

[mcollina]

Likely so, however the team has not had time to work on it.

[metcoder95]

I imagine we'll need to make git blame? 😅

[kylo5aby]

seems its better on 20.x, but compares to 16.x, Duplex is still slower.

node-benchmark-compare result_16_20.csv 
 confidence improvement accuracy (*) (**) (***)
streams/creation.js kind='duplex' n=50000000 *** -55.43 % ±0.95% ±1.27% ±1.65%
streams/creation.js kind='readable' n=50000000 *** -28.60 % ±0.99% ±1.32% ±1.73%
streams/creation.js kind='transform' n=50000000 -1.31 % ±3.19% ±4.25% ±5.55%
streams/creation.js kind='writable' n=50000000 *** -31.30 % ±0.96% ±1.28% ±1.67%
streams/destroy.js kind='duplex' n=1000000 *** -12.42 % ±1.07% ±1.43% ±1.85%
streams/destroy.js kind='readable' n=1000000 *** -5.76 % ±1.19% ±1.58% ±2.06%
streams/destroy.js kind='transform' n=1000000 *** -3.69 % ±1.13% ±1.50% ±1.96%
streams/destroy.js kind='writable' n=1000000 *** -11.04 % ±1.83% ±2.44% ±3.18%
streams/pipe-object-mode.js n=5000000 *** 2.64 % ±1.11% ±1.48% ±1.92%
streams/pipe.js n=5000000 -0.23 % ±1.79% ±2.38% ±3.10%
streams/readable-async-iterator.js sync='no' n=100000 *** 9.45 % ±0.85% ±1.13% ±1.47%
streams/readable-async-iterator.js sync='yes' n=100000 *** 3.55 % ±1.09% ±1.45% ±1.88%
streams/readable-bigread.js n=1000 *** 5.49 % ±0.84% ±1.12% ±1.46%
streams/readable-bigunevenread.js n=1000 *** -8.65 % ±3.77% ±5.02% ±6.54%
streams/readable-boundaryread.js type='buffer' n=2000 0.05 % ±0.43% ±0.57% ±0.75%
streams/readable-boundaryread.js type='string' n=2000 *** 50.98 % ±0.80% ±1.07% ±1.40%
streams/readable-from.js n=10000000 ** 1.57 % ±1.03% ±1.37% ±1.79%
streams/readable-readall.js n=5000 *** -3.67 % ±1.39% ±1.87% ±2.48%
streams/readable-unevenread.js n=1000 *** -32.08 % ±0.24% ±0.32% ±0.42%
streams/writable-manywrites.js len=1024 callback='no' writev='no' sync='no' n=2000000 ** 1.70 % ±0.99% ±1.33% ±1.74%
streams/writable-manywrites.js len=1024 callback='no' writev='no' sync='yes' n=2000000 *** 14.23 % ±1.77% ±2.35% ±3.07%
streams/writable-manywrites.js len=1024 callback='no' writev='yes' sync='no' n=2000000 *** 19.58 % ±1.17% ±1.57% ±2.06%
streams/writable-manywrites.js len=1024 callback='no' writev='yes' sync='yes' n=2000000 *** 5.26 % ±1.16% ±1.54% ±2.00%
streams/writable-manywrites.js len=1024 callback='yes' writev='no' sync='no' n=2000000 ** 1.48 % ±0.87% ±1.17% ±1.53%
streams/writable-manywrites.js len=1024 callback='yes' writev='no' sync='yes' n=2000000 *** 14.33 % ±1.78% ±2.39% ±3.14%
streams/writable-manywrites.js len=1024 callback='yes' writev='yes' sync='no' n=2000000 *** 20.86 % ±1.15% ±1.53% ±1.99%
streams/writable-manywrites.js len=1024 callback='yes' writev='yes' sync='yes' n=2000000 *** 3.77 % ±1.18% ±1.57% ±2.05%
streams/writable-manywrites.js len=32768 callback='no' writev='no' sync='no' n=2000000 *** 2.23 % ±0.64% ±0.85% ±1.11%
streams/writable-manywrites.js len=32768 callback='no' writev='no' sync='yes' n=2000000 *** 7.94 % ±1.43% ±1.92% ±2.51%
streams/writable-manywrites.js len=32768 callback='no' writev='yes' sync='no' n=2000000 *** 4.99 % ±0.99% ±1.33% ±1.76%
streams/writable-manywrites.js len=32768 callback='no' writev='yes' sync='yes' n=2000000 ** -4.69 % ±2.77% ±3.68% ±4.79%
streams/writable-manywrites.js len=32768 callback='yes' writev='no' sync='no' n=2000000 ** 1.87 % ±1.35% ±1.81% ±2.38%
streams/writable-manywrites.js len=32768 callback='yes' writev='no' sync='yes' n=2000000 *** 6.62 % ±0.91% ±1.21% ±1.59%
streams/writable-manywrites.js len=32768 callback='yes' writev='yes' sync='no' n=2000000 *** 3.48 % ±1.15% ±1.55% ±2.05%
streams/writable-manywrites.js len=32768 callback='yes' writev='yes' sync='yes' n=2000000 *** -4.57 % ±1.75% ±2.34% ±3.08%

[ronag]

What about v21?

[kylo5aby]

v21 has improved compared to v20.x, I believe maybe because the performance of writable streams has been enhanced.

node-benchmark-compare result_16_21.csv 
 confidence improvement accuracy (*) (**) (***)
streams/creation.js kind='duplex' n=50000000 *** -44.15 % ±1.33% ±1.78% ±2.33%
streams/creation.js kind='readable' n=50000000 *** -23.09 % ±1.08% ±1.44% ±1.88%
streams/creation.js kind='transform' n=50000000 *** 25.96 % ±3.58% ±4.83% ±6.41%
streams/creation.js kind='writable' n=50000000 *** -16.77 % ±1.37% ±1.84% ±2.41%
streams/destroy.js kind='duplex' n=1000000 *** 38.30 % ±1.11% ±1.48% ±1.93%
streams/destroy.js kind='readable' n=1000000 *** 3.61 % ±1.05% ±1.40% ±1.82%
streams/destroy.js kind='transform' n=1000000 *** 42.59 % ±1.20% ±1.60% ±2.10%
streams/destroy.js kind='writable' n=1000000 *** -5.30 % ±1.21% ±1.62% ±2.11%
streams/pipe-object-mode.js n=5000000 *** -3.99 % ±2.08% ±2.80% ±3.70%
streams/pipe.js n=5000000 *** -15.25 % ±1.08% ±1.45% ±1.91%
streams/readable-async-iterator.js sync='no' n=100000 0.40 % ±0.55% ±0.73% ±0.96%
streams/readable-async-iterator.js sync='yes' n=100000 *** -9.26 % ±0.73% ±0.97% ±1.27%
streams/readable-bigread.js n=1000 *** 4.81 % ±0.58% ±0.77% ±1.00%
streams/readable-bigunevenread.js n=1000 *** 10.32 % ±2.46% ±3.31% ±4.39%
streams/readable-boundaryread.js type='buffer' n=2000 *** -2.11 % ±0.80% ±1.07% ±1.41%
streams/readable-boundaryread.js type='string' n=2000 *** 48.42 % ±0.87% ±1.17% ±1.54%
streams/readable-from.js n=10000000 -1.97 % ±2.50% ±3.34% ±4.38%
streams/readable-readall.js n=5000 ** 5.01 % ±2.85% ±3.82% ±5.03%
streams/readable-unevenread.js n=1000 *** -31.79 % ±0.34% ±0.45% ±0.59%
streams/writable-manywrites.js len=1024 callback='no' writev='no' sync='no' n=2000000 *** -4.70 % ±0.70% ±0.93% ±1.22%
streams/writable-manywrites.js len=1024 callback='no' writev='no' sync='yes' n=2000000 *** -5.16 % ±1.11% ±1.48% ±1.94%
streams/writable-manywrites.js len=1024 callback='no' writev='yes' sync='no' n=2000000 *** -9.61 % ±2.61% ±3.51% ±4.66%
streams/writable-manywrites.js len=1024 callback='no' writev='yes' sync='yes' n=2000000 *** -3.92 % ±0.87% ±1.16% ±1.53%
streams/writable-manywrites.js len=1024 callback='yes' writev='no' sync='no' n=2000000 *** -11.22 % ±0.76% ±1.02% ±1.33%
streams/writable-manywrites.js len=1024 callback='yes' writev='no' sync='yes' n=2000000 *** -16.76 % ±0.79% ±1.05% ±1.37%
streams/writable-manywrites.js len=1024 callback='yes' writev='yes' sync='no' n=2000000 *** -4.37 % ±1.20% ±1.62% ±2.14%
streams/writable-manywrites.js len=1024 callback='yes' writev='yes' sync='yes' n=2000000 *** -12.36 % ±0.80% ±1.07% ±1.41%
streams/writable-manywrites.js len=32768 callback='no' writev='no' sync='no' n=2000000 *** 11.51 % ±1.94% ±2.61% ±3.45%
streams/writable-manywrites.js len=32768 callback='no' writev='no' sync='yes' n=2000000 *** 391.49 % ±4.78% ±6.45% ±8.55%
streams/writable-manywrites.js len=32768 callback='no' writev='yes' sync='no' n=2000000 *** 15.08 % ±1.77% ±2.38% ±3.15%
streams/writable-manywrites.js len=32768 callback='no' writev='yes' sync='yes' n=2000000 *** 344.29 % ±1.68% ±2.25% ±2.96%
streams/writable-manywrites.js len=32768 callback='yes' writev='no' sync='no' n=2000000 *** 9.33 % ±1.24% ±1.65% ±2.16%
streams/writable-manywrites.js len=32768 callback='yes' writev='no' sync='yes' n=2000000 *** 340.15 % ±1.50% ±2.02% ±2.67%
streams/writable-manywrites.js len=32768 callback='yes' writev='yes' sync='no' n=2000000 *** 13.13 % ±1.70% ±2.28% ±3.01%
streams/writable-manywrites.js len=32768 callback='yes' writev='yes' sync='yes' n=2000000 *** 319.79 % ±3.98% ±5.32% ±6.97%

[krassx]

It would be great if all the stream optimizations can be backported to 20, which is about to become LTS.