Skip to content

Conversation

@Trott
Copy link
Member

This reverts commit 3af175f.

Opening this as a draft PR so I can easily run benchmarks.

@nodejs-github-botnodejs-github-bot added the http2 Issues or PRs related to the http2 subsystem. label Jan 11, 2021
@Trott
Copy link
MemberAuthor

@aduh95
Copy link
Contributor

Benchmark didn't show any improvement or regressions

Details
 confidence improvement accuracy (*) (**) (***) http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=100 requests=100 -2.70 % ±9.61% ±12.79% ±16.64% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=100 requests=1000 -4.85 % ±10.39% ±13.82% ±18.00% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=100 requests=5000 -0.87 % ±9.96% ±13.25% ±17.24% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=10 requests=100 -2.19 % ±9.99% ±13.29% ±17.29% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=10 requests=1000 3.52 % ±9.29% ±12.35% ±16.08% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=10 requests=5000 -5.22 % ±8.91% ±11.85% ±15.42% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=1 requests=100 1.22 % ±10.64% ±14.16% ±18.44% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=1 requests=1000 5.16 % ±9.37% ±12.47% ±16.24% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=1 requests=5000 -2.09 % ±9.71% ±12.92% ±16.81% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=200 requests=100 0.55 % ±9.83% ±13.08% ±17.03% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=200 requests=1000 1.41 % ±10.18% ±13.54% ±17.62% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=200 requests=5000 -4.72 % ±10.60% ±14.10% ±18.35% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=20 requests=100 8.76 % ±10.09% ±13.42% ±17.47% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=20 requests=1000 -7.23 % ±10.07% ±13.41% ±17.47% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=20 requests=5000 1.36 % ±10.19% ±13.55% ±17.64% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=40 requests=100 -6.32 % ±10.05% ±13.38% ±17.43% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=40 requests=1000 -1.56 % ±10.79% ±14.36% ±18.69% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=40 requests=5000 2.06 % ±10.40% ±13.84% ±18.02% http2/headers.js nheaders=0 n=1000 -0.50 % ±4.52% ±6.02% ±7.86% http2/headers.js nheaders=1000 n=1000 -0.83 % ±2.91% ±3.87% ±5.04% http2/headers.js nheaders=100 n=1000 -0.07 % ±4.56% ±6.07% ±7.90% http2/headers.js nheaders=10 n=1000 -1.27 % ±2.06% ±2.75% ±3.58% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=100 requests=100 7.02 % ±13.69% ±18.23% ±23.76% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=100 requests=1000 -2.62 % ±13.17% ±17.53% ±22.84% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=100 requests=5000 4.61 % ±15.12% ±20.12% ±26.19% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=10 requests=100 -2.68 % ±12.83% ±17.07% ±22.21% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=10 requests=1000 8.76 % ±13.75% ±18.30% ±23.83% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=10 requests=5000 -6.43 % ±12.76% ±16.97% ±22.10% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=1 requests=100 1.68 % ±12.68% ±16.88% ±21.98% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=1 requests=1000 -4.64 % ±12.78% ±17.03% ±22.22% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=1 requests=5000 7.55 % ±15.70% ±20.91% ±27.27% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=200 requests=100 -0.99 % ±11.82% ±15.72% ±20.47% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=200 requests=1000 9.45 % ±11.50% ±15.31% ±19.92% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=200 requests=5000 -11.39 % ±12.12% ±16.14% ±21.02% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=20 requests=100 13.60 % ±14.58% ±19.40% ±25.26% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=20 requests=1000 7.43 % ±14.09% ±18.77% ±24.49% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=20 requests=5000 -0.31 % ±14.73% ±19.60% ±25.52% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=40 requests=100 -7.11 % ±13.67% ±18.23% ±23.80% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=40 requests=1000 -0.97 % ±13.76% ±18.31% ±23.86% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=40 requests=5000 11.44 % ±14.36% ±19.12% ±24.92% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=100 requests=100 2.65 % ±13.61% ±18.38% ±24.49% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=100 requests=1000 -1.82 % ±12.70% ±17.12% ±22.73% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=100 requests=5000 2.22 % ±12.83% ±17.29% ±22.95% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=10 requests=100 -3.94 % ±12.72% ±17.13% ±22.72% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=10 requests=1000 10.01 % ±14.96% ±20.16% ±26.77% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=10 requests=5000 0.31 % ±15.57% ±20.99% ±27.86% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=1 requests=100 -6.24 % ±13.18% ±17.78% ±23.62% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=1 requests=1000 1.96 % ±12.18% ±16.42% ±21.80% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=1 requests=5000 -4.24 % ±12.69% ±17.13% ±22.79% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=200 requests=100 -0.02 % ±12.99% ±17.50% ±23.20% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=200 requests=1000 -4.55 % ±12.13% ±16.34% ±21.70% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=200 requests=5000 0.17 % ±14.07% ±18.97% ±25.21% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=20 requests=100 11.79 % ±12.68% ±17.13% ±22.82% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=20 requests=1000 -0.57 % ±13.04% ±17.59% ±23.37% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=20 requests=5000 4.28 % ±12.77% ±17.32% ±23.23% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=40 requests=100 -0.09 % ±14.01% ±18.87% ±25.02% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=40 requests=1000 -5.79 % ±13.18% ±17.77% ±23.59% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=40 requests=5000 -6.98 % ±12.10% ±16.32% ±21.69% Be aware that when doing many comparisons the risk of a false-positive result increases. In this case there are 58 comparisons, you can thus expect the following amount of false-positive results: 2.90 false positives, when considering a 5% risk acceptance (*, **, ***), 0.58 false positives, when considering a 1% risk acceptance (**, ***), 0.06 false positives, when considering a 0.1% risk acceptance (***) ++ mv output110121-044155.csv /w/bnch-comp confidence improvement accuracy (*) (**) (***) http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=100 requests=100 -2.70 % ±9.61% ±12.79% ±16.64% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=100 requests=1000 -4.85 % ±10.39% ±13.82% ±18.00% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=100 requests=5000 -0.87 % ±9.96% ±13.25% ±17.24% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=10 requests=100 -2.19 % ±9.99% ±13.29% ±17.29% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=10 requests=1000 3.52 % ±9.29% ±12.35% ±16.08% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=10 requests=5000 -5.22 % ±8.91% ±11.85% ±15.42% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=1 requests=100 1.22 % ±10.64% ±14.16% ±18.44% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=1 requests=1000 5.16 % ±9.37% ±12.47% ±16.24% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=1 requests=5000 -2.09 % ±9.71% ±12.92% ±16.81% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=200 requests=100 0.55 % ±9.83% ±13.08% ±17.03% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=200 requests=1000 1.41 % ±10.18% ±13.54% ±17.62% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=200 requests=5000 -4.72 % ±10.60% ±14.10% ±18.35% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=20 requests=100 8.76 % ±10.09% ±13.42% ±17.47% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=20 requests=1000 -7.23 % ±10.07% ±13.41% ±17.47% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=20 requests=5000 1.36 % ±10.19% ±13.55% ±17.64% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=40 requests=100 -6.32 % ±10.05% ±13.38% ±17.43% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=40 requests=1000 -1.56 % ±10.79% ±14.36% ±18.69% http2/compat.js duration=5 benchmarker='test-double-http2' clients=2 streams=40 requests=5000 2.06 % ±10.40% ±13.84% ±18.02% http2/headers.js nheaders=0 n=1000 -0.50 % ±4.52% ±6.02% ±7.86% http2/headers.js nheaders=1000 n=1000 -0.83 % ±2.91% ±3.87% ±5.04% http2/headers.js nheaders=100 n=1000 -0.07 % ±4.56% ±6.07% ±7.90% http2/headers.js nheaders=10 n=1000 -1.27 % ±2.06% ±2.75% ±3.58% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=100 requests=100 7.02 % ±13.69% ±18.23% ±23.76% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=100 requests=1000 -2.62 % ±13.17% ±17.53% ±22.84% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=100 requests=5000 4.61 % ±15.12% ±20.12% ±26.19% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=10 requests=100 -2.68 % ±12.83% ±17.07% ±22.21% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=10 requests=1000 8.76 % ±13.75% ±18.30% ±23.83% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=10 requests=5000 -6.43 % ±12.76% ±16.97% ±22.10% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=1 requests=100 1.68 % ±12.68% ±16.88% ±21.98% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=1 requests=1000 -4.64 % ±12.78% ±17.03% ±22.22% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=1 requests=5000 7.55 % ±15.70% ±20.91% ±27.27% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=200 requests=100 -0.99 % ±11.82% ±15.72% ±20.47% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=200 requests=1000 9.45 % ±11.50% ±15.31% ±19.92% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=200 requests=5000 -11.39 % ±12.12% ±16.14% ±21.02% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=20 requests=100 13.60 % ±14.58% ±19.40% ±25.26% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=20 requests=1000 7.43 % ±14.09% ±18.77% ±24.49% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=20 requests=5000 -0.31 % ±14.73% ±19.60% ±25.52% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=40 requests=100 -7.11 % ±13.67% ±18.23% ±23.80% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=40 requests=1000 -0.97 % ±13.76% ±18.31% ±23.86% http2/respond-with-fd.js duration=5 benchmarker='test-double-http2' clients=2 streams=40 requests=5000 11.44 % ±14.36% ±19.12% ±24.92% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=100 requests=100 2.65 % ±13.61% ±18.38% ±24.49% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=100 requests=1000 -1.82 % ±12.70% ±17.12% ±22.73% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=100 requests=5000 2.22 % ±12.83% ±17.29% ±22.95% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=10 requests=100 -3.94 % ±12.72% ±17.13% ±22.72% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=10 requests=1000 10.01 % ±14.96% ±20.16% ±26.77% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=10 requests=5000 0.31 % ±15.57% ±20.99% ±27.86% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=1 requests=100 -6.24 % ±13.18% ±17.78% ±23.62% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=1 requests=1000 1.96 % ±12.18% ±16.42% ±21.80% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=1 requests=5000 -4.24 % ±12.69% ±17.13% ±22.79% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=200 requests=100 -0.02 % ±12.99% ±17.50% ±23.20% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=200 requests=1000 -4.55 % ±12.13% ±16.34% ±21.70% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=200 requests=5000 0.17 % ±14.07% ±18.97% ±25.21% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=20 requests=100 11.79 % ±12.68% ±17.13% ±22.82% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=20 requests=1000 -0.57 % ±13.04% ±17.59% ±23.37% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=20 requests=5000 4.28 % ±12.77% ±17.32% ±23.23% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=40 requests=100 -0.09 % ±14.01% ±18.87% ±25.02% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=40 requests=1000 -5.79 % ±13.18% ±17.77% ±23.59% http2/simple.js duration=5 benchmarker='test-double-http2' clients=2 streams=40 requests=5000 -6.98 % ±12.10% ±16.32% ±21.69% Be aware that when doing many comparisons the risk of a false-positive result increases. In this case there are 58 comparisons, you can thus expect the following amount of false-positive results: 2.90 false positives, when considering a 5% risk acceptance (*, **, ***), 0.58 false positives, when considering a 1% risk acceptance (**, ***), 0.06 false positives, when considering a 0.1% risk acceptance (***) ++ mv output110121-044155.csv /w/bnch-comp 

@TrottTrott closed this Jan 11, 2021
@TrottTrott deleted the revert-it branch September 25, 2022 17:07
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

http2Issues or PRs related to the http2 subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@Trott@aduh95@nodejs-github-bot