buffer: optimize decoding wrapped base64 data#12146
Closed
Uh oh!
There was an error while loading. Please reload this page.
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
nodejs-github-bot added buffer 
ContributorAuthor
aqrln commented Mar 31, 2017
I haven't run the whole benchmark suite yet, only symlinked the exiting and the new benchmarks into a new directory locally: |
This was referenced Mar 31, 2017
Contributor
Fishrock123 commented Mar 31, 2017
jasnell reviewed Mar 31, 2017
Member
There was a problem hiding this comment.
minor nit but this can be simplified just a bit by doing...
const line = 'abcd'.repeat(charsPerLine / 4) + '\n'; const buffer = Buffer.alloc(bytesCount, line); ContributorAuthor
There was a problem hiding this comment.
Done. Thanks for the suggestion!
The fast base64 decoder used to switch to the slow one permanently when it saw a whitespace or other garbage character. Since the most common situation such characters may be encountered in is line-wrapped base64 data, a more profitable strategy is to decode a single 24-bit group with the slow decoder and then continue running the fast algorithm. Refs: nodejs#12114
ContributorAuthor
aqrln commented Mar 31, 2017
Rebased to incorporate changes from #11995. |
ContributorAuthor
aqrln commented Apr 4, 2017
Can I get a fresh CI run? |
Contributor
trevnorris commented Apr 4, 2017
trevnorris approved these changes Apr 4, 2017
Contributor
trevnorris left a comment
trevnorris left a comment There was a problem hiding this comment.
not sure I'm qualified to sign off on this but LGTM
ContributorAuthor
/cc @bnoordhuis (based on Git history) |
jasnell approved these changes Apr 4, 2017
Member
jasnell left a comment
jasnell left a comment There was a problem hiding this comment.
This LGTM but it would be really nice to have some cctests for this header file
Member
jasnell commented Apr 4, 2017
(to be clear, the cctest can be added separately :-) ...) |
jasnell pushed a commit that referenced this pull request Apr 4, 2017
The fast base64 decoder used to switch to the slow one permanently when it saw a whitespace or other garbage character. Since the most common situation such characters may be encountered in is line-wrapped base64 data, a more profitable strategy is to decode a single 24-bit group with the slow decoder and then continue running the fast algorithm. PR-URL: #12146 Ref: #12114 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Trevor Norris <[email protected]> Reviewed-By: James M Snell <[email protected]>
Member
jasnell commented Apr 4, 2017
Landed in e77a83f |
aqrln added a commit to aqrln/node that referenced this pull request Apr 5, 2017
This commit adds C++ tests for `base64_encode()` and `base64_decode()` functions defined in `base64.h`. The functionality is already being tested indirectly in JavaScript tests for Buffer, but it won't hurt to test the low-level functions too, especially given that they aren't only used in the internal Buffer implementation, Chrome inspector protocol support relies upon them too. Refs: nodejs#12146 (comment)
3 tasks
aqrln added a commit that referenced this pull request Apr 8, 2017
This commit adds C++ tests for `base64_encode()` and `base64_decode()` functions defined in `base64.h`. The functionality is already being tested indirectly in JavaScript tests for Buffer, but it won't hurt to test the low-level functions too, especially given that they aren't only used in the internal Buffer implementation, Chrome inspector protocol support relies upon them too. PR-URL: #12238 Refs: #12146 (comment) Reviewed-By: James M Snell <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Daniel Bevenius <[email protected]>
italoacasas pushed a commit to italoacasas/node that referenced this pull request Apr 10, 2017
The fast base64 decoder used to switch to the slow one permanently when it saw a whitespace or other garbage character. Since the most common situation such characters may be encountered in is line-wrapped base64 data, a more profitable strategy is to decode a single 24-bit group with the slow decoder and then continue running the fast algorithm. PR-URL: nodejs#12146 Ref: nodejs#12114 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Trevor Norris <[email protected]> Reviewed-By: James M Snell <[email protected]>
Contributor
MylesBorins commented Apr 18, 2017
should we backport? Assuming if so we should wait a bit |
MylesBorins added the baking-for-lts 
Member
addaleax commented Jun 13, 2017
It seems to be the root cause of #13657, so no. |
addaleax added dont-land-on-v4.x and removed baking-for-lts gibfahn pushed a commit that referenced this pull request Jun 18, 2017
This commit adds C++ tests for `base64_encode()` and `base64_decode()` functions defined in `base64.h`. The functionality is already being tested indirectly in JavaScript tests for Buffer, but it won't hurt to test the low-level functions too, especially given that they aren't only used in the internal Buffer implementation, Chrome inspector protocol support relies upon them too. PR-URL: #12238 Refs: #12146 (comment) Reviewed-By: James M Snell <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Daniel Bevenius <[email protected]>
gibfahn pushed a commit that referenced this pull request Jun 20, 2017
This commit adds C++ tests for `base64_encode()` and `base64_decode()` functions defined in `base64.h`. The functionality is already being tested indirectly in JavaScript tests for Buffer, but it won't hurt to test the low-level functions too, especially given that they aren't only used in the internal Buffer implementation, Chrome inspector protocol support relies upon them too. PR-URL: #12238 Refs: #12146 (comment) Reviewed-By: James M Snell <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Daniel Bevenius <[email protected]>
MylesBorins pushed a commit that referenced this pull request Jul 11, 2017
This commit adds C++ tests for `base64_encode()` and `base64_decode()` functions defined in `base64.h`. The functionality is already being tested indirectly in JavaScript tests for Buffer, but it won't hurt to test the low-level functions too, especially given that they aren't only used in the internal Buffer implementation, Chrome inspector protocol support relies upon them too. PR-URL: #12238 Refs: #12146 (comment) Reviewed-By: James M Snell <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Daniel Bevenius <[email protected]>
3 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The fast base64 decoder used to switch to the slow one permanently when
it saw a whitespace or other garbage character. Since the most common
situation such characters may be encountered in is line-wrapped base64
data, a more profitable strategy is to decode a single 24-bit group with
the slow decoder and then continue running the fast algorithm.
Refs: #12114
Checklist
make -j4 test(UNIX), orvcbuild test(Windows) passesAffected core subsystem(s)
buffer