Skip to content

Conversation

@tniessen
Copy link
Member

Preamble: This is a huge change and I will do my best to help with reviewing it. There might still be dozens of places that need some work, but so far, everything seems to be working nicely. I summarized the motivation behind this change in #15113 (comment). There are also lots of possible discussions around this, e.g. whether key derivation should consume / produce key objects etc.


This commit makes multiple important changes:

  1. A new key object API is introduced. The KeyObject class itself is
    not exposed to users, instead, several new APIs can be used to
    construct key objects: createSecretKey, createPrivateKey and
    createPublicKey. The new API also allows to convert between
    different key formats, and even though the API itself is not
    compatible to the WebCrypto standard in any way, it makes
    interoperability much simpler.

  2. Key objects can be used instead of the raw key material in all
    relevant crypto APIs.

  3. The handling of asymmetric keys has been unified and greatly
    improved. Node.js now fully supports both PEM-encoded and
    DER-encoded public and private keys.

  4. Conversions between buffers and strings have been moved to native
    code for sensitive data such as symmetric keys due to security
    considerations such as zeroing temporary buffers.

  5. For compatibility with older versions of the crypto API, this
    change allows to specify Buffers and strings as the "passphrase"
    option when reading or writing an encoded key. Note that this
    can result in unexpected behavior if the password contains a
    null byte.


cc @nodejs/crypto @nodejs/security-wg @nodejs/security

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@tniessentniessen added the semver-minor PRs that contain new features and should be released in the next minor version. label Nov 7, 2018
@nodejs-github-botnodejs-github-bot added the lib / src Issues and PRs related to general changes in the lib or src directory. label Nov 7, 2018
@tniessentniessen added crypto Issues and PRs related to the crypto subsystem. c++ Issues and PRs that require attention from people who are familiar with C++. labels Nov 7, 2018
Copy link
Contributor

@refackrefack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First pass, looks good.
Some guideline nits.

@TimothyGu
Copy link
Member

The new API also allows to convert between different key formats, and even though the API itself is not compatible to the WebCrypto standard in any way, it makes interoperability much simpler.

What are the differences between this new API and the API exposed through web crypto? Could we realistically change this API to be compatible?

@tniessen
Copy link
MemberAuthor

tniessen commented Nov 10, 2018

What are the differences between this new API and the API exposed through web crypto? Could we realistically change this API to be compatible?

WebCrypto has a very different design. I originally called the new API CryptoKey but changed it to KeyObject to avoid confusion with the WebCrypto CryptoKey class. Some of the key differences:

  • CryptoKey objects can be exported as raw/SPKI/PKCS#8 only and that is done using a separate function, exportKey, not using an instance method. Our options are much more complex, e.g. to support encrypted keys, and I think it makes sense for this to be an instance method.

  • The type property is actually compatible to the getType() function of the KeyObject class, and if we decide to implement @addaleax suggestion and turn it into a property, it will match the WebCrypto spec. However, CryptoKey instances have three additional properties that would have a huge impact on the way people can use key objects:

    readonly attribute boolean extractable; readonly attribute object algorithm; readonly attribute object usages; 

    We could implement them, but they don't make sense unless we adopt the whole WebCrypto API in my opinion, and as I said before, I don't think we should do that.

  • Symmetric CryptoKey objects don't have a property describing the size of the underlying key, and the only way to get the type of an asymmetric key is to parse the algorithm identifier associated with the key.


Rebased, old HEAD was 6c92496a7cf304785162ad5cd37632cea916b22b.

@tniessentniessenforce-pushed the crypto-add-key-objects branch from 6c92496 to 29e1fe7CompareNovember 12, 2018 17:20
Copy link
Contributor

@sam-githubsam-github left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like the direction, I think its a good API. Left some comments, mostly on the API. The C++ is a lot to read through, sorry, I ran out of time. I would prefer this not be experimental. I assume it doesn't intentionally break current APIs, and it also isn't particularly complex in terms of its API, so its hard to see it generating much comment or controversy or need to change. Would be great if it can go into 11.x, get some mileage there, and be stable in 12.xx. I doubt it can make it into 10.x, its a lot of churn.

@jasnell
Copy link
Member

Definite +1 on this moving forward. Code generally looking good but won't sign off until it's further along.

@tniessentniessenforce-pushed the crypto-add-key-objects branch from c115514 to e68c257CompareNovember 19, 2018 20:53
@ChALkeR
Copy link
Member

/cc @joepie91

@tniessen
Copy link
MemberAuthor

One question has come up multiple times: Should the key object API permit X509 certificates when constructing public keys? Personally, I think it makes more sense to provide an API for certificates that allows to extract the public key as a key object because a certificate is not the same as an asymmetric key, but others might feel differently.

cc @nodejs/crypto

This commit makes multiple important changes: 1. A new key object API is introduced. The KeyObject class itself is not exposed to users, instead, several new APIs can be used to construct key objects: createSecretKey, createPrivateKey and createPublicKey. The new API also allows to convert between different key formats, and even though the API itself is not compatible to the WebCrypto standard in any way, it makes interoperability much simpler. 2. Key objects can be used instead of the raw key material in all relevant crypto APIs. 3. The handling of asymmetric keys has been unified and greatly improved. Node.js now fully supports both PEM-encoded and DER-encoded public and private keys. 4. Conversions between buffers and strings have been moved to native code for sensitive data such as symmetric keys due to security considerations such as zeroing temporary buffers. 5. For compatibility with older versions of the crypto API, this change allows to specify Buffers and strings as the "passphrase" option when reading or writing an encoded key. Note that this can result in unexpected behavior if the password contains a null byte.
@MylesBorinsMylesBorins mentioned this pull request Dec 25, 2018
MylesBorins pushed a commit that referenced this pull request Dec 26, 2018
This commit makes multiple important changes: 1. A new key object API is introduced. The KeyObject class itself is not exposed to users, instead, several new APIs can be used to construct key objects: createSecretKey, createPrivateKey and createPublicKey. The new API also allows to convert between different key formats, and even though the API itself is not compatible to the WebCrypto standard in any way, it makes interoperability much simpler. 2. Key objects can be used instead of the raw key material in all relevant crypto APIs. 3. The handling of asymmetric keys has been unified and greatly improved. Node.js now fully supports both PEM-encoded and DER-encoded public and private keys. 4. Conversions between buffers and strings have been moved to native code for sensitive data such as symmetric keys due to security considerations such as zeroing temporary buffers. 5. For compatibility with older versions of the crypto API, this change allows to specify Buffers and strings as the "passphrase" option when reading or writing an encoded key. Note that this can result in unexpected behavior if the password contains a null byte. PR-URL: #24234 Reviewed-By: Refael Ackermann <[email protected]> Reviewed-By: James M Snell <[email protected]>
MylesBorins pushed a commit that referenced this pull request Dec 26, 2018
PR-URL: #24234 Reviewed-By: Refael Ackermann <[email protected]> Reviewed-By: James M Snell <[email protected]>
MylesBorins added a commit that referenced this pull request Dec 26, 2018
Notable Changes: * cli: - add --max-http-header-size flag (cjihrig) #24811 * crypto: - always accept certificates as public keys (Tobias Nießen) #24234 - add key object API (Tobias Nießen) [#24234](#24234) - update root certificates (Sam Roberts) #25113 * deps: - upgrade to libuv 1.24.1 (cjihrig) #25078 - upgrade npm to 6.5.0 (Audrey Eschright) #24734 * http: - add maxHeaderSize property (cjihrig) #24860 PR-URL: #25175
MylesBorins added a commit that referenced this pull request Dec 26, 2018
Notable Changes: * cli: - add --max-http-header-size flag (cjihrig) #24811 * crypto: - always accept certificates as public keys (Tobias Nießen) #24234 - add key object API (Tobias Nießen) [#24234](#24234) - update root certificates (Sam Roberts) #25113 * deps: - upgrade to libuv 1.24.1 (cjihrig) #25078 - upgrade npm to 6.5.0 (Audrey Eschright) #24734 * http: - add maxHeaderSize property (cjihrig) #24860 PR-URL: #25175
cjihrig added a commit to cjihrig/node that referenced this pull request Dec 26, 2018
During the time between nodejs#24234 being opened and it landing, a V8 update occurred that deprecated several APIs. This commit fixes the following compiler warnings: ../src/node_crypto.cc:3342:11: warning: 'Set' is deprecated: Use maybe version ../src/node_crypto.cc:3345:13: warning: 'GetFunction' is deprecated: Use maybe version PR-URL: nodejs#25205 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Daniel Bevenius <[email protected]>
MylesBorins added a commit that referenced this pull request Dec 26, 2018
Notable Changes: * cli: - add --max-http-header-size flag (cjihrig) #24811 * crypto: - always accept certificates as public keys (Tobias Nießen) #24234 - add key object API (Tobias Nießen) [#24234](#24234) - update root certificates (Sam Roberts) #25113 * deps: - upgrade to libuv 1.24.1 (cjihrig) #25078 - upgrade npm to 6.5.0 (Audrey Eschright) #24734 * http: - add maxHeaderSize property (cjihrig) #24860 PR-URL: #25175
targos pushed a commit that referenced this pull request Jan 1, 2019
During the time between #24234 being opened and it landing, a V8 update occurred that deprecated several APIs. This commit fixes the following compiler warnings: ../src/node_crypto.cc:3342:11: warning: 'Set' is deprecated: Use maybe version ../src/node_crypto.cc:3345:13: warning: 'GetFunction' is deprecated: Use maybe version PR-URL: #25205 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Daniel Bevenius <[email protected]>
refack pushed a commit to refack/node that referenced this pull request Jan 14, 2019
This commit makes multiple important changes: 1. A new key object API is introduced. The KeyObject class itself is not exposed to users, instead, several new APIs can be used to construct key objects: createSecretKey, createPrivateKey and createPublicKey. The new API also allows to convert between different key formats, and even though the API itself is not compatible to the WebCrypto standard in any way, it makes interoperability much simpler. 2. Key objects can be used instead of the raw key material in all relevant crypto APIs. 3. The handling of asymmetric keys has been unified and greatly improved. Node.js now fully supports both PEM-encoded and DER-encoded public and private keys. 4. Conversions between buffers and strings have been moved to native code for sensitive data such as symmetric keys due to security considerations such as zeroing temporary buffers. 5. For compatibility with older versions of the crypto API, this change allows to specify Buffers and strings as the "passphrase" option when reading or writing an encoded key. Note that this can result in unexpected behavior if the password contains a null byte. PR-URL: nodejs#24234 Reviewed-By: Refael Ackermann <[email protected]> Reviewed-By: James M Snell <[email protected]>
refack pushed a commit to refack/node that referenced this pull request Jan 14, 2019
PR-URL: nodejs#24234 Reviewed-By: Refael Ackermann <[email protected]> Reviewed-By: James M Snell <[email protected]>
refack pushed a commit to refack/node that referenced this pull request Jan 14, 2019
During the time between nodejs#24234 being opened and it landing, a V8 update occurred that deprecated several APIs. This commit fixes the following compiler warnings: ../src/node_crypto.cc:3342:11: warning: 'Set' is deprecated: Use maybe version ../src/node_crypto.cc:3345:13: warning: 'GetFunction' is deprecated: Use maybe version PR-URL: nodejs#25205 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Daniel Bevenius <[email protected]>
refack pushed a commit to refack/node that referenced this pull request Jan 14, 2019
Notable Changes: * cli: - add --max-http-header-size flag (cjihrig) nodejs#24811 * crypto: - always accept certificates as public keys (Tobias Nießen) nodejs#24234 - add key object API (Tobias Nießen) [nodejs#24234](nodejs#24234) - update root certificates (Sam Roberts) nodejs#25113 * deps: - upgrade to libuv 1.24.1 (cjihrig) nodejs#25078 - upgrade npm to 6.5.0 (Audrey Eschright) nodejs#24734 * http: - add maxHeaderSize property (cjihrig) nodejs#24860 PR-URL: nodejs#25175
@parogaparoga mentioned this pull request Mar 2, 2019
4 tasks
paroga added a commit to paroga/node that referenced this pull request Mar 2, 2019
Expose the size of asymetric keys of crypto key object from the crypto module added in v11.6.0 (nodejs#24234)
BridgeAR pushed a commit to BridgeAR/node that referenced this pull request Mar 6, 2019
Expose the size of asymetric keys of crypto key object from the crypto module added in v11.6.0. PR-URL: nodejs#26387 Refs: nodejs#24234 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]>
BridgeAR pushed a commit to BridgeAR/node that referenced this pull request Mar 12, 2019
Expose the size of asymetric keys of crypto key object from the crypto module added in v11.6.0. PR-URL: nodejs#26387 Refs: nodejs#24234 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]>
@tniessentniessen mentioned this pull request Mar 29, 2019
16 tasks
@benbucksch
Copy link
Contributor

benbucksch commented May 20, 2020

The new API introduced here is extremely ambiguous and confusing.
We have a "KeyObject" and a "key object", which are not the same type.
We have KeyObject type, key object, key parameter, and key property. All different.
We have key.key.key. Not kidding.

tniessen added a commit to tniessen/node that referenced this pull request May 4, 2025
I added this class in 823d86c in 2018 when we did not yet use `std::optional`. The last uses were removed in 5b9bf39, so remove it. Refs: nodejs#24234 Refs: nodejs#55368
@tniessentniessen mentioned this pull request May 4, 2025
nodejs-github-bot pushed a commit that referenced this pull request May 6, 2025
I added this class in 823d86c in 2018 when we did not yet use `std::optional`. The last uses were removed in 5b9bf39, so remove it. Refs: #24234 Refs: #55368 PR-URL: #58168 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Juan José Arboleda <[email protected]> Reviewed-By: Gerhard Stöbich <[email protected]> Reviewed-By: Daeyeon Jeong <[email protected]> Reviewed-By: Marco Ippolito <[email protected]> Reviewed-By: Luigi Pinca <[email protected]>
targos pushed a commit that referenced this pull request May 16, 2025
I added this class in 823d86c in 2018 when we did not yet use `std::optional`. The last uses were removed in 5b9bf39, so remove it. Refs: #24234 Refs: #55368 PR-URL: #58168 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Juan José Arboleda <[email protected]> Reviewed-By: Gerhard Stöbich <[email protected]> Reviewed-By: Daeyeon Jeong <[email protected]> Reviewed-By: Marco Ippolito <[email protected]> Reviewed-By: Luigi Pinca <[email protected]>
aduh95 pushed a commit that referenced this pull request Jun 10, 2025
I added this class in 823d86c in 2018 when we did not yet use `std::optional`. The last uses were removed in 5b9bf39, so remove it. Refs: #24234 Refs: #55368 PR-URL: #58168 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Juan José Arboleda <[email protected]> Reviewed-By: Gerhard Stöbich <[email protected]> Reviewed-By: Daeyeon Jeong <[email protected]> Reviewed-By: Marco Ippolito <[email protected]> Reviewed-By: Luigi Pinca <[email protected]>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++Issues and PRs that require attention from people who are familiar with C++.cryptoIssues and PRs related to the crypto subsystem.lib / srcIssues and PRs related to general changes in the lib or src directory.notable-changePRs with changes that should be highlighted in changelogs.semver-minorPRs that contain new features and should be released in the next minor version.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

11 participants

@tniessen@nodejs-github-bot@TimothyGu@jasnell@ChALkeR@benbucksch@sam-github@mcollina@refack@addaleax@vsemozhetbyt