Skip to content

Conversation

@addaleax
Copy link
Member

Checklist
  • make -j4 test (UNIX), or vcbuild test nosign (Windows) passes
  • tests and/or benchmarks are included
  • commit message follows commit guidelines
Affected core subsystem(s)

fs, module

Description of change

Reintroduce a realpath cache with the same mechanisms which existed before b488b19 (fs: optimize realpath using uv_fs_realpath()), but only for the synchronous version and with the cache being passed as a hidden option to make sure it is only used internally.

The cache is hidden from userland applications because it has been decided that fully reintroducing as part of the public API might stand in the way of future optimizations.

/cc @nodejs/fs @bzoz

@addaleaxaddaleax added fs Issues and PRs related to the fs subsystem / file system. module Issues and PRs related to the module subsystem. labels Aug 14, 2016
lib/fs.js Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be faster to just do cache.get() and check if it returned undefined?

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mscdex Maybe a bit, yes. I’ve updated the PR with only calls to cache.get().

@Fishrock123Fishrock123 added semver-minor PRs that contain new features and should be released in the next minor version. and removed semver-minor PRs that contain new features and should be released in the next minor version. labels Aug 14, 2016
@addaleax
Copy link
MemberAuthor

@jasnell
Copy link
Member

LGTM . There's Red in CI that looks unrelated.

@bzoz
Copy link
Contributor

bzoz commented Aug 16, 2016

LGTM. The CI failure is 99% unrelated, but still - one more run: https://ci.nodejs.org/job/node-test-pull-request/3692/

@jasnell
Copy link
Member

CITGM: https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/361/

Because this touches module, I'd like a bit more signoff from @nodejs/ctc

lib/module.js Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't this be a const?

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vkurchatkin Yes, makes sense. Done!

@jasnell
Copy link
Member

Marking this semver-minor given that it's technically additive. (I sure will be happy when we have all this fs module stuff settled)

@jasnelljasnell added the semver-minor PRs that contain new features and should be released in the next minor version. label Aug 18, 2016
@jasnell
Copy link
Member

ping @nodejs/ctc

@addaleax
Copy link
MemberAuthor

hmm… anybody from @nodejs/ctc @nodejs/collaborators up for more reviews/thoughts on this?

This is not really urgent, but having it land in the next couple of days would be nice because it’s in a similar situation as Rod’s #8277, and should get a follow-up PR moving the cache symbol to internal/fs.

@addaleax
Copy link
MemberAuthor

hmm… bump again? would anybody feel more comfortable signing off on this if it’s marked dont-land-on-v6.x?

lib/module.js Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be "an empty Map"?

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, thanks for noticing! Done!

Reintroduce a realpath cache with the same mechanisms which existed before b488b19 (`fs: optimize realpath using uv_fs_realpath()`), but only for the synchronous version and with the cache being passed as a hidden option to make sure it is only used internally. The cache is hidden from userland applications because it has been decided that fully reintroducing as part of the public API might stand in the way of future optimizations.
@addaleaxaddaleaxforce-pushed the fs-module-only-rpcache branch from c7efb61 to c9954f4CompareSeptember 10, 2016 10:20
@addaleax
Copy link
MemberAuthor

@jasnell
Copy link
Member

Still LGTM

@addaleax
Copy link
MemberAuthor

addaleax commented Sep 25, 2016

Okay, so, unless anybody objects or something else comes up, I’d like to land this sometime next week. Right now it looks like this doesn’t get any more review/feedback than @bzoz’ and @jasnell’s LGTMs.

If anybody would feel more comfortable with it, they can add a dont-land-on-v6.x label.

Edit: One more CI: https://ci.nodejs.org/job/node-test-commit/5375/

@addaleaxaddaleax self-assigned this Sep 30, 2016
@addaleax
Copy link
MemberAuthor

I’ll land this:

  • Two LGTMs
  • No objections or indications of further review
  • Last two CIs add up as green, and the last CI was green up to a build failure and unrelated flaky tests.

@addaleax
Copy link
MemberAuthor

Landed in c084287

@addaleaxaddaleax deleted the fs-module-only-rpcache branch September 30, 2016 14:11
addaleax added a commit that referenced this pull request Sep 30, 2016
Reintroduce a realpath cache with the same mechanisms which existed before b488b19 (`fs: optimize realpath using uv_fs_realpath()`), but only for the synchronous version and with the cache being passed as a hidden option to make sure it is only used internally. The cache is hidden from userland applications because it has been decided that fully reintroducing as part of the public API might stand in the way of future optimizations. PR-URL: #8100 Reviewed-By: Bartosz Sosnowski <[email protected]> Reviewed-By: James M Snell <[email protected]>
jasnell pushed a commit that referenced this pull request Oct 6, 2016
Reintroduce a realpath cache with the same mechanisms which existed before b488b19 (`fs: optimize realpath using uv_fs_realpath()`), but only for the synchronous version and with the cache being passed as a hidden option to make sure it is only used internally. The cache is hidden from userland applications because it has been decided that fully reintroducing as part of the public API might stand in the way of future optimizations. PR-URL: #8100 Reviewed-By: Bartosz Sosnowski <[email protected]> Reviewed-By: James M Snell <[email protected]>
Fishrock123 pushed a commit that referenced this pull request Oct 11, 2016
Reintroduce a realpath cache with the same mechanisms which existed before b488b19 (`fs: optimize realpath using uv_fs_realpath()`), but only for the synchronous version and with the cache being passed as a hidden option to make sure it is only used internally. The cache is hidden from userland applications because it has been decided that fully reintroducing as part of the public API might stand in the way of future optimizations. PR-URL: #8100 Reviewed-By: Bartosz Sosnowski <[email protected]> Reviewed-By: James M Snell <[email protected]>
Fishrock123 added a commit that referenced this pull request Oct 12, 2016
* fs: - `SyncWriteStream` now inherits from `Stream.Writable`. (Anna Henningsen) #8830 - Practically, this means that when stdio is piped to a file, stdout and stderr will still be `Writable` streams. - `fs.existsSync()` has been undeprecated. `fs.exists()` remains deprecated. (Dan Fabulich) #8364 * http: `http.request()` now accepts a `timeout` option. (Rene Weber) #8101 * module: The module loader now maintains its own realpath cache. (Anna Henningsen) #8100 * npm: Upgraded to 3.10.8 (Kat Marchán) #8706 * stream: `Duplex` streams now show proper `instanceof Stream.Writable`. (Anna Henningsen) #8834 * timers: Improved `setTimeout`/`Interval` performance by up to 22%. (Brian White) #8661 PR-URL: #9034
Fishrock123 added a commit that referenced this pull request Oct 12, 2016
* fs: - `SyncWriteStream` now inherits from `Stream.Writable`. (Anna Henningsen) #8830 - Practically, this means that when stdio is piped to a file, stdout and stderr will still be `Writable` streams. - `fs.existsSync()` has been undeprecated. `fs.exists()` remains deprecated. (Dan Fabulich) #8364 * http: `http.request()` now accepts a `timeout` option. (Rene Weber) #8101 * module: The module loader now maintains its own realpath cache. (Anna Henningsen) #8100 * npm: Upgraded to 3.10.8 (Kat Marchán) #8706 * stream: `Duplex` streams now show proper `instanceof Stream.Writable`. (Anna Henningsen) #8834 * timers: Improved `setTimeout`/`Interval` performance by up to 22%. (Brian White) #8661 PR-URL: #9034
imyller added a commit to imyller/meta-nodejs that referenced this pull request Oct 13, 2016
 * fs: - `SyncWriteStream` now inherits from `Stream.Writable`. (Anna Henningsen) nodejs/node#8830 - Practically, this means that when stdio is piped to a file, stdout and stderr will still be `Writable` streams. - `fs.existsSync()` has been undeprecated. `fs.exists()` remains deprecated. (Dan Fabulich) nodejs/node#8364 * http: `http.request()` now accepts a `timeout` option. (Rene Weber) nodejs/node#8101 * module: The module loader now maintains its own realpath cache. (Anna Henningsen) nodejs/node#8100 * npm: Upgraded to 3.10.8 (Kat Marchan) nodejs/node#8706 * stream: `Duplex` streams now show proper `instanceof Stream.Writable`. (Anna Henningsen) nodejs/node#8834 * timers: Improved `setTimeout`/`Interval` performance by up to 22%. (Brian White) nodejs/node#8661 Signed-off-by: Ilkka Myller <[email protected]>
imyller added a commit to imyller/meta-nodejs that referenced this pull request Oct 13, 2016
 * fs: - `SyncWriteStream` now inherits from `Stream.Writable`. (Anna Henningsen) nodejs/node#8830 - Practically, this means that when stdio is piped to a file, stdout and stderr will still be `Writable` streams. - `fs.existsSync()` has been undeprecated. `fs.exists()` remains deprecated. (Dan Fabulich) nodejs/node#8364 * http: `http.request()` now accepts a `timeout` option. (Rene Weber) nodejs/node#8101 * module: The module loader now maintains its own realpath cache. (Anna Henningsen) nodejs/node#8100 * npm: Upgraded to 3.10.8 (Kat Marchan) nodejs/node#8706 * stream: `Duplex` streams now show proper `instanceof Stream.Writable`. (Anna Henningsen) nodejs/node#8834 * timers: Improved `setTimeout`/`Interval` performance by up to 22%. (Brian White) nodejs/node#8661 Signed-off-by: Ilkka Myller <[email protected]>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fsIssues and PRs related to the fs subsystem / file system.moduleIssues and PRs related to the module subsystem.semver-minorPRs that contain new features and should be released in the next minor version.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants

@addaleax@jasnell@bzoz@mscdex@Fishrock123@cjihrig@vkurchatkin