fs: fix createReadStream(…,{end: n}) for non-seekable fds

[addaleax]

82bdf8f fixed an issue by silently modifying the start
option for the case when only end is passed, in order to perform
reads from a specified range in the file.

However, that approach does not work for non-seekable files, since
a numeric start option means that positioned reads will be used
to read data from the file.

This patch fixes that, and instead ends reading after a specified
size by adjusting the read buffer size.

This way we avoid re-introducing the bug that 82bdf8f fixed,
and align behaviour with the native file stream mechanism
introduced in #18936 as well.

Fixes: #19240
Refs: #18121

Checklist

• make -j4 test (UNIX), or vcbuild test (Windows) passes
• tests and/or benchmarks are included
• commit message follows commit guidelines

[addaleax]

Alternative to #19326 and #19328

CI: https://ci.nodejs.org/job/node-test-commit/16888/
CI: https://ci.nodejs.org/job/node-test-commit/16889/

[addaleax]

The changes to lib/fs.js land cleanly on v8.x and v6.x, but it looks like the test file parts would require explicit backporting (neighbouring line conflicts/different common tmpdir interface).

[addaleax]

@nodejs/build I think the original CI attempt here left the Windows tmpdirs in a bogus state… could you maybe refresh those dirs (make sure they can be rimraf’ed) and start one more CI for this?

[joaocgreis]

This PR creates a fifo on Windows, even if it does not use it. However, that fifo cannot be removed by tmpdir.refresh(), causing other tests to fail (I believe this boils down to an issue with libuv, I'm still looking into that). For now, can you make it so that the fifo is not even created for Windows?

[MoonBall]

There is probably a inconsistent behavior. The end option of fs.createReadStream('non-seekable-file',{end: 3 }) doesn't work as a normal file.

Assuming the content of a file is:

abcdefg 

If the file is a normal file, then fs.createReadStream('non-seekable-file',{end: 3 }) will get 'abcd'. If the file is non-seekable file and the current file position is 2, then fs. createReadStream('non-seekable-file',{end: 3 }) will get 'cdef'.

[addaleax]

If the file is non-seekable file and the current file position is 2

@MoonBall No; non-seekable files have no such concept as a “current position” (or a “content”). That’s the reason why this is failing in the first place.

For now, can you make it so that the fifo is not even created for Windows?

@joaocgreis Done!

CI: https://ci.nodejs.org/job/node-test-commit/16898/

[MoonBall]

@addaleax LGTM.
But there is a small change about the end option. Please see the code below.

const fs = require('fs'); const fd = fs.openSync('a.txt', 'r'); // a.txt's content is `123456789` const buf = Buffer.alloc(3); fs.read(fd, buf, 0, 2); const readable = fs.createReadStream(null,{fd: fd, end: 3 }); readable.on('data', data => console.log(data.toString())); 

Executing this code in current stable Node.js, the output is 3456789, but now the output is 3456.

[addaleax]

Executing this code in current stable Node.js, the output is 3456789, but now the output is 3456.

You’re right, that’s a change. I can’t really see how 3456789 would be an expected output if you explicitly pass end: 3, so I think that’s okay?

[jasnell]

Would need docs. Otherwise LGTM

[addaleax]

@jasnell This is a bugfix, it doesn’t introduce a new option but it rather makes a specific combination of existing ones work… what would you like to see in the documentation?

[jasnell]

oh right right... sorry :-) (I had mistakenly read fix in the title as add ... :-) ..)

[mcollina]

LGTM

[MoonBall]

You’re right, that’s a change. I can’t really see how 3456789 would be an expected output if you explicitly pass end: 3, so I think that’s okay?

that's okay.

[addaleax]

Fresh CI: https://ci.nodejs.org/job/node-test-commit/16968/

[addaleax]

Landed in 2e37618

[joaocgreis]

Just for reference, libuv fix for the issue I mentioned above: libuv/libuv#1774