stream: implement finished() for ReadableStream and WritableStream#46205

debadree25 · 2023-01-13T21:01:30Z

Implemented finished() for ReadableStreams and WritableStreams and added tests.

Refs: #39316

nodejs-github-bot · 2023-01-13T21:01:34Z

Review requested:

@nodejs/streams

lib/internal/streams/end-of-stream.js

test/parallel/test-stream-end-of-streams.js

test/parallel/test-stream-finished.js

debadree25 · 2023-01-14T09:43:18Z

So far now its not breaking any more tests hence opening the PR for review

ronag · 2023-01-14T09:49:45Z

lib/internal/webstreams/readablestream.js

+thrownewERR_INVALID_THIS('ReadableStream');
+returnthis[kState].streamClosed.promise;
+}
+


You can't add new public properties to web streams. That is not allowed by the spec.

ok updating this to private access?

Have updated to access using [kState]

lib/internal/streams/utils.js

mcollina

Can you please add some more tests for error conditions? You'd also need tests for Duplex too.

debadree25 · 2023-01-14T10:18:35Z

Can you please add some more tests for error conditions? You'd also need tests for Duplex too.

Understood trying to add the same!

debadree25 · 2023-01-14T11:20:11Z

Converting this to a draft again, since there seems to be some work left and it seems to be more complex than I anticipated 😅 but any comments on the general approach I am taking regarding adding promises is it a good enough path? @mcollina, @ronag

lib/internal/streams/end-of-stream.js

ronag · 2023-01-14T11:42:02Z

lib/internal/streams/end-of-stream.js

+PromisePrototypeThen(
+stream[kState].streamClosed,
+()=>callback.call(stream),
+(err)=>callback.call(stream,err)


You need to call the callback with a process.nextTick

understood updating

fixed this to properly use process.nextTick

ronag · 2023-01-14T11:58:36Z

lib/internal/webstreams/readablestream.js

 functionreadableStreamClose(stream){
 assert(stream[kState].state==='readable');
 stream[kState].state='closed';
+stream[kState].streamClosed?.resolve?.();


Can streamClosed be nully?

I think it can because initially when I tested without null checks it failed

Then I think there might be a bug somewhere. What does it mean when it's nully?

Okay trying to investigate

Fixed this, removed the null checks basically it was breaking tests where tee() was being used to construct streams

lib/internal/webstreams/readablestream.js

lib/internal/streams/utils.js

debadree25 · 2023-01-14T21:02:41Z

Have added a number of tests on the error conditions and promises on both readable & writable streams,

You'd also need tests for Duplex too.

Didn't get you here?

@mcollina

debadree25 · 2023-01-14T21:03:32Z

Reopening the PR again for review, it seems stable now

mcollina

lgtm

nodejs-github-bot · 2023-01-14T21:49:01Z

CI: https://ci.nodejs.org/job/node-test-pull-request/48988/

Refs: nodejs#39316

Refs: #46190 Refs: #46205 (comment) PR-URL: #46315 Reviewed-By: Robert Nagy <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Darshan Sen <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Anna Henningsen <[email protected]>

PR-URL: #46312 Refs: #46190 Refs: #46205 Reviewed-By: Antoine du Hamel <[email protected]>

Notable changes: * deps: * upgrade npm to 9.3.1 (npm team) #46242 * doc: * add parallelism note to os.cpus() (Colin Ihrig) #45895 * http: * join authorization headers (Marco Ippolito) #45982 * improved timeout defaults handling (Paolo Insogna) #45778 * stream: * implement finished() for ReadableStream and WritableStream (Debadree Chatterjee) #46205 PR-URL: #46396

Refs: #46205 PR-URL: #46403 Refs: #37354 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Robert Nagy <[email protected]>

PR-URL: nodejs#46312 Refs: nodejs#46190 Refs: nodejs#46205 Reviewed-By: Antoine du Hamel <[email protected]>

Refs: nodejs#46190 Refs: nodejs#46205 (comment) PR-URL: nodejs#46315 Reviewed-By: Robert Nagy <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Darshan Sen <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Anna Henningsen <[email protected]>

Refs: nodejs#46205 PR-URL: nodejs#46403 Refs: nodejs#37354 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Robert Nagy <[email protected]>

PR-URL: nodejs#46312 Refs: nodejs#46190 Refs: nodejs#46205 Reviewed-By: Antoine du Hamel <[email protected]>

Refs: nodejs#46190 Refs: nodejs#46205 (comment) PR-URL: nodejs#46315 Reviewed-By: Robert Nagy <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Darshan Sen <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Anna Henningsen <[email protected]>

Refs: nodejs#46205 PR-URL: nodejs#46403 Refs: nodejs#37354 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Robert Nagy <[email protected]>

PR-URL: nodejs#46312 Refs: nodejs#46190 Refs: nodejs#46205 Reviewed-By: Antoine du Hamel <[email protected]>

Refs: nodejs#46190 Refs: nodejs#46205 (comment) PR-URL: nodejs#46315 Reviewed-By: Robert Nagy <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Darshan Sen <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Anna Henningsen <[email protected]>

Refs: nodejs#46205 PR-URL: nodejs#46403 Refs: nodejs#37354 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Robert Nagy <[email protected]>

Refs: #46190 Refs: #46205 (comment) PR-URL: #46315 Reviewed-By: Robert Nagy <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Darshan Sen <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Anna Henningsen <[email protected]>

Refs: #46205 PR-URL: #46403 Refs: #37354 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Robert Nagy <[email protected]>

kanongil

While it is nice to be able to know when a stream has closed, I'm not a fan of this feature.

It will be cumbersome (impossible?) to implement in readable-stream for browser usage since it will need to rely on the public WebStreams API.

kanongil · 2023-04-13T11:16:04Z

lib/internal/streams/end-of-stream.js

+if(isReadableStream(stream)||isWritableStream(stream)){
+returneosWeb(stream,options,callback);
+}


These match non-node WebStreams – which I guess is fine, except that eosWeb() will throw a TypeError since the private property kIsClosedPromise will not exist on it. This will in turn mean that the callback is never called.

As in you want eosWeb to additionally have a check to throw TypeError ?

I don't know what node should do – probably still throw some kind of error.
Currently non-node WebStreams would just throw a TypeError from accessing the promise property on the undefinedkIsClosedPromise property here:
node/lib/internal/streams/end-of-stream.js
Line 289 in 4667b07
stream[kIsClosedPromise].promise,

PR-URL: nodejs#46312 Refs: nodejs#46190 Refs: nodejs#46205 Reviewed-By: Antoine du Hamel <[email protected]>

Refs: nodejs#46190 Refs: nodejs#46205 (comment) PR-URL: nodejs#46315 Reviewed-By: Robert Nagy <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Darshan Sen <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Anna Henningsen <[email protected]>

PR-URL: #46312 Backport-PR-URL: #46314 Refs: #46190 Refs: #46205 Reviewed-By: Antoine du Hamel <[email protected]>

Refs: #46190 Refs: #46205 (comment) PR-URL: #46315 Backport-PR-URL: #46314 Reviewed-By: Robert Nagy <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Darshan Sen <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Anna Henningsen <[email protected]>

PR-URL: nodejs#46312 Backport-PR-URL: nodejs#46314 Refs: nodejs#46190 Refs: nodejs#46205 Reviewed-By: Antoine du Hamel <[email protected]>

Refs: nodejs#46190 Refs: nodejs#46205 (comment) PR-URL: nodejs#46315 Backport-PR-URL: nodejs#46314 Reviewed-By: Robert Nagy <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Darshan Sen <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Anna Henningsen <[email protected]>

nodejs-github-bot added needs-ci PRs that need a full CI run. web streams labels Jan 13, 2023

RaisinTen reviewed Jan 14, 2023
View reviewed changes

lib/internal/streams/end-of-stream.js Outdated Show resolvedHide resolved
lib/internal/streams/end-of-stream.jsShow resolvedHide resolved
test/parallel/test-stream-end-of-streams.jsShow resolvedHide resolved
test/parallel/test-stream-finished.jsShow resolvedHide resolved

debadree25 force-pushed the ft/finished-webstreams branch from d99602f to e036f56Compare January 14, 2023 08:09

debadree25 marked this pull request as ready for review January 14, 2023 09:43

debadree25 changed the title ~~streams: implement finished() for webstreams~~stream: implement finished() for webstreamsJan 14, 2023

ronag requested changes Jan 14, 2023
View reviewed changes

mcollina requested changes Jan 14, 2023
View reviewed changes

debadree25 marked this pull request as draft January 14, 2023 11:18

ronag requested changes Jan 14, 2023
View reviewed changes

ronag reviewed Jan 14, 2023
View reviewed changes

ronag requested changes Jan 14, 2023
View reviewed changes

lib/internal/webstreams/readablestream.js Outdated Show resolvedHide resolved
lib/internal/streams/utils.js Outdated Show resolvedHide resolved

debadree25 marked this pull request as ready for review January 14, 2023 21:03

mcollina approved these changes Jan 14, 2023
View reviewed changes

mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Jan 14, 2023

github-actionsbot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jan 14, 2023

debadree25 changed the title ~~stream: implement finished() for webstreams~~stream: implement finished() for ReadableStream and WritableStreamJan 15, 2023

debadree25 added 6 commits January 15, 2023 14:41

stream: implement finished() for ReadableStream and WritableStream
fd2fa5a
Refs: nodejs#39316

fixup! restore tests
7938b4a

fixup! add non-stream check
f0b8c92

fixup! fix return
3937b22

fixup! remove public property
70630b3

fixup! use the promise and use process.nextTick
2d7ea18

ruyadorno pushed a commit that referenced this pull request Feb 1, 2023
doc: include webstreams in finished() and Duplex.from() parameters
996bac0
PR-URL: #46312 Refs: #46190 Refs: #46205 Reviewed-By: Antoine du Hamel <[email protected]>

debadree25 added a commit to debadree25/node that referenced this pull request Feb 27, 2023
doc: include webstreams in finished() and Duplex.from() parameters
8ae52bf
PR-URL: nodejs#46312 Refs: nodejs#46190 Refs: nodejs#46205 Reviewed-By: Antoine du Hamel <[email protected]>

debadree25 added a commit to debadree25/node that referenced this pull request Feb 27, 2023
doc: include webstreams in finished() and Duplex.from() parameters
edaee13
PR-URL: nodejs#46312 Refs: nodejs#46190 Refs: nodejs#46205 Reviewed-By: Antoine du Hamel <[email protected]>

debadree25 added a commit to debadree25/node that referenced this pull request Feb 27, 2023
doc: include webstreams in finished() and Duplex.from() parameters
7eb38e1
PR-URL: nodejs#46312 Refs: nodejs#46190 Refs: nodejs#46205 Reviewed-By: Antoine du Hamel <[email protected]>

kanongil reviewed Apr 13, 2023
View reviewed changes

wooorm mentioned this pull request Apr 20, 2023
does not work with Node's stream.pipeline() built-in unifiedjs/unified-stream#1
Closed
4 tasks

debadree25 added a commit to debadree25/node that referenced this pull request May 23, 2023
doc: include webstreams in finished() and Duplex.from() parameters
631afeb
PR-URL: nodejs#46312 Refs: nodejs#46190 Refs: nodejs#46205 Reviewed-By: Antoine du Hamel <[email protected]>

danielleadams pushed a commit that referenced this pull request May 29, 2023
doc: include webstreams in finished() and Duplex.from() parameters
402c1a3
PR-URL: #46312 Backport-PR-URL: #46314 Refs: #46190 Refs: #46205 Reviewed-By: Antoine du Hamel <[email protected]>

KhafraDev mentioned this pull request Apr 15, 2024
stream.finished docs are lacking #52544
Closed

Uh oh!

stream: implement finished() for ReadableStream and WritableStream#46205

stream: implement finished() for ReadableStream and WritableStream #46205

Uh oh!

Conversation

debadree25 commented Jan 13, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nodejs-github-bot commented Jan 13, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

debadree25 commented Jan 14, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mcollina left a comment

Choose a reason for hiding this comment

Uh oh!

debadree25 commented Jan 14, 2023

Uh oh!

debadree25 commented Jan 14, 2023

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

debadree25 commented Jan 14, 2023

Uh oh!

debadree25 commented Jan 14, 2023

Uh oh!

mcollina left a comment

Choose a reason for hiding this comment

Uh oh!

nodejs-github-bot commented Jan 14, 2023

Uh oh!

kanongil left a comment

Choose a reason for hiding this comment

Uh oh!

kanongilApr 13, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

debadree25 commented Jan 13, 2023•
edited
Loading

kanongilApr 13, 2023•
edited
Loading