Skip to content

Conversation

@legendecas
Copy link
Member

@legendecaslegendecas commented Apr 23, 2023

src: remove aliased buffer weak callback
An AliasedBuffer can be nested in a BaseObject and their weak callbacks
invoke order are not guaranteed. Prevent aliased buffer from being
accessed in its weak callback as its validness can be checked with the
emptiness of the persistent handle.

src: make realm binding data store weak
The binding data must be weak so that it won't keep the realm reachable
from strong GC roots indefinitely. The wrapper object of binding data
should be referenced from JavaScript, thus the binding data should be
reachable throughout the lifetime of the realm.

Fixes#47353

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/realm
  • @nodejs/startup
  • @nodejs/url

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Apr 23, 2023
@legendecaslegendecas added the realm Issues and PRs related to the ShadowRealm API and node::Realm label Apr 23, 2023
@legendecaslegendecas marked this pull request as draft April 23, 2023 17:34
// This is necessary to avoid cleaning up base objects before their scheduled
// weak callbacks are invoked, which can lead to accessing to v8 apis during
// the first pass of the weak callback.
realm->env()->SetImmediate([realm](Environment* env){delete realm});
Copy link
Member

@joyeecheungjoyeecheungApr 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just remembered that technically shadow realms can outlive the environment...(which is why I added the if (env_ != nullptr) branch in the destructor of shadow realms, I think I ran into that before)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is going to lead to crashes if the ream has access to any libuv handles within it. The problem is that any handles could emit data and ultimately call into JS.

IMHO the mechanism needs to be more sophisticated and we should track any native resource that the realm can spawn and let the ream be collected only after they have all been disposed.

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the ideas. Also, I noticed that the ASan build is failing for the URL binding data. I'm going to see how to address the failure too.

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added a cleanup for ShadowRealms when the environment is being freed. This ensures that no shadow realms can outlive the lifetime of the environment. Similar to BaseObjects, no BaseObjects can outlive their creation realm (https://github.com/nodejs/node/blob/main/src/node_realm.cc#L27).

As for the ASan failure of the URL binding data, I've removed the aliased buffer's weak callback as the validness of the aliased buffer can be checked with the emptiness of the persistent handle.

Would you mind taking a look again? Thank you!

@legendecaslegendecasforce-pushed the shadow-realm/binding-data branch 2 times, most recently from 2f14395 to eadbf52CompareMay 8, 2023 17:32
@legendecaslegendecas marked this pull request as ready for review May 8, 2023 17:42
Copy link
Member

@mcollinamcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@legendecaslegendecas added the request-ci Add this label to start a Jenkins CI on a PR. label May 9, 2023
@github-actionsgithub-actionsbot removed the request-ci Add this label to start a Jenkins CI on a PR. label May 9, 2023
@nodejs-github-bot
Copy link
Collaborator

@legendecas
Copy link
MemberAuthor

@joyeecheung would you mind taking a look at this again? Thank you! <3

}

for (auto realm : shadow_realms_){
realm->OnEnvironmentDestruct();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a test that creates a bunch of shadow realms and then generates a heap snapshot? I think that's how I ran into the shadow realms outliving shadow realms issue and is why this is necessary.

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a new test test/pummel/test-heapdump-shadow-realm.js.

@legendecaslegendecas added the request-ci Add this label to start a Jenkins CI on a PR. label Jun 8, 2023
@github-actionsgithub-actionsbot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jun 8, 2023
@nodejs-github-bot
Copy link
Collaborator

An AliasedBuffer can be nested in a BaseObject and their weak callbacks invoke order are not guaranteed. Prevent aliased buffer from being accessed in its weak callback as its validness can be checked with the emptiness of the persistent handle.
The binding data must be weak so that it won't keep the realm reachable from strong GC roots indefinitely. The wrapper object of binding data should be referenced from JavaScript, thus the binding data should be reachable throughout the lifetime of the realm.
@legendecaslegendecasforce-pushed the shadow-realm/binding-data branch from 46ce6c0 to 952cabcCompareJune 13, 2023 09:46
@legendecaslegendecas added the request-ci Add this label to start a Jenkins CI on a PR. label Jun 13, 2023
@github-actionsgithub-actionsbot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jun 13, 2023
@legendecas
Copy link
MemberAuthor

Rebased on the tip of the main branch to address CI coverage reports.

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@legendecaslegendecas added commit-queue Add this label to land a pull request using GitHub Actions. commit-queue-rebase Add this label to allow the Commit Queue to land a PR in several commits. labels Jun 14, 2023
@nodejs-github-botnodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Jun 14, 2023
@nodejs-github-bot
Copy link
Collaborator

Landed in 8bc6e19...ac0853c

nodejs-github-bot pushed a commit that referenced this pull request Jun 14, 2023
An AliasedBuffer can be nested in a BaseObject and their weak callbacks invoke order are not guaranteed. Prevent aliased buffer from being accessed in its weak callback as its validness can be checked with the emptiness of the persistent handle. PR-URL: #47688 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Joyee Cheung <[email protected]>
nodejs-github-bot pushed a commit that referenced this pull request Jun 14, 2023
The binding data must be weak so that it won't keep the realm reachable from strong GC roots indefinitely. The wrapper object of binding data should be referenced from JavaScript, thus the binding data should be reachable throughout the lifetime of the realm. PR-URL: #47688 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Joyee Cheung <[email protected]>
@legendecaslegendecas deleted the shadow-realm/binding-data branch June 14, 2023 02:28
RafaelGSS pushed a commit that referenced this pull request Jul 3, 2023
An AliasedBuffer can be nested in a BaseObject and their weak callbacks invoke order are not guaranteed. Prevent aliased buffer from being accessed in its weak callback as its validness can be checked with the emptiness of the persistent handle. PR-URL: #47688 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Joyee Cheung <[email protected]>
RafaelGSS pushed a commit that referenced this pull request Jul 3, 2023
The binding data must be weak so that it won't keep the realm reachable from strong GC roots indefinitely. The wrapper object of binding data should be referenced from JavaScript, thus the binding data should be reachable throughout the lifetime of the realm. PR-URL: #47688 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Joyee Cheung <[email protected]>
@RafaelGSSRafaelGSS mentioned this pull request Jul 3, 2023
Ceres6 pushed a commit to Ceres6/node that referenced this pull request Aug 14, 2023
An AliasedBuffer can be nested in a BaseObject and their weak callbacks invoke order are not guaranteed. Prevent aliased buffer from being accessed in its weak callback as its validness can be checked with the emptiness of the persistent handle. PR-URL: nodejs#47688 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Joyee Cheung <[email protected]>
Ceres6 pushed a commit to Ceres6/node that referenced this pull request Aug 14, 2023
The binding data must be weak so that it won't keep the realm reachable from strong GC roots indefinitely. The wrapper object of binding data should be referenced from JavaScript, thus the binding data should be reachable throughout the lifetime of the realm. PR-URL: nodejs#47688 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Joyee Cheung <[email protected]>
Ceres6 pushed a commit to Ceres6/node that referenced this pull request Aug 14, 2023
An AliasedBuffer can be nested in a BaseObject and their weak callbacks invoke order are not guaranteed. Prevent aliased buffer from being accessed in its weak callback as its validness can be checked with the emptiness of the persistent handle. PR-URL: nodejs#47688 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Joyee Cheung <[email protected]>
Ceres6 pushed a commit to Ceres6/node that referenced this pull request Aug 14, 2023
The binding data must be weak so that it won't keep the realm reachable from strong GC roots indefinitely. The wrapper object of binding data should be referenced from JavaScript, thus the binding data should be reachable throughout the lifetime of the realm. PR-URL: nodejs#47688 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Joyee Cheung <[email protected]>
@ruyadorno
Copy link
Member

These commits do not land cleanly on v18.x-staging and will need manual backport in case we want this to land on v18.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++Issues and PRs that require attention from people who are familiar with C++.commit-queue-rebaseAdd this label to allow the Commit Queue to land a PR in several commits.lib / srcIssues and PRs related to general changes in the lib or src directory.needs-ciPRs that need a full CI run.realmIssues and PRs related to the ShadowRealm API and node::Realm

Projects

None yet

Development

Successfully merging this pull request may close these issues.

memory leak in shadow realms

6 participants

@legendecas@nodejs-github-bot@ruyadorno@mcollina@jasnell@joyeecheung