Uh oh!
There was an error while loading. Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork 34k
gh-112075: Make PyDictKeysObject thread-safe#114741
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Uh oh!
There was an error while loading. Please reload this page.
Conversation
DinoV commented Jan 30, 2024 • edited by bedevere-app bot
Loading Uh oh!
There was an error while loading. Please reload this page.
edited by bedevere-app bot
Uh oh!
There was an error while loading. Please reload this page.
fc6beda to 137eebeCompareObjects/dictobject.c Outdated
| _Py_DecRefTotal(_PyInterpreterState_GET()); | ||
| #endif | ||
| if (--dk->dk_refcnt==0){ | ||
| if (_Py_atomic_add_ssize(&dk->dk_refcnt, -1) ==1){ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about adding a macro for the free-threading version and the default version?
corona10Jan 30, 2024 • edited
Loading Uh oh!
There was an error while loading. Please reload this page.
edited
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did similar approach at listobject: 393cbef
See: _Py_SET_ITEMREF
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@corona10 is there a significant difference in performance between the two builds? I'm not sure if it is worth it to add these preprocessor guards everywhere if there is no measurable effect :) Is there a different motivation than performance?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Regardless of whether we do the macro approach everywhere, I think it's a good idea in this specific case (specifically for performance).
137eebe to 0891529CompareUh oh!
There was an error while loading. Please reload this page.
0891529 to 398cb23Compare
colesbury left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This mostly looks good to me.
I don't think we want to be locking around shared_keys_usable_size(). It's both not sufficient for thread-safety at most of the call sites and not what we want to be doing for performance reasons.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
fe12c49 to 87c3446CompareUh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Objects/dictobject.c Outdated
| _Py_atomic_store_ssize(&keys->dk_nentries, keys->dk_nentries+1); | ||
| _Py_atomic_store_ssize(&keys->dk_usable, keys->dk_usable-1); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can use a weaker ordering here that will be faster, especially on x86 where "release" doesn't require any memory barrier:
_Py_atomic_store_ssize_relaxed(&keys->dk_nentries, keys->dk_nentries + 1); _Py_atomic_store_ssize_release(&keys->dk_usable, keys->dk_usable - 1); (I don't think we have _Py_atomic_store_ssize_release yet, though)
I find the memory orderings hard to reason correctly about, so I like to model them with CDSChecker. Here's the model I used for this:
https://github.com/colesbury/c11-model-checker/blob/cpython-models/test/gh-112075.c
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
a22f723 to 808dd89Compare
colesbury left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One minor formatting comment, but otherwise LGTM
Objects/dictobject.c Outdated
| if (_PyDict_HasSplitTable(mp)){ | ||
| LOCK_KEYS(keys); | ||
| dictkeys_incref(keys); | ||
| } else{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| }else{ | |
| } | |
| else{ |
808dd89 to 4914ba8Compare90bea6a to 88ab576Compare88ab576 to a9d3666CompareAdds locking for shared PyDictKeysObject's for dictionaries
Adds locking for shared PyDictKeysObject's for dictionaries
Adds locking for shared PyDictKeysObject's for dictionaries
Adds locking for shared keys
dictobjects thread-safe in--disable-gilbuilds #112075