gh-121464: Make concurrent iteration over enumerate safe under free-threading#125734
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We make concurrent iteration with
enumeratethread-safe (e.g. not crash, correct results are not guaranteed, see #124397) byUsing
_PyObject_IsUniquelyReferencedfor the re-use of the result tupleIn
enum_nextusingFT_ATOMIC_LOAD_SSIZE_RELAXED/FT_ATOMIC_STORE_SSIZE_RELAXEDfor loading and saving of the mutableen->en_indexIn
enum_next_long(andenum_reduce) we have to deal withen->en_longindexwhich is mutable. Since the objects inen_longindexneed to be properly refcounted this is a bit hard to handle. Two options:i) Use a lock. This is a simple approach and the performance cost might be acceptable since use cases of
enumeratewith numbers that do not fit in a long are rareii) The
en->en_longindexcan be atomically swapped withstepped_upusing_Py_atomic_exchange_ptr. With this approachen->en_longindexis either zero, or has a refcount of 1. The thread performing the swap ofen->en_longindexcan decrement the olden->en_longindex. A branch with this approach is enumerate_ft_v3.In this PR we pick option i) because it is simpler and more robust.