3.11.4: ValueError when inverting enum.Flag member with mask member

[The-Compiler]

With code such as:

import enum

class Flag(enum.Flag):
 A = 0x01
 B = 0x02
 Mask = 0xff

print(~Flag.A)

Python 3.10.11 prints Flag.B, and so does Python 3.11.3. However, with Python 3.11.4, this happens instead:

Traceback (most recent call last):
 File "/home/florian/tmp/f.py", line 9, in <module>
 print(~Flag.A)
 ^^^^^^^
 File "/usr/lib/python3.11/enum.py", line 1542, in __invert__
 self._inverted_ = self.__class__(self._flag_mask_ ^ self._value_)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/lib/python3.11/enum.py", line 711, in __call__
 return cls.__new__(cls, value)
 ^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/lib/python3.11/enum.py", line 1136, in __new__
 raise exc
 File "/usr/lib/python3.11/enum.py", line 1113, in __new__
 result = cls._missing_(value)
 ^^^^^^^^^^^^^^^^^^^^
 File "/usr/lib/python3.11/enum.py", line 1454, in _missing_
 raise ValueError('%r: no members with value %r' % (cls, unknown))
ValueError: <flag 'Flag'>: no members with value 252

As a workaround, a detour via .value works in this case:

>>> Flag((Flag.A | Flag.B).value & ~Flag.A.value)
<Flag.B: 2>

This causes issues with PyQt, which has the following flags (as bindings from C++):

>>> from PyQt6.QtCore import Qt
>>> for e in Qt.KeyboardModifier:
... print(f"{e.name} = {hex(e.value)}")
... 
NoModifier = 0x0
ShiftModifier = 0x2000000
ControlModifier = 0x4000000
AltModifier = 0x8000000
MetaModifier = 0x10000000
KeypadModifier = 0x20000000
GroupSwitchModifier = 0x40000000
KeyboardModifierMask = 0xfe000000

(Output from Python 3.10 - with Python 3.11, KeyboardModifierMask goes missing in the output, and so does Flag.Mask above, but that seems like a different issue?)

With my project, I'm trying to remove a modifier from the given flags. With Python 3.10.11 and 3.11.3:

>>> (Qt.KeyboardModifier.ShiftModifier | Qt.KeyboardModifier.ControlModifier) & ~Qt.KeyboardModifier.ControlModifier
<KeyboardModifier.ShiftModifier: 33554432>

But with Python 3.11.4, same issue as above:

>>> from PyQt6.QtCore import Qt
>>> (Qt.KeyboardModifier.ShiftModifier | Qt.KeyboardModifier.ControlModifier) & ~Qt.KeyboardModifier.ControlModifier
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/usr/lib/python3.11/enum.py", line 1542, in __invert__
 self._inverted_ = self.__class__(self._flag_mask_ ^ self._value_)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/lib/python3.11/enum.py", line 711, in __call__
 return cls.__new__(cls, value)
 ^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/lib/python3.11/enum.py", line 1136, in __new__
 raise exc
 File "/usr/lib/python3.11/enum.py", line 1113, in __new__
 result = cls._missing_(value)
 ^^^^^^^^^^^^^^^^^^^^
 File "/usr/lib/python3.11/enum.py", line 1454, in _missing_
 raise ValueError('%r: no members with value %r' % (cls, unknown))
ValueError: <flag 'KeyboardModifier'>: no members with value 2147483648

As a culprit, I suspect:

• enum.CONFORM behavior breaks backwards compatibility #103365
• gh-103365: [Enum] STRICT boundary corrections #103494
• [3.11] gh-103365: [Enum] STRICT boundary corrections (GH-103494) #103513

cc @ethanfurman @benburrill

Linked PRs

• gh-105497: [Enum] Fix flag inversion when alias/mask members exist. #105542
• [3.11] gh-105497: [Enum] Fix Flag inversion when alias/mask members exist. (GH-105542) #105571
• [3.12] gh-105497: [Enum] Fix Flag inversion when alias/mask members exist. (GH-105542) #105572
• gh-105497: [Enum] Fix flag mask inversion when unnamed flags exist #106468
• [3.12] gh-105497: [Enum] Fix flag mask inversion when unnamed flags exist (GH-106468) #106620
• [3.11] gh-105497: [Enum] Fix flag mask inversion when unnamed flags exist (GH-106468) #106621

[The-Compiler]

Thanks @ethanfurman for the quick fix! I can confirm this fixes things for my project (qutebrowser) as well. I suppose this can be closed then?

[benburrill]

May be worth noting that there are still a few subtle differences between the new Flag behavior and the behavior from 3.10.

In Python 3.10, ~Flag.Mask == Flag(0), and Flag(252) would have raised a ValueError.
In Python 3.11.3, ~Flag.Mask == Flag(0) == Flag(252) due to the behavior of CONFORM
In Python 3.11.4, ~Flag.Mask == Flag(0) != Flag(252) (STRICT is still not strict enough to restore 3.10 behavior)
With @ethanfurman's fix, ~Flag.Mask == Flag(252) != Flag(0).

I think this change is probably fine for most cases, but it would be a good idea to add some more test cases.