Add support for sentinels (PEP 661)

[Viicos]

Fixes#591.

Regarding the pickling behavior, no matter if the sentinel is defined at the module level or in a nested scope, this seems tricky to handle. Ideally, I'd like to have the following working:

sentinel=Sentinel('sentinel') classA: def__init__(self): self.s=sentinela=A() a.sissentinel# Trueimportpicklea_r=pickle.loads(pickle.dumps(a)) a_r.sissentinel# False

Considering unplicking data can be done across interpreter runs, this isn't really achievable, but still will be surprising.

[TeamSpen210]

For pickling, there's some builtin behaviour which might be perfect for this. If __reduce__ returns the sentinel name, and __module__ is defined (via sys._getframemodulename()), pickle will import the module then lookup attributes to find the value. It'll even check and error on pickling if the sentinel isn't present there. This'll make your example work, by looking up __main__.sentinel. If that global wasn't present, self.s = Sentinel("a.s") would also work.

[JelleZijlstra]

I'm worried the pickling behavior the PEP wants may change over time. One option would be to make them not pickleable for now. That way, we can always add pickling as a feature later, but if we add pickling now and need to change the behavior later, that may cause issues for users.

[Viicos]

Yes anyway this will be better discussed in the PEP thread. I removed the __reduce__() implementation, let me know if you actually want to raise an error.

[JelleZijlstra]

Thanks! Can you make it raise an error? Otherwise I think it will pickle by value by default which is probably not what we'd want.

[Viicos]

Yes, makes sense, added a type error following prior art: https://github.com/search?q=repo%3Apython%2Fcpython+%22Cannot+pickle%22&type=code.

[JelleZijlstra]

Thanks! Could you also add this to the documentation?

[JelleZijlstra]

Fully-qualified? That doesn't match the PEP which does MISSING = Sentinel("MISSING").

[Viicos]

This was copied verbatim from https://github.com/taleinat/python-stdlib-sentinels/, updated.

[Viicos]

Actually the repr construction by splitting dots don't make sense anymore, so I also updated the logic in the last logic.

[TeamSpen210]

It'd make sense to be fully qualified inside the module, so Sentinel("SomeClass.CONST"). Pickle expects that if we decide to use it. It's recommended in the PEP under "Additional Notes", but not required.

[Viicos]

It'd make sense to be fully qualified inside the module, so Sentinel("SomeClass.CONST"). Pickle expects that if we decide to use it. It's recommended in the PEP under "Additional Notes", but not required.

This will have to be decided in the PEP discussion.

[JelleZijlstra]

Would you mind fixing the lint? I think we need to make Union unconditionally defined.

[HexDecimal]

This implementation of Sentinel is impractical in its current state.

Sentinel's are supposed to be unique to the module it's defined in, allowing Sentinel's with the same name from separate modules to be equal is unexpected and broken behavior.

importbar, baz# Both with 'MISSING = Sentinel("MISSING")'# Correct implementation would pass this assertassertbar.MISSING!=baz.MISSING# These modules mean "missing" in different ways!

__copy__ has been ignored, so this allows copies which break the identity of the Sentinel.

importcopysentinel=Sentinel('sentinel') # Correct implementation would pass this assertassertsentineliscopy.copy(sentinel) # Any copy must have the same identity!

An important feature of Sentinel's is that their identity is preserved even during serialization, but Pickle support seems to have been rejected.

At the same time this implemented a repr attribute which will cause its own issues and was specifically rejected by PEP 661 as being too complex to add.

>>>Sentinel("sentinel", repr="foo") ==Sentinel("sentinel", repr="bar") False# 'repr' makes these unique?

[JelleZijlstra]

allowing Sentinel's with the same name from separate modules to be equal is unexpected and broken behavior

The implementation doesn't do that. Since __eq__ is not defined, equality is by identity.

__copy__ has been ignored, so this allows copies which break the identity of the Sentinel.

In fact copy.copy throws an error because it calls __getstate__.

An important feature of Sentinel's is that their identity is preserved even during serialization, but Pickle support seems to have been rejected.

We'll likely add Pickle support later but I do not believe it is essential. We're waiting for the PEP author to decide what it will look like.

At the same time this implemented a repr attribute which will cause its own issues

Good point, I thought our implementation came from a proposed reference implementation. I'm not sure what the issues are that you're alluding to, but if the PEP doesn't end up supporting this argument we can deprecate it from typing-extensions.

[HexDecimal]

Sorry. I've used dataclasses so often that I forgot the default behavior of Python classes. Here I am making several wrong assumptions about how equality works.

We'll likely add Pickle support later but I do not believe it is essential.

I've used sentinels in my own Python projects and pickling them is essential. I currently use sentinel-value but I'd wish to use something more standard. I don't really need to be in a hurry though.

We're waiting for the PEP author to decide what it will look like.

Does this count? #591 (comment)
I think attribute lookup appears to be very forward compatible, one could seamlessly replace a typing_extensions.Sentinel definition with a typing.Sentinel and the replacement class will be loaded instead of the class originally pickled.

[Viicos]

@HexDecimal all this discussion should live in the PEP thread instead, where you can provide feedback on this initial implementation. Regarding pickling, I already started giving thoughts in the last comment of the thread.