types.UnionType is not subscriptable

[wyfo]

BPO	45418
Nosy	@gvanrossum, @serhiy-storchaka, @wyfo, @Fidget-Spinner

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-10-10.07:24:44.303>
created_at = <Date 2021-10-09.21:37:56.188>
labels = ['3.10']
title = 'types.UnionType is not subscriptable'
updated_at = <Date 2021-10-10.18:36:04.246>
user = 'https://github.com/wyfo'

bugs.python.org fields:

activity = <Date 2021-10-10.18:36:04.246>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2021-10-10.07:24:44.303>
closer = 'joperez'
components = []
creation = <Date 2021-10-09.21:37:56.188>
creator = 'joperez'
dependencies = []
files = []
hgrepos = []
issue_num = 45418
keywords = []
message_count = 8.0
messages = ['403554', '403573', '403575', '403576', '403594', '403596', '403597', '403605']
nosy_count = 4.0
nosy_names = ['gvanrossum', 'serhiy.storchaka', 'joperez', 'kj']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue45418'
versions = ['Python 3.10']

[wyfo]

types.UnionType is not subscriptable, and this is an issue when type manipulations are done.

A common maniputation I've to do is to substitute all the TypeVar of a potential generic type by their specialization in the given context.
For example, given a class:

@dataclass
class Foo(Generic[T]):
 bar: list[T]
 baz: T | None

in the case of Foo[int], I want to compute the effective type of the fields, which will be list[int] and int | None.
It could be done pretty easily by a recursive function:

def substitute(tp, type_vars: dict):
 origin, args = get_origin(tp), get_args(tp)
 if isinstance(tp, TypeVar):
 return type_vars.get(tp, tp)
 elif origin is Annotated:
 return Annotated[(substitute(args[0], type_vars), *args[1:])]
 else:
 return origin[tuple(substitute(arg) for arg in args)] # this line fails for types.UnionType

And this is not the only manipulation I've to do on generic types. In fact, all my library (apischema) is broken in Python 3.10 because of types.UnionType.
I've to substitute types.UnionType by typing.Union everywhere to make things work; types.UnionType is just not usable for dynamic manipulations.

I've read PEP-604 and it doesn't mention if types.UnionType should be subscriptable or not. Is there a reason for not making it subscriptable?

[Fidget-Spinner]

I don't understand your example, T | None doesn't return a types.Union object, it returns typing.Union/typing.Optional. (I'm assuming this T is the TypeVar in typing). Which *is* subscriptable.

>>> (T | None)[int].__origin__
typing.Union

If you meant to say: why is typing.Union[] allowed, but not types.UnionType[]? That is intentional. types.UnionType is only meant for builtin types. Once you union with *any* type from typing, it will convert to a typing.Union.

>>> type(int | str)
<class 'types.UnionType'>

>>> int | str | T
typing.Union[int, str, ~T]

If you intend to reconstruct a types.Union from another types.Union, you can do:

args = get_args(int | str)
import operator, functools
functools.reduce(operator.or_, args)

And guard this code with an isinstance(tp, types.UnionType) check.

[wyfo]

Indeed, sorry, my example was bad. My library was raising at several place, and I've extrapolated about generic substitution.

I've indeed other substitutions (without TypeVar), and because they were failing, I've assumed that all of my substitutions were failing; I was wrong about generic one.

For example, if I want to substitute int | Collection[int] to int | list[int], I will have to replace types.UnionType by typing.Union or use reduce, while it was not necessary in 3.9 where I could just write get_origin(tp)[new_args].

So I'll have to add some if in my code.

[Fidget-Spinner]

No worries!

So I'll have to add some if in my code.

Yeah, we had to do that in the typing module too. Hope you manage to fix your library without much trouble.

[gvanrossum]

If you meant to say: why is typing.Union[] allowed, but not
types.UnionType[]? That is intentional. types.UnionType is only meant for
builtin types. Once you union with *any* type from typing, it will convert
to a typing.Union.

But why? Just so types.UnionType (if it has a typevar) doesn’t have to
support subscriptions? Even if this saves us now, I agree with OP that it
ought to allow it, so we can deprecate typing.Union properly. And e.g.
dict[str, T] works.

--Guido (mobile)

[Fidget-Spinner]

@guido,

I hope I didn't misunderstand you, but to clarify, what OP is asking is an alternative way to construct types.UnionType objects and write:

types.UnionType[int, str]

like how we used to write before 3.10:

typing.Union[int, str]

I don't know why we need this. We can write int | str. The reason for PEP-604 in the first place was to avoid the subscript syntax and use | since it's cleaner. OP's use case is for reconstructing types.UnionType objects easily, but functools.reduce(operator.or_, args) works.

Re: TypeVar subscription; PEP-604 syntax already supports that. We used to implement that in C. After Serhiy's Great Cleanup, a bitwise OR with a TypeVar automatically converts types.UnionType to typing.Union. So all the TypeVar support is now done in Python.

>>> type(int | str)
<class 'types.UnionType'>

>>> (int | str | T)[dict]
typing.Union[int, str, dict]

[gvanrossum]

Oh, I see. No, they must make another special case, like for Annotated.--
--Guido (mobile)

[serhiy-storchaka]

types.UnionType corresponds to typing._UnionGenericAlias, not typing.Union.

We can make (int | str | T)[dict] returning an instance of types.UnionType instead of an instance of typing._UnionGenericAlias. But it will be a breaking change, because typing._UnionGenericAlias and types.UnionType are different and not completely compatible types. We should wait some time before making such changes, so all user code will be made supporting both typing._UnionGenericAlias and types.UnionType.

If the user code does something special like substituting int | Collection[int] to int | list[int], it should have some additional ifs in any case, otherwise it will not recognize new typing types including types.UnionTypes. And subscription does not work in all typing types, we have copy_with() for some types and special cases for others in the code of the typing module. I am going to unify it finally, but it takes time, my time and user's time to migrate to new idioms.