gh-90953: Emit deprecation warnings for deprecated ast features

[serhiy-storchaka]

I am not sure that it is worth to include it in 3.11. It may be better to wait for 3.12 or 3.13.

https://bugs.python.org/issue46797

• Issue: ast.Constant.n deprecated without warning #90953

[AlexWaygood]

Could we hold off doing this until 3.7 is EOL? It is quite hard to write version-compatible code otherwise, at least if you want to avoid deprecation warnings. (I'm speaking as a maintainer of https://github.com/PyCQA/flake8-pyi, which makes heavy use of the ast module.)

[serhiy-storchaka]

Yes, this is my intention.

[serhiy-storchaka]

3.7 will be EOL at 2023-06, before the release of 3.12. I think this PR can already be merged in 3.12. Or wait for 3.13?

[AlexWaygood]

3.7 will be EOL at 2023-06, before the release of 3.12. I think this PR can already be merged in 3.12. Or wait for 3.13?

ExtSlice was only deprecated in 3.9. Other than that one, however, I'm happy with this PR being merged in 3.12 :)

[serhiy-storchaka]

Well, I am going to merge it.

Note, that it does not remove previously deprecated feature. It only adds runtime warnings for already long deprecated (3 or more releases ago) features. It is much much more smoother than standard practice of adding runtime warnings at the moment of deprecation.

[AlexWaygood]

I support merging this before 3.12 is released, but I would still leave the deprecation warning for ExtSlice to a future PR, personally. Unlike the other deprecated features, it was deprecated in 3.9, not 3.7.

[hugovk]

(Merge conflict resolved)

There's about a week until the 3.12 cutoff. I think it's okay to include ExtSlice here, it's replacement Tuple has been around for a long time, and ast.*ExtSlice shows up in 22 projects in the PyPI top 5k:

❯ python3 ~/github/misc/cpython/search_pypi_top.py -q ."ast.*ExtSlice"./pyre-check-nightly-0.0.101680433836.tar.gz: pyre-check-nightly-0.0.101680433836/typeshed/stubs/pyflakes/pyflakes/checker.pyi: def EXTSLICE(self, tree: ast.ExtSlice, omit: _OmitType = ...) -> None: ..../reportlab-3.6.12.tar.gz: reportlab-3.6.12/src/reportlab/lib/rl_safe_eval.py: elif isinstance(slice_, ast.ExtSlice):./typed_ast-1.5.4.tar.gz: typed_ast-1.5.4/ast27/Python/Python-ast.c: value = ast2obj_list(o->v.ExtSlice.dims, ast2obj_slice);./typed_ast-1.5.4.tar.gz: typed_ast-1.5.4/ast3/Python/Python-ast.c: value = ast2obj_list(o->v.ExtSlice.dims, ast2obj_slice);./mypy-1.1.1.tar.gz: mypy-1.1.1/mypy/fastparse.py: is_py38_or_earlier and isinstance(n.slice, ast3.ExtSlice)./mypy-1.1.1.tar.gz: mypy-1.1.1/mypy/fastparse.py: def visit_ExtSlice(self, n: ast3.ExtSlice) -> TupleExpr:./mypy-1.1.1.tar.gz: mypy-1.1.1/mypy/fastparse.py: assert isinstance(n.slice, ast3.ExtSlice)./pure_eval-0.2.2.tar.gz: pure_eval-0.2.2/pure_eval/core.py: elif isinstance(index, ast.ExtSlice):./gast-0.5.3.tar.gz: gast-0.5.3/gast/ast3.py: new_slice = ast.ExtSlice(./gast-0.5.3.tar.gz: gast-0.5.3/gast/ast2.py: new_slice = ast.ExtSlice([adjust_slice(self._visit(elt))./ipython-8.12.0.tar.gz: ipython-8.12.0/IPython/core/guarded_eval.py: if isinstance(node, ast.ExtSlice):./flake8_expression_complexity-0.0.11.tar.gz: flake8_expression_complexity-0.0.11/flake8_expression_complexity/utils/complexity.py: (ast.ExtSlice, 'ext_slice'),./astroid-2.15.1.tar.gz: astroid-2.15.1/astroid/nodes/node_classes.py: """Class representing an :class:`ast.ExtSlice` node../astroid-2.15.1.tar.gz: astroid-2.15.1/astroid/rebuilder.py: def visit(self, node: ast.ExtSlice, parent: nodes.Subscript) -> nodes.Tuple:./astroid-2.15.1.tar.gz: astroid-2.15.1/astroid/rebuilder.py: self, node: ast.ExtSlice, parent: nodes.Subscript./asttokens-2.2.1.tar.gz: asttokens-2.2.1/asttokens/asttokens.py: ast.Slice, ast.ExtSlice, ast.Index, ast.keyword,./asttokens-2.2.1.tar.gz: asttokens-2.2.1/asttokens/asttokens.py: - ``ast.ExtSlice``./griffe-0.25.5.tar.gz: griffe-0.25.5/src/griffe/agents/nodes.py: from ast import ExtSlice as NodeExtSlice./numba-0.56.4.tar.gz: numba-0.56.4/numba/tests/test_stencils.py: slice=ast.ExtSlice(./numba-0.56.4.tar.gz: numba-0.56.4/numba/tests/test_stencils.py: if isinstance(node.slice, ast.ExtSlice):./pytype-2023.3.31.tar.gz: pytype-2023.3.31/pytype/typeshed/stubs/pyflakes/pyflakes/checker.pyi: def EXTSLICE(self, tree: ast.ExtSlice, omit: _OmitType = ...) -> None: ..../RestrictedPython-6.0.tar.gz: RestrictedPython-6.0/src/RestrictedPython/transformer.py: elif isinstance(slice_, ast.ExtSlice):Time: 0:00:17.604108Found 22 matching lines in 14 projects

But I'm also okay to splitting it into another PR and merging it after the 3.13 branch in about a week and a few days.

In any case, let's add this to What's New.

[hugovk]

A suggested What's New entry:

* The following :mod:`ast` features have been deprecated in documentation since Python 3.8 and now raise a proper :exc:`DeprecationWarning`: :class:`!ast.Num`, :class:`!ast.Str`, :class:`!ast.Bytes`, :class:`!ast.NameConstant` and :class:`!ast.Ellipsis`; use :class:`ast.Constant` instead. Similarly since 3.9, instead of :class:`!ast.Index` use the index value directly; and instead of :class:`!ast.ExtSlice` use :class:`ast.Tuple`. (Contributed by Serhiy Storchaka in :gh:`90953`.)

[AlexWaygood]

Well, my position remains the same: I'd love to see this get into 3.12, but I'd really rather we excluded ExtSlice and Index from this PR for now. It's not so much a matter of finding a replacement for the class -- the issue is that it could really complicate life for people writing linters using ast.NodeVisitor subclasses to inspect Python source code, if you want to support Python 3.8-3.12, and you want to avoid DeprecationWarnings.

Life's already complicated enough as it is, with how frequently the Python AST changes from version to version :)

[hugovk]

Okay, would you like to create a new PR with just the 3.8 ones and we can merge that into 3.12 before the cutoff?

Then we can merge this with the 3.9 ones into 3.13 after the cutoff?

[AlexWaygood]

Okay, would you like to create a new PR with just the 3.8 ones and we can merge that into 3.12 before the cutoff?

Sounds good. I'll list @serhiy-storchaka as a co-author.

[AlexWaygood]

(Accidentally pushed to this PR instead of #104199. Force-pushed to undo that.)

[furkanonder]

cc: @isidentical

[hugovk]

Python 3.12 beta is out, this can now be updated to be included in main/3.13.

[serhiy-storchaka]

Closed in favor of #104199.

@AlexWaygood, do you mind to create a new PR for ExtSlice and Index?

[AlexWaygood]

Closed in favor of #104199.

@AlexWaygood, do you mind to create a new PR for ExtSlice and Index?

Sure, I'm working on it now