Exclude ceval.c from the C-analyzer tool

[AlexWaygood]

The "check if generated files are up to date" CI check appears to be currently failing on all PRs (but not on pushes to main)

See, for example:

• gh-94440: Fix issue of ProcessPoolExecutor shutdown hanging #94468
• Simplify and speed-up math.hypot() and math.dist() #102734
• gh-87868: Sort and remove duplicates in getenvironment() #102731

This appears to be because the C-analyzer tool doesn't like the #line directives introduced in 70185de. I'm advised by the message printed to the terminal in https://github.com/python/cpython/actions/runs/4428706945/jobs/7768216988#step:14:84 that this is the appropriate short-term fix!

Cc. @ericsnowcurrently

Automerge-Triggered-By: GH:AlexWaygood

[zware]

Also need to open an issue to fix the parser :)

[AlexWaygood]

Also need to open an issue to fix the parser :)

Done!

• C-analyzer tool cannot parse #line directives #102737

[miss-islington]

@AlexWaygood: Status check is done, and it's a failure or timed out ❌.

[AlexWaygood]

I'm confused by the patchcheck.py failures -- there aren't any whitespace issues in my PR, as far as I can see. When I run it locally, patchcheck.py seems to make sweeping changes to Tools/c-analyzer/cpython/_parser.py. I can ignore the failure and merge anyway, but I'm not sure if I should...

[AlexWaygood]

Alternatively I could just add Tools/c-analyzer/cpython to the directories ignored by patchcheck.py.

[AlexWaygood]

(Here's the diff if I run patchcheck.py locally)

Details

diff --git a/Tools/c-analyzer/cpython/_parser.py b/Tools/c-analyzer/cpython/_parser.py index a2911e030f..57f17c5c59 100644 --- a/Tools/c-analyzer/cpython/_parser.py+++ b/Tools/c-analyzer/cpython/_parser.py@@ -105,25 +105,25 @@ def clean_lines(text): INCL_DIRS = clean_lines(''' # @begin=tsv@ -glob dirname-* .-* ./Include-* ./Include/internal--Modules/_decimal/**/*.c Modules/_decimal/libmpdec-Modules/_elementtree.c Modules/expat-Modules/_hacl/*.c Modules/_hacl/include-Modules/_hacl/*.h Modules/_hacl/include-Modules/md5module.c Modules/_hacl/include-Modules/sha1module.c Modules/_hacl/include-Modules/sha2module.c Modules/_hacl/include-Objects/stringlib/*.h Objects+glob dirname+* .+* ./Include+* ./Include/internal++Modules/_decimal/**/*.c Modules/_decimal/libmpdec+Modules/_elementtree.c Modules/expat+Modules/_hacl/*.c Modules/_hacl/include+Modules/_hacl/*.h Modules/_hacl/include+Modules/md5module.c Modules/_hacl/include+Modules/sha1module.c Modules/_hacl/include+Modules/sha2module.c Modules/_hacl/include+Objects/stringlib/*.h Objects # possible system-installed headers, just in case -Modules/_tkinter.c /usr/include/tcl8.6-Modules/_uuidmodule.c /usr/include/uuid-Modules/nismodule.c /usr/include/tirpc-Modules/tkappinit.c /usr/include/tcl+Modules/_tkinter.c /usr/include/tcl8.6+Modules/_uuidmodule.c /usr/include/uuid+Modules/nismodule.c /usr/include/tirpc+Modules/tkappinit.c /usr/include/tcl # @end=tsv@ ''')[1:] @@ -131,36 +131,36 @@ def clean_lines(text): INCLUDES = clean_lines(''' # @begin=tsv@ -glob include+glob include-**/*.h Python.h-Include/**/*.h object.h+**/*.h Python.h+Include/**/*.h object.h # for Py_HAVE_CONDVAR -Include/internal/pycore_gil.h pycore_condvar.h-Python/thread_pthread.h pycore_condvar.h+Include/internal/pycore_gil.h pycore_condvar.h+Python/thread_pthread.h pycore_condvar.h # other -Objects/stringlib/join.h stringlib/stringdefs.h-Objects/stringlib/ctype.h stringlib/stringdefs.h-Objects/stringlib/transmogrify.h stringlib/stringdefs.h-#Objects/stringlib/fastsearch.h stringlib/stringdefs.h-#Objects/stringlib/count.h stringlib/stringdefs.h-#Objects/stringlib/find.h stringlib/stringdefs.h-#Objects/stringlib/partition.h stringlib/stringdefs.h-#Objects/stringlib/split.h stringlib/stringdefs.h-Objects/stringlib/fastsearch.h stringlib/ucs1lib.h-Objects/stringlib/count.h stringlib/ucs1lib.h-Objects/stringlib/find.h stringlib/ucs1lib.h-Objects/stringlib/partition.h stringlib/ucs1lib.h-Objects/stringlib/split.h stringlib/ucs1lib.h-Objects/stringlib/find_max_char.h Objects/stringlib/ucs1lib.h-Objects/stringlib/count.h Objects/stringlib/fastsearch.h-Objects/stringlib/find.h Objects/stringlib/fastsearch.h-Objects/stringlib/partition.h Objects/stringlib/fastsearch.h-Objects/stringlib/replace.h Objects/stringlib/fastsearch.h-Objects/stringlib/split.h Objects/stringlib/fastsearch.h+Objects/stringlib/join.h stringlib/stringdefs.h+Objects/stringlib/ctype.h stringlib/stringdefs.h+Objects/stringlib/transmogrify.h stringlib/stringdefs.h+#Objects/stringlib/fastsearch.h stringlib/stringdefs.h+#Objects/stringlib/count.h stringlib/stringdefs.h+#Objects/stringlib/find.h stringlib/stringdefs.h+#Objects/stringlib/partition.h stringlib/stringdefs.h+#Objects/stringlib/split.h stringlib/stringdefs.h+Objects/stringlib/fastsearch.h stringlib/ucs1lib.h+Objects/stringlib/count.h stringlib/ucs1lib.h+Objects/stringlib/find.h stringlib/ucs1lib.h+Objects/stringlib/partition.h stringlib/ucs1lib.h+Objects/stringlib/split.h stringlib/ucs1lib.h+Objects/stringlib/find_max_char.h Objects/stringlib/ucs1lib.h+Objects/stringlib/count.h Objects/stringlib/fastsearch.h+Objects/stringlib/find.h Objects/stringlib/fastsearch.h+Objects/stringlib/partition.h Objects/stringlib/fastsearch.h+Objects/stringlib/replace.h Objects/stringlib/fastsearch.h+Objects/stringlib/split.h Objects/stringlib/fastsearch.h # @end=tsv@ ''')[1:] @@ -168,123 +168,123 @@ def clean_lines(text): MACROS = clean_lines(''' # @begin=tsv@ -glob name value--Include/internal/*.h Py_BUILD_CORE 1-Python/**/*.c Py_BUILD_CORE 1-Python/**/*.h Py_BUILD_CORE 1-Parser/**/*.c Py_BUILD_CORE 1-Parser/**/*.h Py_BUILD_CORE 1-Objects/**/*.c Py_BUILD_CORE 1-Objects/**/*.h Py_BUILD_CORE 1--Modules/_asynciomodule.c Py_BUILD_CORE 1-Modules/_codecsmodule.c Py_BUILD_CORE 1-Modules/_collectionsmodule.c Py_BUILD_CORE 1-Modules/_ctypes/_ctypes.c Py_BUILD_CORE 1-Modules/_ctypes/cfield.c Py_BUILD_CORE 1-Modules/_cursesmodule.c Py_BUILD_CORE 1-Modules/_datetimemodule.c Py_BUILD_CORE 1-Modules/_functoolsmodule.c Py_BUILD_CORE 1-Modules/_heapqmodule.c Py_BUILD_CORE 1-Modules/_io/*.c Py_BUILD_CORE 1-Modules/_io/*.h Py_BUILD_CORE 1-Modules/_localemodule.c Py_BUILD_CORE 1-Modules/_operator.c Py_BUILD_CORE 1-Modules/_posixsubprocess.c Py_BUILD_CORE 1-Modules/_sre/sre.c Py_BUILD_CORE 1-Modules/_threadmodule.c Py_BUILD_CORE 1-Modules/_tracemalloc.c Py_BUILD_CORE 1-Modules/_weakref.c Py_BUILD_CORE 1-Modules/_zoneinfo.c Py_BUILD_CORE 1-Modules/atexitmodule.c Py_BUILD_CORE 1-Modules/cmathmodule.c Py_BUILD_CORE 1-Modules/faulthandler.c Py_BUILD_CORE 1-Modules/gcmodule.c Py_BUILD_CORE 1-Modules/getpath.c Py_BUILD_CORE 1-Modules/getpath_noop.c Py_BUILD_CORE 1-Modules/itertoolsmodule.c Py_BUILD_CORE 1-Modules/main.c Py_BUILD_CORE 1-Modules/mathmodule.c Py_BUILD_CORE 1-Modules/posixmodule.c Py_BUILD_CORE 1-Modules/sha256module.c Py_BUILD_CORE 1-Modules/sha512module.c Py_BUILD_CORE 1-Modules/signalmodule.c Py_BUILD_CORE 1-Modules/symtablemodule.c Py_BUILD_CORE 1-Modules/timemodule.c Py_BUILD_CORE 1-Modules/unicodedata.c Py_BUILD_CORE 1--Modules/_json.c Py_BUILD_CORE_BUILTIN 1-Modules/_pickle.c Py_BUILD_CORE_BUILTIN 1-Modules/_testinternalcapi.c Py_BUILD_CORE_BUILTIN 1--Include/cpython/abstract.h Py_CPYTHON_ABSTRACTOBJECT_H 1-Include/cpython/bytearrayobject.h Py_CPYTHON_BYTEARRAYOBJECT_H 1-Include/cpython/bytesobject.h Py_CPYTHON_BYTESOBJECT_H 1-Include/cpython/ceval.h Py_CPYTHON_CEVAL_H 1-Include/cpython/code.h Py_CPYTHON_CODE_H 1-Include/cpython/dictobject.h Py_CPYTHON_DICTOBJECT_H 1-Include/cpython/fileobject.h Py_CPYTHON_FILEOBJECT_H 1-Include/cpython/fileutils.h Py_CPYTHON_FILEUTILS_H 1-Include/cpython/frameobject.h Py_CPYTHON_FRAMEOBJECT_H 1-Include/cpython/import.h Py_CPYTHON_IMPORT_H 1-Include/cpython/listobject.h Py_CPYTHON_LISTOBJECT_H 1-Include/cpython/methodobject.h Py_CPYTHON_METHODOBJECT_H 1-Include/cpython/object.h Py_CPYTHON_OBJECT_H 1-Include/cpython/objimpl.h Py_CPYTHON_OBJIMPL_H 1-Include/cpython/pyerrors.h Py_CPYTHON_ERRORS_H 1-Include/cpython/pylifecycle.h Py_CPYTHON_PYLIFECYCLE_H 1-Include/cpython/pymem.h Py_CPYTHON_PYMEM_H 1-Include/cpython/pystate.h Py_CPYTHON_PYSTATE_H 1-Include/cpython/sysmodule.h Py_CPYTHON_SYSMODULE_H 1-Include/cpython/traceback.h Py_CPYTHON_TRACEBACK_H 1-Include/cpython/tupleobject.h Py_CPYTHON_TUPLEOBJECT_H 1-Include/cpython/unicodeobject.h Py_CPYTHON_UNICODEOBJECT_H 1+glob name value++Include/internal/*.h Py_BUILD_CORE 1+Python/**/*.c Py_BUILD_CORE 1+Python/**/*.h Py_BUILD_CORE 1+Parser/**/*.c Py_BUILD_CORE 1+Parser/**/*.h Py_BUILD_CORE 1+Objects/**/*.c Py_BUILD_CORE 1+Objects/**/*.h Py_BUILD_CORE 1++Modules/_asynciomodule.c Py_BUILD_CORE 1+Modules/_codecsmodule.c Py_BUILD_CORE 1+Modules/_collectionsmodule.c Py_BUILD_CORE 1+Modules/_ctypes/_ctypes.c Py_BUILD_CORE 1+Modules/_ctypes/cfield.c Py_BUILD_CORE 1+Modules/_cursesmodule.c Py_BUILD_CORE 1+Modules/_datetimemodule.c Py_BUILD_CORE 1+Modules/_functoolsmodule.c Py_BUILD_CORE 1+Modules/_heapqmodule.c Py_BUILD_CORE 1+Modules/_io/*.c Py_BUILD_CORE 1+Modules/_io/*.h Py_BUILD_CORE 1+Modules/_localemodule.c Py_BUILD_CORE 1+Modules/_operator.c Py_BUILD_CORE 1+Modules/_posixsubprocess.c Py_BUILD_CORE 1+Modules/_sre/sre.c Py_BUILD_CORE 1+Modules/_threadmodule.c Py_BUILD_CORE 1+Modules/_tracemalloc.c Py_BUILD_CORE 1+Modules/_weakref.c Py_BUILD_CORE 1+Modules/_zoneinfo.c Py_BUILD_CORE 1+Modules/atexitmodule.c Py_BUILD_CORE 1+Modules/cmathmodule.c Py_BUILD_CORE 1+Modules/faulthandler.c Py_BUILD_CORE 1+Modules/gcmodule.c Py_BUILD_CORE 1+Modules/getpath.c Py_BUILD_CORE 1+Modules/getpath_noop.c Py_BUILD_CORE 1+Modules/itertoolsmodule.c Py_BUILD_CORE 1+Modules/main.c Py_BUILD_CORE 1+Modules/mathmodule.c Py_BUILD_CORE 1+Modules/posixmodule.c Py_BUILD_CORE 1+Modules/sha256module.c Py_BUILD_CORE 1+Modules/sha512module.c Py_BUILD_CORE 1+Modules/signalmodule.c Py_BUILD_CORE 1+Modules/symtablemodule.c Py_BUILD_CORE 1+Modules/timemodule.c Py_BUILD_CORE 1+Modules/unicodedata.c Py_BUILD_CORE 1++Modules/_json.c Py_BUILD_CORE_BUILTIN 1+Modules/_pickle.c Py_BUILD_CORE_BUILTIN 1+Modules/_testinternalcapi.c Py_BUILD_CORE_BUILTIN 1++Include/cpython/abstract.h Py_CPYTHON_ABSTRACTOBJECT_H 1+Include/cpython/bytearrayobject.h Py_CPYTHON_BYTEARRAYOBJECT_H 1+Include/cpython/bytesobject.h Py_CPYTHON_BYTESOBJECT_H 1+Include/cpython/ceval.h Py_CPYTHON_CEVAL_H 1+Include/cpython/code.h Py_CPYTHON_CODE_H 1+Include/cpython/dictobject.h Py_CPYTHON_DICTOBJECT_H 1+Include/cpython/fileobject.h Py_CPYTHON_FILEOBJECT_H 1+Include/cpython/fileutils.h Py_CPYTHON_FILEUTILS_H 1+Include/cpython/frameobject.h Py_CPYTHON_FRAMEOBJECT_H 1+Include/cpython/import.h Py_CPYTHON_IMPORT_H 1+Include/cpython/listobject.h Py_CPYTHON_LISTOBJECT_H 1+Include/cpython/methodobject.h Py_CPYTHON_METHODOBJECT_H 1+Include/cpython/object.h Py_CPYTHON_OBJECT_H 1+Include/cpython/objimpl.h Py_CPYTHON_OBJIMPL_H 1+Include/cpython/pyerrors.h Py_CPYTHON_ERRORS_H 1+Include/cpython/pylifecycle.h Py_CPYTHON_PYLIFECYCLE_H 1+Include/cpython/pymem.h Py_CPYTHON_PYMEM_H 1+Include/cpython/pystate.h Py_CPYTHON_PYSTATE_H 1+Include/cpython/sysmodule.h Py_CPYTHON_SYSMODULE_H 1+Include/cpython/traceback.h Py_CPYTHON_TRACEBACK_H 1+Include/cpython/tupleobject.h Py_CPYTHON_TUPLEOBJECT_H 1+Include/cpython/unicodeobject.h Py_CPYTHON_UNICODEOBJECT_H 1 # implied include of <unistd.h> -Include/**/*.h _POSIX_THREADS 1-Include/**/*.h HAVE_PTHREAD_H 1+Include/**/*.h _POSIX_THREADS 1+Include/**/*.h HAVE_PTHREAD_H 1 # from pyconfig.h -Include/cpython/pthread_stubs.h HAVE_PTHREAD_STUBS 1-Python/thread_pthread_stubs.h HAVE_PTHREAD_STUBS 1+Include/cpython/pthread_stubs.h HAVE_PTHREAD_STUBS 1+Python/thread_pthread_stubs.h HAVE_PTHREAD_STUBS 1 # from Objects/bytesobject.c -Objects/stringlib/partition.h STRINGLIB_GET_EMPTY() bytes_get_empty()-Objects/stringlib/join.h STRINGLIB_MUTABLE 0-Objects/stringlib/partition.h STRINGLIB_MUTABLE 0-Objects/stringlib/split.h STRINGLIB_MUTABLE 0-Objects/stringlib/transmogrify.h STRINGLIB_MUTABLE 0+Objects/stringlib/partition.h STRINGLIB_GET_EMPTY() bytes_get_empty()+Objects/stringlib/join.h STRINGLIB_MUTABLE 0+Objects/stringlib/partition.h STRINGLIB_MUTABLE 0+Objects/stringlib/split.h STRINGLIB_MUTABLE 0+Objects/stringlib/transmogrify.h STRINGLIB_MUTABLE 0 # from Makefile -Modules/getpath.c PYTHONPATH 1-Modules/getpath.c PREFIX ...-Modules/getpath.c EXEC_PREFIX ...-Modules/getpath.c VERSION ...-Modules/getpath.c VPATH ...-Modules/getpath.c PLATLIBDIR ...-#Modules/_dbmmodule.c USE_GDBM_COMPAT 1-Modules/_dbmmodule.c USE_NDBM 1-#Modules/_dbmmodule.c USE_BERKDB 1+Modules/getpath.c PYTHONPATH 1+Modules/getpath.c PREFIX ...+Modules/getpath.c EXEC_PREFIX ...+Modules/getpath.c VERSION ...+Modules/getpath.c VPATH ...+Modules/getpath.c PLATLIBDIR ...+#Modules/_dbmmodule.c USE_GDBM_COMPAT 1+Modules/_dbmmodule.c USE_NDBM 1+#Modules/_dbmmodule.c USE_BERKDB 1 # See: setup.py -Modules/_decimal/**/*.c CONFIG_64 1-Modules/_decimal/**/*.c ASM 1-Modules/expat/xmlparse.c HAVE_EXPAT_CONFIG_H 1-Modules/expat/xmlparse.c XML_POOR_ENTROPY 1-Modules/_dbmmodule.c HAVE_GDBM_DASH_NDBM_H 1+Modules/_decimal/**/*.c CONFIG_64 1+Modules/_decimal/**/*.c ASM 1+Modules/expat/xmlparse.c HAVE_EXPAT_CONFIG_H 1+Modules/expat/xmlparse.c XML_POOR_ENTROPY 1+Modules/_dbmmodule.c HAVE_GDBM_DASH_NDBM_H 1 # from Modules/_sha3/sha3module.c -Modules/_sha3/kcp/KeccakP-1600-inplace32BI.c PLATFORM_BYTE_ORDER 4321 # force big-endian-Modules/_sha3/kcp/*.c KeccakOpt 64-Modules/_sha3/kcp/*.c KeccakP200_excluded 1-Modules/_sha3/kcp/*.c KeccakP400_excluded 1-Modules/_sha3/kcp/*.c KeccakP800_excluded 1+Modules/_sha3/kcp/KeccakP-1600-inplace32BI.c PLATFORM_BYTE_ORDER 4321 # force big-endian+Modules/_sha3/kcp/*.c KeccakOpt 64+Modules/_sha3/kcp/*.c KeccakP200_excluded 1+Modules/_sha3/kcp/*.c KeccakP400_excluded 1+Modules/_sha3/kcp/*.c KeccakP800_excluded 1 # others -Modules/_sre/sre_lib.h LOCAL(type) static inline type-Modules/_sre/sre_lib.h SRE(F) sre_ucs2_##F-Objects/stringlib/codecs.h STRINGLIB_IS_UNICODE 1+Modules/_sre/sre_lib.h LOCAL(type) static inline type+Modules/_sre/sre_lib.h SRE(F) sre_ucs2_##F+Objects/stringlib/codecs.h STRINGLIB_IS_UNICODE 1 # @end=tsv@ ''')[1:]

[zware]

patchcheck.py checks the full changed file, not just changed lines, and is pretty militant about removing tabs. You can ignore the failure here and merge manually, though a proper fix would be nice (done separately :)).

Merging manually will also give you the chance to clean up the commit message; @miss-islington's automerge uses the PR description, which is a bit much for a commit message in this case.

[AlexWaygood]

patchcheck.py checks the full changed file, not just changed lines, and is pretty militant about removing tabs. You can ignore the failure here and merge manually, though a proper fix would be nice (done separately :)).

It's weird to me that it only checks PRs, and doesn't run on pushes to main!

[AlexWaygood]

Merging manually will also give you the chance to clean up the commit message; @miss-islington's automerge uses the PR description, which is a bit much for a commit message in this case.

Whoops, didn't see that until too late. I edited it a bit before merging, but it's probably still too verbose for your tastes, I guess -- sorry about that!

[zware]

It's weird to me that it only checks PRs, and doesn't run on pushes to main!

It's patchcheck for a reason ;). It also predates GitHub and the PR workflow by several years, and hasn't seen much change since that switch.

Whoops, didn't see that until too late. I edited it a bit before merging, but it's probably still too verbose for your tastes, I guess -- sorry about that!

No problem; I edited it in and didn't get the edit done before the merge. Just something to keep in mind when using automerge :)

[ericsnowcurrently]

Thanks for taking care of this @AlexWaygood. I'll get things sorted out as soon as I can.

[ericsnowcurrently]

@AlexWaygood, regarding the patchcheck failure, please see gh-92266.