bpo-45522: Allow to disable freelists on build time

[tiran]

Freelists for object structs can now be disabled. A new configure
option --without-freelists can be used to disable all freelists
except empty tuple singleton. Internal Py*_MAXFREELIST macros can now
be defined as 0 without causing compiler warnings and segfaults.

Signed-off-by: Christian Heimes [email protected]

https://bugs.python.org/issue45522

Automerge-Triggered-By: GH:tiran

[tiran]

• Need to find a better way to deal with empty tuple singleton

[tiran]

One unit test is failing when building Python --without-freelists. I assume the tests relies on freelists internally.

test test_exceptions failed -- Traceback (most recent call last): File "/home/heimes/dev/python/cpython/Lib/test/test_exceptions.py", line 1340, in test_recursion_normalizing_with_no_memory self.assertIn(b'Fatal Python error: _PyErr_NormalizeException: ' ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError: b'Fatal Python error: _PyErr_NormalizeException: Cannot recover from MemoryErrors while normalizing exceptions.' not found in b'sys.excepthook is missing\nobject address : 0x7f289e86e270\nobject refcount : 1\nobject type : 0x8212c0\nobject type name: MemoryError\nobject repr : \nlost sys.stderr\n' 

[vstinner]

What is your motivation for this change? Is it an experimental change?

Freelists are important for Python performance. Shipping a Python without them doesn't seem useful.

I disabled freelists in the past when working on subinterpreters. But I modified the code in the meanwhile to make them per-interpreter.

[corona10]

@tiran
Considering @vstinner 's concern(?), what about adding this package when mimalloc option is adopted?

[tiran]

What is your motivation for this change? Is it an experimental change?

Freelists are important for Python performance. Shipping a Python without them doesn't seem useful.

My motivations are listed in the ticket. tl;dr it's for alternative memory allocators like mimalloc. Mimalloc in particular has its own internal freelists system. An experimental build of Python with mimalloc and without freelists has shown minimal slown only.

If you don't like the configure option, then I can split the PR up into the internal C code fixes and the public configure option. I definitely want to land the C code changes. They also fix various bugs when Py*_MAXFREELIST are set to 0.

[tiran]

PR GH-29086 is a minimal version of this PR without the configure flag.

[corona10]

@tiran I prefer this version for our future experiment :)

[corona10]

I am okay with this PR for future experiments.
I believe that mimalloc will be adopted at least for improving performance.

see: Benchmark result which Trian and Neil digged at this sprint -> https://gist.github.com/pablogsal/8027937b71cd30f17aaaa5ef7c885d3e