gh-90815: Add mimalloc memory allocator

[tiran]

https://bugs.python.org/issue46657

gh-90815

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @tiran for commit d4d545e5d54cced48793a1e67d36553588d4c560 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @tiran for commit 973b6178c0b46c2cd839b06cbe543ea039962442 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @tiran for commit 9b8d0dac0c8ca184f0d7a93ee4b880edcacf64fd 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @tiran for commit a9c29e66902900022bd74357647d0cc8c1538292 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @tiran for commit e7231f186360d756df7d1320e39433fb46077e99 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @tiran for commit 2472ecbf4e4723186145d3476dcbee7ef95cb8a4 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @tiran for commit 10498c3826865ac62811be3a61db019b01f23fc8 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

[neonene]

JFYI on Windows,

• test_fstring.test_many_expressions reserves 150MB memory which seems never released. The following case holds 500MB:

build_fstr = lambda n: "f'" + ('{1}' * n) + "'" exec(build_fstr(350)*1000) 

• MSVC fails to build x86 PGO, without generating python311*.pgc profiles if test_embed.EmbeddingTests.test_bpo20891 is used for an exercise. The same goes for x64 PGO when trained with only the test. (Fixed or improved by gh-94790)

[tiran]

I have rebased the PR again.

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @tiran for commit 65af44c 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

[ericsnowcurrently]

Thanks!

[bobfang1992]

Hi, just want to check if this is still under development?

[zooba]

tiran is currently taking a break, so it's not being actively developed. I think we're still interested in using mimalloc though? Probably @ericsnowcurrently is the best person to sync with.

[erlend-aasland]

Probably @ericsnowcurrently is the best person to sync with.

... or @colesbury.

[ericsnowcurrently]

FWIW, I don't have much relationship with this effort currently. At one point I thought I needed it for per-interpreter GIL, but realized later I didn't. That said, I would like to see this happen as I think it opens up several valuable opportunities to us. Furthermore, I did work with @daanx, the creator of mimalloc, to address the remaining issues, so this PR should be mostly ready, aside from conflicts and pulling in the latest mimalloc.

Ultimately, this change will probably need a PEP. (I also don't know what our options are regarding linking mimalloc in rather than vendoring it.) Other than that, I don't have much insight to offer, sadly.

[daanx]

Hi, @daanx here from mimalloc -- let me know if I can help with this PR. Just wanted to add that it would be good to merge the latest version of mimalloc as recently I made many improvements in particular for Valgrind and ASAN (and ETW) tools.

[erlend-aasland]

Superseded by #109914.