Skip to content

Conversation

@isidentical
Copy link
Member

@isidenticalisidentical commented Jul 2, 2021

The new resizing system works like this;

$ cat t.py a + a + a + b + c + a + a + a + b + c + a + a + a + b + c + a + a + a + b + c [repeated 99 more times] $ ./python t.py RESIZE: prev len = 32, new len = 66 FINAL SIZE: 56 ----------------------------------------------------- RESIZE: prev len = 32, new len = 66 RESIZE: prev len = 66, new len = 134 RESIZE: prev len = 134, new len = 270 RESIZE: prev len = 270, new len = 542 RESIZE: prev len = 542, new len = 1086 RESIZE: prev len = 1086, new len = 2174 RESIZE: prev len = 2174, new len = 4350 RESIZE: prev len = 4350, new len = 8702 FINAL SIZE: 8004 

So now we do considerably lower number of _PyBytes_Resize calls.

https://bugs.python.org/issue43950

Automerge-Triggered-By: GH:isidentical

Copy link
Member

@pablogsalpablogsal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can confirm that this speeds up the allocation quite a lot as expected 👍

❯ time ./python -m test test_compile -m test_stack_overflow -v == CPython 3.11.0a0 (heads/optimize-co-positions:760daf9858, Jul 2 2021, 19:08:05) [GCC 11.1.0] == Linux-5.12.13-arch1-2-x86_64-with-glibc2.33 little-endian == cwd: /home/pablogsal/github/python/main/build/test_python_15988æ == CPU count: 36 == encodings: locale=UTF-8, FS=utf-8 0:00:00 load avg: 2.15 Run tests sequentially 0:00:00 load avg: 2.15 [1/1] test_compile test_stack_overflow (test.test_compile.TestSpecifics) ... ok ---------------------------------------------------------------------- Ran 1 test in 3.945s OK == Tests result: SUCCESS == 1 test OK. Total duration: 4.0 sec Tests result: SUCCESS ./python -m test test_compile -m test_stack_overflow -v 3.77s user 0.47s system 99% cpu 4.241 total 

@miss-islington
Copy link
Contributor

@isidentical: Status check is done, and it's a success ❌ .

@isidentical
Copy link
MemberAuthor

concurrent_futures error, irrelevant, restarting the CI

@miss-islington
Copy link
Contributor

@isidentical: Status check is done, and it's a success ✅ .

@miss-islington
Copy link
Contributor

Sorry, I can't merge this PR. Reason: 3 of 5 required status checks have not succeeded: 2 expected..

@miss-islington
Copy link
Contributor

@isidentical: Status check is done, and it's a success ✅ .

@miss-islingtonmiss-islington merged commit ec8759b into python:mainJul 2, 2021
@isidenticalisidentical changed the title bpo-43950: optimize column table assembling with pre-sizing objectbpo-43950: optimize column table assembling with pre-sizing the column tableJul 3, 2021
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@isidentical@miss-islington@pablogsal@the-knights-who-say-ni@bedevere-bot