gh-118761: Improve import time by lazy import of traceback

[donbarbos]

TODO:

• Improve import time modules by lazy import traceback
• Remove unused import fcntl

CPython configure flags:

./configure --enable-optimizations --with-lto --enable-loadable-sqlite-extensions

Hyperfine benchmark for import traceback:

$ hyperfine --warmup 11 "./python -c 'import traceback'" Benchmark 1: ./python -c 'import traceback' Time (mean ± σ): 17.0 ms ± 0.5 ms [User: 14.2 ms, System: 2.7 ms] Range (min … max): 16.5 ms … 22.7 ms 167 runs

Hyperfine benchmarks for updated modules:

import _pyrepl experiments (no diff)

➜ cpython git:(lazy-imports-traceback) ✗ hyperfine --warmup 11 --runs 3000"./python -c 'import _pyrepl'" Benchmark 1: ./python -c 'import _pyrepl' Time (mean ± σ): 9.2ms± 0.4ms[User: 7.3 ms, System: 1.8 ms] Range (min … max): 8.6ms… 14.7ms3000 runs ➜ cpython git:(3d3a4beefe) ✗ hyperfine --warmup 11 --runs 3000"./python -c 'import _pyrepl'" Benchmark 1: ./python -c 'import _pyrepl' Time (mean ± σ): 9.3ms± 0.3ms[User: 7.4 ms, System: 1.8 ms] Range (min … max): 8.7ms… 14.7ms3000 runs 

import code experiments

➜ cpython git:(lazy-imports-traceback) ✗ hyperfine --warmup 11 --runs 3000"./python -c 'import code'" Benchmark 1: ./python -c 'import code' Time (mean ± σ): 10.0ms± 0.5ms[User: 8.2 ms, System: 1.8 ms] Range (min … max): 9.3ms… 15.8ms3000 runs ➜ cpython git:(3d3a4beefe) ✗ hyperfine --warmup 11 --runs 3000"./python -c 'import code'" Benchmark 1: ./python -c 'import code' Time (mean ± σ): 17.8ms± 0.8ms[User: 14.9 ms, System: 2.9 ms] Range (min … max): 17.0ms… 28.7ms3000 runs 

import doctest experiments (no diff)

➜ cpython git:(lazy-imports-traceback) ✗ hyperfine --warmup 11"./python -c 'import doctest'" Benchmark 1: ./python -c 'import doctest' Time (mean ± σ): 34.8ms± 0.3ms[User: 30.2 ms, System: 4.6 ms] Range (min … max): 34.2ms… 35.7ms84 runs ➜ cpython git:(3d3a4beefe) ✗ hyperfine --warmup 11"./python -c 'import doctest'" Benchmark 1: ./python -c 'import doctest' Time (mean ± σ): 35.6ms± 0.5ms[User: 30.8 ms, System: 4.8 ms] Range (min … max): 34.9ms… 37.9ms81 runs 

import logging experiments (no diff)

➜ cpython git:(lazy-imports-traceback) ✗ hyperfine --warmup 11 --runs 2000"./python -c 'import logging'" Benchmark 1: ./python -c 'import logging' Time (mean ± σ): 20.9ms± 0.7ms[User: 17.7 ms, System: 3.1 ms] Range (min … max): 20.1ms… 32.7ms2000 runs ➜ cpython git:(3d3a4beefe) ✗ hyperfine --warmup 11 --runs 2000"./python -c 'import logging'" Benchmark 1: ./python -c 'import logging' Time (mean ± σ): 20.7ms± 0.5ms[User: 17.7 ms, System: 3.0 ms] Range (min … max): 20.0ms… 30.8ms2000 runs 

import pdb experiments (no diff)

➜ cpython git:(lazy-imports-traceback) ✗ hyperfine --warmup 11 --runs 2000"./python -c 'import pdb'" Benchmark 1: ./python -c 'import pdb' Time (mean ± σ): 29.3ms± 0.9ms[User: 25.0 ms, System: 4.2 ms] Range (min … max): 28.2ms… 37.9ms2000 runs ➜ cpython git:(3d3a4beefe) ✗ hyperfine --warmup 11 --runs 2000"./python -c 'import pdb'" Benchmark 1: ./python -c 'import pdb' Time (mean ± σ): 30.1ms± 1.3ms[User: 25.9 ms, System: 4.2 ms] Range (min … max): 28.8ms… 44.2ms2000 runs 

import py_compile experiments

➜ cpython git:(lazy-imports-traceback) ✗ hyperfine --warmup 11 --runs 2000"./python -c 'import py_compile'" Benchmark 1: ./python -c 'import py_compile' Time (mean ± σ): 11.7ms± 0.5ms[User: 9.6 ms, System: 2.0 ms] Range (min … max): 10.9ms… 19.5ms2000 runs ➜ cpython git:(3d3a4beefe) ✗ hyperfine --warmup 11"./python -c 'import py_compile'" Benchmark 1: ./python -c 'import py_compile' Time (mean ± σ): 18.2ms± 0.5ms[User: 15.4 ms, System: 2.7 ms] Range (min … max): 17.4ms… 21.0ms159 runs 

import pydoc experiments

➜ cpython git:(lazy-imports-traceback) ✗ hyperfine --warmup 11"./python -c 'import pydoc'" Benchmark 1: ./python -c 'import pydoc' Time (mean ± σ): 32.6ms± 0.4ms[User: 28.4 ms, System: 4.2 ms] Range (min … max): 31.8ms… 34.3ms87 runs ➜ cpython git:(3d3a4beefe) ✗ hyperfine --warmup 11"./python -c 'import pydoc'" Benchmark 1: ./python -c 'import pydoc' Time (mean ± σ): 34.3ms± 0.3ms[User: 30.2 ms, System: 4.1 ms] Range (min … max): 33.5ms… 35.2ms84 runs 

import unittest experiments

➜ cpython git:(lazy-imports-traceback) ✗ hyperfine --warmup 11"./python -c 'import unittest'" Benchmark 1: ./python -c 'import unittest' Time (mean ± σ): 21.9ms± 0.3ms[User: 18.4 ms, System: 3.4 ms] Range (min … max): 21.3ms… 22.8ms133 runs ➜ cpython git:(3d3a4beefe) ✗ hyperfine --warmup 11"./python -c 'import unittest'" Benchmark 1: ./python -c 'import unittest' Time (mean ± σ): 23.3ms± 0.3ms[User: 19.5 ms, System: 3.7 ms] Range (min … max): 22.6ms… 24.3ms123 runs 

import xmlrpc experiments (no diff)

➜ cpython git:(lazy-imports-traceback) ✗ hyperfine --warmup 11 --runs 3000"./python -c 'import xmlrpc'" Benchmark 1: ./python -c 'import xmlrpc' Time (mean ± σ): 9.1ms± 0.4ms[User: 7.3 ms, System: 1.8 ms] Range (min … max): 8.7ms… 14.6ms3000 runs ➜ cpython git:(3d3a4beefe) ✗ hyperfine --warmup 11 --runs 3000"./python -c 'import xmlrpc'" Benchmark 1: ./python -c 'import xmlrpc' Time (mean ± σ): 9.1ms± 0.4ms[User: 7.4 ms, System: 1.7 ms] Range (min … max): 8.6ms… 19.6ms3000 runs ➜ cpython git:(lazy-imports-traceback) ✗ hyperfine --warmup 11"./python -c 'import xmlrpc'" Benchmark 1: ./python -c 'import xmlrpc' Time (mean ± σ): 8.8ms± 0.6ms[User: 7.3 ms, System: 1.5 ms] Range (min … max): 8.3ms… 14.5ms277 runs ➜ cpython git:(3d3a4beefe) ✗ hyperfine --warmup 11"./python -c 'import xmlrpc'" Benchmark 1: ./python -c 'import xmlrpc' Time (mean ± σ): 8.7ms± 0.2ms[User: 7.4 ms, System: 1.4 ms] Range (min … max): 8.2ms… 9.8ms303 runs 

• Issue: Improve import time of various stdlib modules #118761

[picnixz]

traceback is generally used in conjunction with other debugging modules, so I'm not sure it's worth the change. I'd like to mention that I also benchmark the import times using -X importtime as they don't suffer from startup time. Now, if we don't see improvements overall, I don't think we should actually do the change. I only changed modules for which import time is reduced by at least 20% (using -X importtime) or where we gain something noticeable on hyperfine. Most of your results show <10% improvements or no changes at all, so I don't think we need to do this. We definitely don't want to make all imports as lazy IMO.

[AA-Turner]

Please see my comment in your other PR, and also Benedikt's above.

A

[donbarbos]

Ok, i realized that this PR was a mistake and i will improve import time of each module separately. Maybe i will send PR for same module but without some other imports