bpo-45506: Re-enable test_embed.

[ericsnowcurrently]

In gh-29063 I ended up disabling test_embed on non-Windows by accident. This gets it running again.

https://bugs.python.org/issue45506

[ericsnowcurrently]

Hmm, GH-29274 broke ASAN for PyLongObject but this wasn't caught because test_embed was skipped. Guess I'll be fixing that too. 😞

asan report

$ Programs/_testembed test_run_main_loop Py_RunMain(): sys.argv=['-c', 'arg2'] ================================================================= ==25816==ERROR: AddressSanitizer: heap-use-after-free on address 0x6340000017e0 at pc 0x5655423764d5 bp 0x7ffda59989b0 sp 0x7ffda59989a0 READ of size 8 at 0x6340000017e0 thread T0 #0 0x5655423764d4 in PyLong_AsSsize_t Objects/longobject.c:553 #1 0x5655423e56d5 in get_type_attr_as_size Objects/structseq.c:42 #2 0x5655423e56d5 in PyStructSequence_New Objects/structseq.c:58 #3 0x565542879fa6 in PyFloat_GetInfo Objects/floatobject.c:97 #4 0x5655425f6555 in _PySys_InitCore Python/sysmodule.c:2782 #5 0x5655425f6555 in _PySys_Create Python/sysmodule.c:3074 #6 0x565542599271 in pycore_interp_init Python/pylifecycle.c:823 #7 0x56554259ff5d in pyinit_config Python/pylifecycle.c:865 #8 0x56554259ff5d in pyinit_core Python/pylifecycle.c:1028 #9 0x5655425a111e in Py_InitializeFromConfig Python/pylifecycle.c:1218 #10 0x565542316f69 in init_from_config_clear Programs/_testembed.c:398 #11 0x565542316f69 in test_run_main Programs/_testembed.c:1672 #12 0x565542317122 in test_run_main_loop Programs/_testembed.c:1683 #13 0x7f8cd539eb96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96) #14 0x565542312579 in _start (/home/esnow/projects/work/cpython-perf/cpython/Programs/_testembed+0x19a579) 0x6340000017e0 is located 4064 bytes inside of 119848-byte region [0x634000000800,0x63400001dc28) freed by thread T0 here: #0 0x7f8cd62107a8 in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xde7a8) #1 0x56554259ba6b in finalize_interp_delete Python/pylifecycle.c:1699 #2 0x56554259ba6b in Py_FinalizeEx Python/pylifecycle.c:1859 #3 0x56554262f356 in Py_RunMain Modules/main.c:669 #4 0x565542316fa3 in test_run_main Programs/_testembed.c:1674 #5 0x565542317122 in test_run_main_loop Programs/_testembed.c:1683 #6 0x7f8cd539eb96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96) previously allocated by thread T0 here: #0 0x7f8cd6210d28 in __interceptor_calloc (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xded28) #1 0x5655425a6b62 in PyInterpreterState_New Python/pystate.c:214 #2 0x56554259fcf4 in pycore_create_interpreter Python/pylifecycle.c:630 #3 0x56554259fcf4 in pyinit_config Python/pylifecycle.c:859 #4 0x56554259fcf4 in pyinit_core Python/pylifecycle.c:1028 #5 0x5655425a111e in Py_InitializeFromConfig Python/pylifecycle.c:1218 #6 0x565542316f69 in init_from_config_clear Programs/_testembed.c:398 #7 0x565542316f69 in test_run_main Programs/_testembed.c:1672 #8 0x565542317122 in test_run_main_loop Programs/_testembed.c:1683 #9 0x7f8cd539eb96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96) SUMMARY: AddressSanitizer: heap-use-after-free Objects/longobject.c:553 in PyLong_AsSsize_t Shadow bytes around the buggy address: 0x0c687fff82a0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c687fff82b0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c687fff82c0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c687fff82d0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c687fff82e0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd =>0x0c687fff82f0: fd fd fd fd fd fd fd fd fd fd fd fd[fd]fd fd fd 0x0c687fff8300: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c687fff8310: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c687fff8320: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c687fff8330: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c687fff8340: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb ==25816==ABORTING 

to duplicate:

./configure --with-address-sanitizer --without-pymalloc make -j8 Programs/_testembed test_run_main_loop 

@markshannon

[ericsnowcurrently]

FYI, "ASAN_OPTIONS=abort_on_error=1" has been useful to break in GDB.

[ericsnowcurrently]

The problem is that _PyStructSequence_InitType() sets a number of small ints on the type's dict. If it's a static type (and never cleared) then it will still point to the small int objects from the original interpreter no matter how many times you finalize and re-init the runtime. Hence the ASan failures.

[markshannon]

See https://bugs.python.org/issue45691