gh-105540: Convert pytest tests of cases_generator to regular tests

[sobolevn]

I went ahead and converted them :)

Now they will be executed the same way as regular tests.
I've taken the approach we use for https://github.com/python/cpython/blob/main/Lib/test/test_clinic.py

Local run:

» ./python.exe -m test -v test_generated_cases == CPython 3.13.0a0 (heads/main:da98ed0aa04, Jul 12 2023, 23:50:11) [Clang 14.0.3 (clang-1403.0.22.14.1)] == macOS-13.4.1-arm64-arm-64bit little-endian == Python build: debug == cwd: /Users/sobolev/Desktop/cpython/build/test_python_77465æ == CPU count: 12 == encodings: locale=UTF-8, FS=utf-8 0:00:00 load avg: 1.56 Run tests sequentially 0:00:00 load avg: 1.56 [1/1] test_generated_cases test_effect_sizes (test.test_generated_cases.TestEffects.test_effect_sizes) ... ok test_array_error_if (test.test_generated_cases.TestGeneratedCases.test_array_error_if) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpnwp2ip6m Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp83kj4oij ok test_array_input (test.test_generated_cases.TestGeneratedCases.test_array_input) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpyq_8mt3h Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmptssmvzyp ok test_array_input_output (test.test_generated_cases.TestGeneratedCases.test_array_input_output) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp9ej806j9 Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp_rqt1jh7 ok test_array_output (test.test_generated_cases.TestGeneratedCases.test_array_output) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp1j80dud3 Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp0a1lgx20 ok test_binary_op (test.test_generated_cases.TestGeneratedCases.test_binary_op) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp6tkafxpr Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp8nc34ekw ok test_cache_effect (test.test_generated_cases.TestGeneratedCases.test_cache_effect) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp0cbo72dj Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp8qhl_sfc ok test_cond_effect (test.test_generated_cases.TestGeneratedCases.test_cond_effect) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpx4adjvf8 Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpq81jd5gj ok test_error_if_plain (test.test_generated_cases.TestGeneratedCases.test_error_if_plain) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmplae6z0p3 Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpiritl87t ok test_error_if_plain_with_comment (test.test_generated_cases.TestGeneratedCases.test_error_if_plain_with_comment) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpc9w7n9zq Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmposc9ygia ok test_error_if_pop (test.test_generated_cases.TestGeneratedCases.test_error_if_pop) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpyu93upkp Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpcual0bhu ok test_inst_no_args (test.test_generated_cases.TestGeneratedCases.test_inst_no_args) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpcchbpx7s Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpbqgi_phs ok test_inst_one_pop (test.test_generated_cases.TestGeneratedCases.test_inst_one_pop) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpn9cco114 Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp1s9r5wbx ok test_inst_one_push (test.test_generated_cases.TestGeneratedCases.test_inst_one_push) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp7c48tflr Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpda_z7uoc ok test_inst_one_push_one_pop (test.test_generated_cases.TestGeneratedCases.test_inst_one_push_one_pop) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp7alht925 Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp_0m7n5zw ok test_macro_cond_effect (test.test_generated_cases.TestGeneratedCases.test_macro_cond_effect) ... Read 2 instructions/ops, 1 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpm55x7lbt Wrote 0 instructions, 1 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp2mad7oyz ok test_macro_instruction (test.test_generated_cases.TestGeneratedCases.test_macro_instruction) ... Read 3 instructions/ops, 1 macros, 0 pseudos, and 1 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp7igsdpls Wrote 2 instructions, 1 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpzqxw_efb ok test_overlap (test.test_generated_cases.TestGeneratedCases.test_overlap) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp_ct08u9k Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmphn9i5p0i ok test_predictions_and_eval_breaker (test.test_generated_cases.TestGeneratedCases.test_predictions_and_eval_breaker) ... Read 2 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp8w5ljdry Wrote 2 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp6jjdnwrr ok test_suppress_dispatch (test.test_generated_cases.TestGeneratedCases.test_suppress_dispatch) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpbdh1a1wn Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpygejojvf ok ---------------------------------------------------------------------- Ran 20 tests in 0.021s OK == Tests result: SUCCESS == 1 test OK. Total duration: 81 ms Tests result: SUCCESS 

• Issue: cases_generator tests aren't run by CI, and are broken #105540

[sobolevn]

Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows NT or later).

Fixing tests failure. I don't have access to Win, so I will experiment :)

[gvanrossum]

Thank you, thank you, thank you!

[gvanrossum]

That is incorrect, and is not the reason why we go through this much effort. The prettified filename is incorporated in the output (as part of the header explaining that this was a generated file) and must be the same regardless of where it was generated, otherwise you get spurious diffs when e.g. running the tool on Windows.

[sobolevn]

Looks like something broke our tests :(

[gvanrossum]

Huh, tests pass for me locally. Let me see if I can merge.

[gvanrossum]

If the tests pass this time I will merge this upstream. I don't care enough about the one comment to hold it up, and I'd like to add some new tests.

[gvanrossum]

Oh dang. I wish the closing """ of the input/output text fragments were one tab stop to the right. But it doesn't matter enough for another PR.

[sobolevn]

Thanks! I will continue to make them better, since they now are a part of Lib/test :)

[gvanrossum]

Great, we need more eyeballs on this code generator.