Derby #16: Convert 50 sites to Argument Clinic across 9 files

[larryhastings]

BPO	20184
Nosy	@ncoghlan, @larryhastings, @serhiy-storchaka, @1st1, @shihai1991
PRs	gh-64383: Migrate builtins.vars and builtins.dir to Argument Clinic #18512 bpo-20184: Convert termios to Argument Clinic. #22693
Files	issue20184_builtin_conversion.diff: Approach to ensuring all builtins have signatures issue20184_builtin_conversion_v2.diff: All non-type callables in bltinmodule.c converted or classified issue20184_builtin_conversion_v3.diff: Updated to fix test_pydoc and test_gdb issue20184_builtin_conversion_v4.diff: Updated for AC changes issue20184_builtin_conversion_v5.diff: Additional cleanups in the test case issue20184_builtin_conversion_v6.diff: Updated for AC changes in the lead up to 3.4.0 issue20184_builtin_conversion_v7.diff: Updated to apply to trunk (August 2014) dbm_clinic.patch dbm_clinic_2.patch json_clinic.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2014-01-08.00:21:44.691>
labels = ['extension-modules', 'type-feature', 'expert-argument-clinic']
title = 'Derby #16: Convert 50 sites to Argument Clinic across 9 files'
updated_at = <Date 2020-10-18.14:54:14.974>
user = 'https://github.com/larryhastings'

bugs.python.org fields:

activity = <Date 2020-10-18.14:54:14.974>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Extension Modules', 'Argument Clinic']
creation = <Date 2014-01-08.00:21:44.691>
creator = 'larry'
dependencies = []
files = ['33696', '33701', '33740', '33860', '33861', '34664', '36385', '39032', '39070', '39292']
hgrepos = []
issue_num = 20184
keywords = ['patch']
message_count = 20.0
messages = ['207642', '209159', '209179', '209203', '209409', '209953', '210218', '210375', '215164', '224765', '225385', '225423', '225424', '225425', '241094', '241226', '241347', '242588', '362008', '378872']
nosy_count = 7.0
nosy_names = ['ncoghlan', 'larry', 'nadeem.vawda', 'python-dev', 'serhiy.storchaka', 'yselivanov', 'shihai1991']
pr_nums = ['18512', '22693']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue20184'
versions = ['Python 3.5']

[larryhastings]

This issue is part of the Great Argument Clinic Conversion Derby,
where we're trying to convert as much of Python 3.4 to use
Argument Clinic as we can before Release Candidate 1 on January 19.

This issue asks you to change the following bundle of files:
Modules/faulthandler.c: 7 sites
Modules/_pickle.c: 7 sites
Modules/_lzmamodule.c: 7 sites
Python/bltinmodule.c: 6 sites
Modules/termios.c: 6 sites
Modules/syslogmodule.c: 6 sites
Modules/_dbmmodule.c: 4 sites
Modules/_gdbmmodule.c: 2 sites
Modules/_json.c: 5 sites

Talk to me (larry) if you only want to attack part of a bundle.

For instructions on how to convert a function to work with Argument
Clinic, read the "howto":
http://docs.python.org/dev/howto/clinic.html

[ncoghlan]

I'm going to specifically tackle the "bltinmodule" side of things using a test driven approach: adding a new test to test_inspect that checks all the builtin names have signatures (I'll explicitly exclude ones that are known not to be handled, like range and slice).

[ncoghlan]

Attached patch shows the new test case I'm using to ensure that all callable builtins have signatures, and once we get it that way, it stays that way.

Preliminary goal is signatures for all the non-type objects, and once I get to that point, I'll propose it for review.

[ncoghlan]

More comprehensive patch uploaded - all the non-type callables implemented in bltinmodule.c have been converted or classified with a reason for not being converted yet (see the new test in test_inspect.py for details, as well as the AC 3.4 and AC 3.5 comments in the module itself).

I also cleaned up the docstrings for the builtins I actually changed in the patch. There were a few that had never been properly updated for the Py3k transition.

There are still a couple of test failures with this version - the doctest tests get confused by the fact ord and chr now have a doctest in their docstrings, and test_gdb is definitely not in a happy place (that has always been temperamental, though).

I also just realised the Unicode character in the new ord and chr docstrings could pose a compatibility problem at the C compiler source encoding level, so we may have to reconsider that (even though I was rather happy to sneak that obscure Monty Python reference in there).

[ncoghlan]

Assigning to myself to make it clear that bltinmodule is the only part of this still under consideration for 3.4.

The test_pydoc and test_gdb failures pointed to real issues with the previous patch:

• the pydoc errors themselves were incidental, indicating that I had added doctests to chr and ord. However, those new doctests used a Unicode character in a C string, which seems like a recipe for portability trouble. I took those doctests out again, and updated the prose docs instead. I like my obscure multilingual Monty Python reference and would like to keep it now I thought of it :)

• the gdb error suggests that gdb is relying on being able to find builtin_id based on its exact signature, including the parameter names. Rather than trying to figure out the full details of that, I've instead partially reverted its conversion to Argument Clinic by disabling the input block and restoring the old parameter names in the function signature. We can change it back to full conversion for 3.5, after AC has the ability to use different names in the Python signature and in the C implementation function.

This does mean we'll want to update the signature by hand once you merge the patch changing the indicators that argument clinic is looking for.

[ncoghlan]

Larry, if this version looks good to you, I'd like to commit it.

• id() is now back to being a properly generated AC function (since AC can now preserve the old C level signature)

• sorted() is partially converted and has a __text_signature__ compatible docstring. However, full conversion will have to wait for the ability to preserve the kwds dict, since that's the API exposed by the list object.

• with the new never-triggered-by-accident AC syntax, the test case now ensures that all the builtins that are expected to *not* expose signature info at this point, don't. As they're converted for 3.5, that will force them to be added to the list of functions that are checked for compatibility.

I'm thinking it's probably worth flagging this new test as a CPython implementation detail test, though.

[ncoghlan]

Larry, I'd like to include the most-builtin-functions signature support for 3.4, but since the AC Derby is over, it's your call as release manager.

If it looks good to you, go ahead and commit it (I won't have time until later in the week), otherwise just drop the priority and punt it to 3.5.

[ncoghlan]

Interpreting the lack of response as "this can wait until 3.5".