The Great Argument Clinic Conversion Derby Meta-Issue

[larryhastings]

BPO	20187
Nosy	@warsaw, @larryhastings, @bitdancer, @serhiy-storchaka
Dependencies	bpo-20133: Derby: Convert the audioop module to use Argument Clinic bpo-20148: Derby: Convert the _sre module to use Argument Clinic bpo-20151: Derby: Convert the binascii module to use Argument Clinic bpo-20152: Derby Use Sphinx 1.4.9 for now #15: Convert 50 sites to Argument Clinic across 9 files bpo-20159: Derby bpo-29527: Disable broken Travis docs job #7: Convert 51 sites to Argument Clinic across 3 files -> Derby: Convert the ElementTree module to use Argument Clinic bpo-20168: Derby: Convert the _tkinter module to use Argument Clinic bpo-20170: Derby Support "bpo-" in Misc/NEWS #1: Convert 137 sites to Argument Clinic in Modules/posixmodule.c bpo-20171: Derby Rename README to README.rst and enhance formatting #2: Convert 115 sites to Argument Clinic in Modules/_cursesmodule.c bpo-20172: Derby bpo-29403: Fix mock's broken autospec behavior on method-bound builtin functions #3: Convert 67 sites to Argument Clinic across 4 files (Windows) bpo-20173: Derby Update Python Software Foundation Copyright Year. #4: Convert 53 sites to Argument Clinic across 5 files bpo-20174: Derby Convert READMEs to rst #5: Convert 50 sites to Argument Clinic across 3 files bpo-20175: Derby Add Pycharm's .idea directory to gitignore #6: Convert 50 sites to Argument Clinic across 8 files bpo-20177: Derby Change some mercurial/ hg.python.org references. #8: Convert 28 sites to Argument Clinic across 2 files bpo-20178: Derby Fix some deprecation warnings in Doc/conf.py #9: Convert 52 sites to Argument Clinic across 11 files bpo-20179: Derby bpo-29474: Improve documentation for weakref.WeakValueDictionary #10: Convert 50 sites to Argument Clinic across 4 files bpo-20180: Derby bpo-29474: Improve documentation for weakref.WeakValueDictionary #11: Convert 50 sites to Argument Clinic across 9 files bpo-20181: Derby bpo-29524: Add Objects/call.c file #12: Convert 50 sites to Argument Clinic across 4 files bpo-20182: Derby Fix bpo-29528 Use a secure variable to stop spam #13: Convert 50 sites to Argument Clinic across 5 files bpo-20183: Derby Disable Travis docs job until a fix is found #14: Convert 41 sites to Argument Clinic across 5 files bpo-20184: Derby Make Travis docs build more lenient #16: Convert 50 sites to Argument Clinic across 9 files bpo-20185: Derby bpo-29438: fixed use-after-free in key sharing dict #17: Convert 49 sites to Argument Clinic across 13 files bpo-20186: Derby Rename Doc/README.txt to Doc/README.rst #18: Convert 31 sites to Argument Clinic across 23 files

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/larryhastings'
closed_at = None
created_at = <Date 2014-01-08.00:31:45.475>
labels = ['type-feature', 'expert-email', 'expert-argument-clinic']
title = 'The Great Argument Clinic Conversion Derby Meta-Issue'
updated_at = <Date 2018-06-14.17:51:04.719>
user = 'https://github.com/larryhastings'

bugs.python.org fields:

activity = <Date 2018-06-14.17:51:04.719>
actor = 'BreamoreBoy'
assignee = 'larry'
closed = False
closed_date = None
closer = None
components = ['email', 'Argument Clinic']
creation = <Date 2014-01-08.00:31:45.475>
creator = 'larry'
dependencies = ['20133', '20148', '20151', '20152', '20159', '20168', '20170', '20171', '20172', '20173', '20174', '20175', '20177', '20178', '20179', '20180', '20181', '20182', '20183', '20184', '20185', '20186']
files = []
hgrepos = []
issue_num = 20187
keywords = []
message_count = 9.0
messages = ['207649', '207650', '207697', '208152', '208170', '208171', '208172', '208686', '208687']
nosy_count = 4.0
nosy_names = ['barry', 'larry', 'r.david.murray', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue20187'
versions = ['Python 3.4']

[larryhastings]

This is a meta-issue tracking all the Argument Clinic Derby sub-issues.

[larryhastings]

The issue tracker isn't letting me add all the dependencies to the dependency list. For the record, here are the numbers:

bpo-20133
bpo-20148
bpo-20151
bpo-20152
bpo-20159
bpo-20168
bpo-20170
bpo-20171
bpo-20172
bpo-20173
bpo-20174
bpo-20175
bpo-20177
bpo-20178
bpo-20179
bpo-20180
bpo-20181
bpo-20182
bpo-20183
bpo-20184
bpo-20185
bpo-20186

(so far)

[larryhastings]

Here's how the files were parceled out into the various bundles:

bpo-20170
Derby #1: Convert 137 sites to Argument Clinic in Modules/posixmodule.c

[Modules/posixmodule.c](https://github.com/python/cpython/blob/main/Modules/posixmodule.c): 137 sites

bpo-20171
Derby #2: Convert 115 sites to Argument Clinic in Modules/_cursesmodule.c

[Modules/_cursesmodule.c](https://github.com/python/cpython/blob/main/Modules/_cursesmodule.c): 115 sites

bpo-20172
Derby #3: Convert 67 sites to Argument Clinic across 4 files

[PC/msvcrtmodule.c](https://github.com/python/cpython/blob/main/PC/msvcrtmodule.c): 18 sites
[PC/winreg.c](https://github.com/python/cpython/blob/main/PC/winreg.c): 24 sites
[PC/winsound.c](https://github.com/python/cpython/blob/main/PC/winsound.c): 3 sites
[Modules/_winapi.c](https://github.com/python/cpython/blob/main/Modules/_winapi.c): 22 sites

bpo-20173
Derby #4: Convert 53 sites to Argument Clinic across 5 files

[Modules/_codecsmodule.c](https://github.com/python/cpython/blob/main/Modules/_codecsmodule.c): 43 sites
[Modules/sha1module.c](https://github.com/python/cpython/blob/main/Modules/sha1module.c): 2 sites
[Modules/sha256module.c](https://github.com/python/cpython/blob/main/Modules/sha256module.c): 3 sites
[Modules/sha512module.c](https://github.com/python/cpython/blob/main/Modules/sha512module.c): 3 sites
[Modules/md5module.c](https://github.com/python/cpython/blob/main/Modules/md5module.c): 2 sites

bpo-20174
Derby #5: Convert 50 sites to Argument Clinic across 3 files

[Modules/socketmodule.c](https://github.com/python/cpython/blob/main/Modules/socketmodule.c): 47 sites
[Modules/socketmodule.h](https://github.com/python/cpython/blob/main/Modules/socketmodule.h): 1 sites
[Modules/_functoolsmodule.c](https://github.com/python/cpython/blob/main/Modules/_functoolsmodule.c): 2 sites

bpo-20175
Derby #6: Convert 50 sites to Argument Clinic across 8 files

[Modules/_io/bufferedio.c](https://github.com/python/cpython/blob/main/Modules/_io/bufferedio.c): 13 sites
[Modules/_io/bytesio.c](https://github.com/python/cpython/blob/main/Modules/_io/bytesio.c): 6 sites
[Modules/_io/fileio.c](https://github.com/python/cpython/blob/main/Modules/_io/fileio.c): 6 sites
[Modules/_io/iobase.c](https://github.com/python/cpython/blob/main/Modules/_io/iobase.c): 4 sites
[Modules/_io/stringio.c](https://github.com/python/cpython/blob/main/Modules/_io/stringio.c): 5 sites
[Modules/_io/textio.c](https://github.com/python/cpython/blob/main/Modules/_io/textio.c): 8 sites
[Modules/_multiprocessing/multiprocessing.c](https://github.com/python/cpython/blob/main/Modules/_multiprocessing/multiprocessing.c): 3 sites
[Modules/_multiprocessing/semaphore.c](https://github.com/python/cpython/blob/main/Modules/_multiprocessing/semaphore.c): 5 sites

bpo-20159
Derby #7: Convert 51 sites to Argument Clinic across 3 files

[Modules/_elementtree.c](https://github.com/python/cpython/blob/main/Modules/_elementtree.c): 31 sites
[PC/_msi.c](https://github.com/python/cpython/blob/main/PC/_msi.c): 15 sites
[PC/bdist_wininst/install.c](https://github.com/python/cpython/blob/main/PC/bdist_wininst/install.c): 5 sites

bpo-20177
Derby #8: Convert 52 sites to Argument Clinic across 3 files

[Modules/_datetimemodule.c](https://github.com/python/cpython/blob/main/Modules/_datetimemodule.c): 20 sites
[Modules/timemodule.c](https://github.com/python/cpython/blob/main/Modules/timemodule.c): 8 sites
[Modules/_decimal/_decimal.c](https://github.com/python/cpython/blob/main/Modules/_decimal/_decimal.c): 24 sites

bpo-20178
Derby #9: Convert 52 sites to Argument Clinic across 11 files

[Modules/_ctypes/_ctypes.c](https://github.com/python/cpython/blob/main/Modules/_ctypes/_ctypes.c): 8 sites
[Modules/_ctypes/_ctypes_test.c](https://github.com/python/cpython/blob/main/Modules/_ctypes/_ctypes_test.c): 1 sites
[Modules/_ctypes/callproc.c](https://github.com/python/cpython/blob/main/Modules/_ctypes/callproc.c): 14 sites
[Modules/_ctypes/stgdict.c](https://github.com/python/cpython/blob/main/Modules/_ctypes/stgdict.c): 0 sites
[Modules/_curses_panel.c](https://github.com/python/cpython/blob/main/Modules/_curses_panel.c): 3 sites
[Modules/_sqlite/cache.c](https://github.com/python/cpython/blob/main/Modules/_sqlite/cache.c): 1 sites
[Modules/_sqlite/connection.c](https://github.com/python/cpython/blob/main/Modules/_sqlite/connection.c): 12 sites
[Modules/_sqlite/cursor.c](https://github.com/python/cpython/blob/main/Modules/_sqlite/cursor.c): 5 sites
[Modules/_sqlite/microprotocols.c](https://github.com/python/cpython/blob/main/Modules/_sqlite/microprotocols.c): 1 sites
[Modules/_sqlite/module.c](https://github.com/python/cpython/blob/main/Modules/_sqlite/module.c): 6 sites
[Modules/_sqlite/row.c](https://github.com/python/cpython/blob/main/Modules/_sqlite/row.c): 1 sites

bpo-20179
Derby #10: Convert 50 sites to Argument Clinic across 4 files

[Modules/overlapped.c](https://github.com/python/cpython/blob/main/Modules/overlapped.c): 21 sites
[Modules/_ssl.c](https://github.com/python/cpython/blob/main/Modules/_ssl.c): 20 sites
[Objects/bytes_methods.c](https://github.com/python/cpython/blob/main/Objects/bytes_methods.c): 1 sites
[Objects/bytesobject.c](https://github.com/python/cpython/blob/main/Objects/bytesobject.c): 8 sites

bpo-20180
Derby #11: Convert 50 sites to Argument Clinic across 9 files

[Modules/_collectionsmodule.c](https://github.com/python/cpython/blob/main/Modules/_collectionsmodule.c): 4 sites
[Modules/itertoolsmodule.c](https://github.com/python/cpython/blob/main/Modules/itertoolsmodule.c): 11 sites
[Modules/_randommodule.c](https://github.com/python/cpython/blob/main/Modules/_randommodule.c): 1 sites
[Objects/unicodeobject.c](https://github.com/python/cpython/blob/main/Objects/unicodeobject.c): 14 sites
[Modules/xxlimited.c](https://github.com/python/cpython/blob/main/Modules/xxlimited.c): 4 sites
[Modules/xxmodule.c](https://github.com/python/cpython/blob/main/Modules/xxmodule.c): 5 sites
[Modules/xxsubtype.c](https://github.com/python/cpython/blob/main/Modules/xxsubtype.c): 5 sites
[Objects/stringlib/find.h](https://github.com/python/cpython/blob/main/Objects/stringlib/find.h): 1 sites
[Objects/stringlib/transmogrify.h](https://github.com/python/cpython/blob/main/Objects/stringlib/transmogrify.h): 5 sites

bpo-20181
Derby #12: Convert 50 sites to Argument Clinic across 4 files

[Objects/bytearrayobject.c](https://github.com/python/cpython/blob/main/Objects/bytearrayobject.c): 13 sites
[Modules/parsermodule.c](https://github.com/python/cpython/blob/main/Modules/parsermodule.c): 13 sites
[Modules/unicodedata.c](https://github.com/python/cpython/blob/main/Modules/unicodedata.c): 12 sites
[Modules/readline.c](https://github.com/python/cpython/blob/main/Modules/readline.c): 12 sites

bpo-20182
Derby #13: Convert 50 sites to Argument Clinic across 5 files

[Modules/signalmodule.c](https://github.com/python/cpython/blob/main/Modules/signalmodule.c): 12 sites
[Modules/selectmodule.c](https://github.com/python/cpython/blob/main/Modules/selectmodule.c): 12 sites
[Modules/zlibmodule.c](https://github.com/python/cpython/blob/main/Modules/zlibmodule.c): 11 sites
[Python/sysmodule.c](https://github.com/python/cpython/blob/main/Python/sysmodule.c): 10 sites
[Modules/_hashopenssl.c](https://github.com/python/cpython/blob/main/Modules/_hashopenssl.c): 5 sites

bpo-20183
Derby #14: Convert 50 sites to Argument Clinic across 6 files

[Modules/mmapmodule.c](https://github.com/python/cpython/blob/main/Modules/mmapmodule.c): 10 sites
[Modules/_localemodule.c](https://github.com/python/cpython/blob/main/Modules/_localemodule.c): 10 sites
[Modules/zipimport.c](https://github.com/python/cpython/blob/main/Modules/zipimport.c): 9 sites
[Modules/ossaudiodev.c](https://github.com/python/cpython/blob/main/Modules/ossaudiodev.c): 9 sites
[Modules/_testbuffer.c](https://github.com/python/cpython/blob/main/Modules/_testbuffer.c): 9 sites
[Modules/_struct.c](https://github.com/python/cpython/blob/main/Modules/_struct.c): 3 sites

bpo-20152
Derby #15: Convert 50 sites to Argument Clinic across 9 files

[Python/import.c](https://github.com/python/cpython/blob/main/Python/import.c): 8 sites
[Modules/cmathmodule.c](https://github.com/python/cpython/blob/main/Modules/cmathmodule.c): 8 sites
[Modules/cjkcodecs/multibytecodec.c](https://github.com/python/cpython/blob/main/Modules/cjkcodecs/multibytecodec.c): 8 sites
[Modules/arraymodule.c](https://github.com/python/cpython/blob/main/Modules/arraymodule.c): 8 sites
[Modules/pyexpat.c](https://github.com/python/cpython/blob/main/Modules/pyexpat.c): 7 sites
[Modules/fcntlmodule.c](https://github.com/python/cpython/blob/main/Modules/fcntlmodule.c): 7 sites
[Modules/pwdmodule.c](https://github.com/python/cpython/blob/main/Modules/pwdmodule.c): 2 sites
[Modules/spwdmodule.c](https://github.com/python/cpython/blob/main/Modules/spwdmodule.c): 1 sites
[Modules/grpmodule.c](https://github.com/python/cpython/blob/main/Modules/grpmodule.c): 1 sites

bpo-20184
Derby #16: Convert 50 sites to Argument Clinic across 9 files

[Modules/faulthandler.c](https://github.com/python/cpython/blob/main/Modules/faulthandler.c): 7 sites
[Modules/_pickle.c](https://github.com/python/cpython/blob/main/Modules/_pickle.c): 7 sites
[Modules/_lzmamodule.c](https://github.com/python/cpython/blob/main/Modules/_lzmamodule.c): 7 sites
[Python/bltinmodule.c](https://github.com/python/cpython/blob/main/Python/bltinmodule.c): 6 sites
[Modules/termios.c](https://github.com/python/cpython/blob/main/Modules/termios.c): 6 sites
[Modules/syslogmodule.c](https://github.com/python/cpython/blob/main/Modules/syslogmodule.c): 6 sites
[Modules/_dbmmodule.c](https://github.com/python/cpython/blob/main/Modules/_dbmmodule.c): 4 sites
[Modules/_gdbmmodule.c](https://github.com/python/cpython/blob/main/Modules/_gdbmmodule.c): 2 sites
[Modules/_json.c](https://github.com/python/cpython/blob/main/Modules/_json.c): 5 sites

bpo-20185
Derby #17: Convert 50 sites to Argument Clinic across 14 files

[Objects/typeobject.c](https://github.com/python/cpython/blob/main/Objects/typeobject.c): 5 sites
[Objects/longobject.c](https://github.com/python/cpython/blob/main/Objects/longobject.c): 5 sites
[Objects/listobject.c](https://github.com/python/cpython/blob/main/Objects/listobject.c): 5 sites
[Objects/floatobject.c](https://github.com/python/cpython/blob/main/Objects/floatobject.c): 4 sites
[Modules/resource.c](https://github.com/python/cpython/blob/main/Modules/resource.c): 4 sites
[Modules/_threadmodule.c](https://github.com/python/cpython/blob/main/Modules/_threadmodule.c): 4 sites
[Modules/_bz2module.c](https://github.com/python/cpython/blob/main/Modules/_bz2module.c): 4 sites
[Modules/_bisectmodule.c](https://github.com/python/cpython/blob/main/Modules/_bisectmodule.c): 4 sites
[Python/marshal.c](https://github.com/python/cpython/blob/main/Python/marshal.c): 3 sites
[Objects/exceptions.c](https://github.com/python/cpython/blob/main/Objects/exceptions.c): 3 sites
[Modules/nismodule.c](https://github.com/python/cpython/blob/main/Modules/nismodule.c): 3 sites
[Modules/gcmodule.c](https://github.com/python/cpython/blob/main/Modules/gcmodule.c): 3 sites
[Python/_warnings.c](https://github.com/python/cpython/blob/main/Python/_warnings.c): 2 sites
[Modules/_cryptmodule.c](https://github.com/python/cpython/blob/main/Modules/_cryptmodule.c): 1 sites

bpo-20186
Derby #18: Convert 31 sites to Argument Clinic across 23 files

[Objects/tupleobject.c](https://github.com/python/cpython/blob/main/Objects/tupleobject.c): 2 sites
[Objects/memoryobject.c](https://github.com/python/cpython/blob/main/Objects/memoryobject.c): 2 sites
[Objects/descrobject.c](https://github.com/python/cpython/blob/main/Objects/descrobject.c): 2 sites
[Objects/complexobject.c](https://github.com/python/cpython/blob/main/Objects/complexobject.c): 2 sites
[Modules/_operator.c](https://github.com/python/cpython/blob/main/Modules/_operator.c): 2 sites
[Modules/_opcode.c](https://github.com/python/cpython/blob/main/Modules/_opcode.c): 2 sites
[Modules/_lsprof.c](https://github.com/python/cpython/blob/main/Modules/_lsprof.c): 2 sites
[Modules/_heapqmodule.c](https://github.com/python/cpython/blob/main/Modules/_heapqmodule.c): 2 sites
[Objects/weakrefobject.c](https://github.com/python/cpython/blob/main/Objects/weakrefobject.c): 1 sites
[Objects/structseq.c](https://github.com/python/cpython/blob/main/Objects/structseq.c): 1 sites
[Objects/rangeobject.c](https://github.com/python/cpython/blob/main/Objects/rangeobject.c): 1 sites
[Objects/object.c](https://github.com/python/cpython/blob/main/Objects/object.c): 1 sites
[Objects/moduleobject.c](https://github.com/python/cpython/blob/main/Objects/moduleobject.c): 1 sites
[Objects/funcobject.c](https://github.com/python/cpython/blob/main/Objects/funcobject.c): 1 sites
[Objects/fileobject.c](https://github.com/python/cpython/blob/main/Objects/fileobject.c): 1 sites
[Objects/enumobject.c](https://github.com/python/cpython/blob/main/Objects/enumobject.c): 1 sites
[Objects/codeobject.c](https://github.com/python/cpython/blob/main/Objects/codeobject.c): 1 sites
[Objects/boolobject.c](https://github.com/python/cpython/blob/main/Objects/boolobject.c): 1 sites
[Modules/symtablemodule.c](https://github.com/python/cpython/blob/main/Modules/symtablemodule.c): 1 sites
[Modules/mathmodule.c](https://github.com/python/cpython/blob/main/Modules/mathmodule.c): 1 sites
[Modules/_tracemalloc.c](https://github.com/python/cpython/blob/main/Modules/_tracemalloc.c): 1 sites
[Modules/_io/_iomodule.c](https://github.com/python/cpython/blob/main/Modules/_io/_iomodule.c): 1 sites
[Modules/_csv.c](https://github.com/python/cpython/blob/main/Modules/_csv.c): 1 sites

[serhiy-storchaka]

I want to warn against the converting of base objects (whose sources lie in Objects/). Their performance is critical and the converting can worsen it. Often they even don't use PyArg_ParseTuple due to performance reasons or because argument parsing is too special for PyArg_ParseTuple. I would suggest to postpone it until Argument Clinic will generate optimized parsing code.

The same is true for some modules. The performance of struct and operator is important. In any case these modules almost not use PyArg_ParseTuple and can be omitted.

[larryhastings]

Can you give me an example of performance degradation due to Argument Clinic? What percentage slowdown is common?

[serhiy-storchaka]

I have no any numbers.

[larryhastings]

Until you have some numbers, your assertion has no weight and should be ignored.

(As Carl Sagan once wrote: "That which can be asserted without evidence can be dismissed without evidence." And as Donald Knuth wrote: "Premature optimization is the root of all evil.")

[taleinat]

I've been working on converting unicode, bytes and bytearray, but their files are separated into Derby groups #10, #11 and #12. Could we create a new issue and move the relevant files to it?

The files I'd like to move:

From Derby #10 (bpo-20179):
Objects/bytes_methods.c
Objects/bytesobject.c

From Derby #11 (bpo-20180):
Objects/unicodeobject.c
Objects/stringlib/find.h
Objects/stringlib/transmogrify.h

From Derby #12 (bpo-20181):
Objects/bytearrayobject.c: 13 sites

[taleinat]

Never mind my previous comment; I realize it would create to much of a mess. Sorry for the noise.

[AA-Turner]

@larryhastings Is this issue still relevant? All the AC PRs listed in the header are either merged or closed.

A