PyType_FromSpec should take metaclass as an argument

[abalkin]

BPO	15870
Nosy	@loewis, @jcea, @amauryfa, @abalkin, @pitrou, @encukou, @lekma, @mattip, @zooba, @seberg, @ctismer
Files	typeobject.diff typeobject.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/abalkin'
closed_at = None
created_at = <Date 2012-09-06.15:49:25.705>
labels = ['interpreter-core', 'type-feature', '3.11']
title = 'PyType_FromSpec should take metaclass as an argument'
updated_at = <Date 2021-10-12.13:28:38.470>
user = 'https://github.com/abalkin'

bugs.python.org fields:

activity = <Date 2021-10-12.13:28:38.470>
actor = 'petr.viktorin'
assignee = 'belopolsky'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2012-09-06.15:49:25.705>
creator = 'belopolsky'
dependencies = []
files = ['27137', '50285']
hgrepos = []
issue_num = 15870
keywords = ['patch', 'needs review']
message_count = 40.0
messages = ['169925', '169928', '169929', '169940', '169942', '169943', '169944', '169951', '169955', '169972', '169977', '398430', '398435', '398751', '398776', '401642', '401651', '401781', '402105', '402511', '402517', '402520', '402521', '402522', '402527', '402551', '402729', '402733', '402734', '402735', '402737', '402738', '402751', '402753', '402793', '402800', '402806', '403231', '403257', '403730']
nosy_count = 16.0
nosy_names = ['loewis', 'jcea', 'amaury.forgeotdarc', 'belopolsky', 'pitrou', 'Arfrever', 'petr.viktorin', 'lekma', 'Alexander.Belopolsky', 'mattip', 'Robin.Schreiber', 'steve.dower', 'seberg', 'Christian.Tismer', 'jhaberman', 'haberman2']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue15870'
versions = ['Python 3.11']

[abalkin]

PyType_FromSpec() is a convenient function to create types dynamically in C extension modules, but its usefulness is limited by the fact that it creates new types using the default metaclass.

I suggest adding a new C API function

PyObject *PyType_FromSpecEx(PyObject *meta, PyType_Spec *spec)

and redefine PyType_FromSpec() as

PyType_FromSpecEx((PyObject *)&PyType_Type, spec)

This functionality cannot be implemented by user because PyType_FromSpec requires access to private slotoffsets table.

A (trivial) patch attached.

[amauryfa]

The patch is a bit light: see how type_new also computes the metaclass from the base classes.

[AlexanderBelopolsky]

On Thu, Sep 6, 2012 at 12:44 PM, Amaury Forgeot d'Arc
<report@bugs.python.org> wrote:

The patch is a bit light: see how type_new also computes the metaclass from the base classes.

This was intentional. I was looking for a lightweight facility to
create heap types. I know metaclass when I call PyType_FromSpec. If
i wanted to invoke the "metaclass from the base classes" logic, I
would just specify an appropriate base class in the spec. This would
still leave an open problem of specifying the metatype for the most
basic class. This is the problem I am trying to solve.

[abalkin]

see how type_new also computes the metaclass from the base classes.

As you can see from my first message, I originally considered PyType_FromSpecEx(PyObject *meta, PyType_Spec *spec) without bases. (In fact I was unaware of the recent addition of PyType_FromSpecWithBases.) Maybe the original signature makes more sense than the one in the patch. Explicitly setting a metaclass is most useful for the most basic type. On the other hand, a fully general function may eventually replace both PyType_FromSpec and PyType_FromSpecWithBases for most uses.

[loewis]

What is your use case for this API?

[AlexanderBelopolsky]

On Sep 6, 2012, at 5:10 PM, Martin v. Löwis <report@bugs.python.org> wrote:

What is your use case for this API?

I can describe my use case, but it is somewhat similar to ctypes. I searched the tracker for a PEP-3121 refactoring applied to ctypes and could not find any. I'll try to come up with a PEP-3121 patch for ctypes using the proposed API.

[loewis]

If it's very special, I'm -0 on this addition. This sounds like this is something very few people would ever need, and they can learn to write more complicated code to achieve the same effect. Convenience API exists to make the common case convenient.

I'm -1 on calling it PyType_FromSpecEx.

[AlexanderBelopolsky]

On Sep 6, 2012, at 6:25 PM, Martin v. Löwis <report@bugs.python.org> wrote:

I'm -1 on calling it PyType_FromSpecEx.

I find it encouraging that you commented on the choice of name. :-) I can live with PyType_FromMetatypeAndSpec and leave out bases. PyType_FromTypeAndSpec is fine too.

On the substance, I don't think this API is just convenience. In my application I have to replace meta type after my type is created with PyType_FromSpec. This is fragile and works only for very simple metatypes.

Let's get back to this discussion once I have a ctypes patch. I there will be a work-around for ctypes it will probably work for my case. (My case is a little bit more complicated because I extend the size of my type objects to store custom metadata. Ctypes fudge this issue by hiding extra data in a custom tp_dict. )

[pitrou]

This API may make it easier to declare ABCs in C.

[loewis]

As for declaring ABCs: I don't think the API is necessary, or even helps. An ABC is best created by *calling* ABCMeta, with the appropriate name, a possibly-empty bases tuple, and a dict. What FromSpec could do is to fill out slots with custom functions, which won't be necessary or desirable for ABCs. The really tedious part may be to put all the abstract methods into the ABC, for which having a TypeSpec doesn't help at all. (But I would certainly agree that simplifying creation of ABCs in extension modules is a worthwhile reason for an API addition)

For the case that Alexander apparently envisions, i.e. metaclasses where the resulting type objects extend the layout of heap types: it should be possible for an extension module to fill out the entire type "from scratch". This will require knowledge of the layout of heap types, so it can't use just the stable ABI - however, doing this through the stable ABI won't be possible, anyway, since the extended layout needs to know how large a HeapType structure is.

If filling out a type with all slots one-by-one is considered too tedious, and patching ob_type too hacky - here is another approach: Use FromSpec to create a type with all slots filled out, then call the metatype to create a subtype of that. I.e. the type which is based on a metatype would actually be a derived class of the type which has the slots defined.

[pitrou]

If filling out a type with all slots one-by-one is considered too
tedious, and patching ob_type too hacky - here is another approach:
Use FromSpec to create a type with all slots filled out, then call the
metatype to create a subtype of that. I.e. the type which is based on
a metatype would actually be a derived class of the type which has the
slots defined.

As a matter of fact, this is what the io module is doing (except that
the derived type is written in Python). It does feel like pointless
complication, though.