subclasses of pathlib.PurePosixPath never call __init__ or __new__

[conchylicultor]

BPO	41109
Nosy	@pitrou, @websurfer5, @Conchylicultor, @Louis-Vincent
PRs	bpo-41109: subclasses of pathlib.Path and pathlib.PurePath now call the subclass's __init__() and __new__() functions when returning new objects #21920

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-06-25.02:51:23.231>
labels = ['type-bug', '3.8', '3.9', '3.10', '3.7', 'library']
title = 'subclasses of pathlib.PurePosixPath never call __init__ or __new__'
updated_at = <Date 2020-08-19.06:37:57.142>
user = 'https://github.com/conchylicultor'

bugs.python.org fields:

activity = <Date 2020-08-19.06:37:57.142>
actor = 'Jeffrey.Kintscher'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2020-06-25.02:51:23.231>
creator = 'conchylicultor'
dependencies = []
files = []
hgrepos = []
issue_num = 41109
keywords = ['patch']
message_count = 12.0
messages = ['372297', '372298', '375030', '375031', '375041', '375047', '375162', '375332', '375333', '375334', '375364', '375638']
nosy_count = 4.0
nosy_names = ['pitrou', 'Jeffrey.Kintscher', 'conchylicultor', 'louis-vincent.boudre']
pr_nums = ['21920']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue41109'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10']

[conchylicultor]

I have a subclass GithubPath of PurePosixPath.

class GithubPath(pathlib.PurePosixPath):

 def __new__(cls, *args, **kwargs):
 print('New')
 return super().__new__(cls, *args, **kwargs)

 def __init__(self, *args, **kwargs):
 print('Init')
 super().__init__()

Calling child.parent create a new GithubPath but without ever calling new nor init. So my subclass is never notified it is created.

p = GithubPath() # Print "New", "Init"

p.parent # Create a new GithubPath but bypass the constructors

The reason seems to be that parent calls _from_parts which create a new object through object.__new__(cls):

cpython/Lib/pathlib.py

Line 689 in cf18c9e

def _from_parts(cls, args, init=True):

A hack is to subclass _init but it seems hacky as it relies on internal implementation detail.

[conchylicultor]

Note that this likely affect all methods which returns new Path by calling _from_parts or _from_parsed_parts, like .absolute, .resolve,...

[websurfer5]

The workaround is to override _init(), which I agree is not desirable.

This is the relevant code in PurePath, which is the super class of PurePosixPath:

 @classmethod
 def _from_parsed_parts(cls, drv, root, parts, init=True):
 self = object.__new__(cls)
 self._drv = drv
 self._root = root
 self._parts = parts
 if init:
 self._init()
 return self

...

 def _init(self):
 # Overridden in concrete Path
 pass

To me, the clean way to get the desired behavior seems like it would be to have _init() call self.__init__().

 def _init(self):
 # Overridden in concrete Path
 self.__init__()

This fixes p.parent, but GithubPath() ends up calling GithubPath.__init__() twice – the first time by PurePath.__new__() calling PurePath._init() and the second time by the GithubPath object creation.

[websurfer5]

Clarification:

PurePath.__new__() calls PurePath._from_parts(), which then calls PurePath._init()

[conchylicultor]

Before solving this issue, I think it would be best to think on a more generic solution on how to make Pathlib more extensible. Related to: https://discuss.python.org/t/make-pathlib-extensible/3428

For instance, if childs created with p.parent(), p / 'subdir' need to forward some state (e.g. RemotePath(path, password=, user=)).

Rather than __init__, maybe there should be some __post_init__ like dataclasses.