pathlib.Path._from_parsed_parts should call cls.__new__(cls)

[qb-cea]

BPO	32665
Nosy	@pitrou, @pablogsal, @qb-cea
Files	reproducer.py: reproducer (tested with python3.4)

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2018-01-25.12:27:25.104>
labels = ['type-bug', 'library']
title = 'pathlib.Path._from_parsed_parts should call cls.__new__(cls)'
updated_at = <Date 2018-01-27.21:38:03.770>
user = 'https://github.com/qb-cea'

bugs.python.org fields:

activity = <Date 2018-01-27.21:38:03.770>
actor = 'pablogsal'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2018-01-25.12:27:25.104>
creator = 'qb-cea'
dependencies = []
files = ['47409']
hgrepos = []
issue_num = 32665
keywords = []
message_count = 5.0
messages = ['310671', '310758', '310888', '310889', '310890']
nosy_count = 3.0
nosy_names = ['pitrou', 'pablogsal', 'qb-cea']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue32665'
versions = ['Python 3.6']

Linked PRs

• GH-76846, GH-85281: Call __new__() and __init__() on pathlib subclasses #102789

[qb-cea]

Hi,

I tried subclassing pathlib.Path and provide it with a new attribute (basically an accessor to an extended attribute). I am rather new to the concept of __slots__ and __new__() but here is how I pictured it should look:

from errno import ENODATA
from os import getxattr, setxattr
from pathlib import Path

class Path_(type(Path())):

 __slots__ = ("new_attr",)

 def __new__(cls, *args, new_attr=None, **kwargs):
 self = super().__new__(cls, *args, **kwargs)
 self._new_attr = new_attr
 return self

 @property
 def new_attr(self):
 if self._new_attr:
 return self._new_attr

 try:
 new_attr = getxattr(self, "user.new_attr")
 except OSError as exc:
 if exc.errno != ENODATA:
 raise exc
 else:
 self._new_attr = new_attr
 return new_attr

 new_attr = b"something_dynamic" # for example uuid4().bytes
 setxattr(self, "user.new_attr", new_attr)
 self._new_attr = new_attr
 return new_attr

The issue I have is that although my class defines its own __new__() method, it is not always called by the methods of pathlib.Path. For example:

>>> Path_("/etc").parent.new_attr
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/path/to/reproducer.py", line 19, in new_attr
 if self._new_attr:
AttributeError: _new_attr

The current workaround I use consists in redefining pathlib.Path's _from_parsed_parts() method in my class: instead of creating a new object using:
object.__new__(cls)
my implementation uses:
cls.__new__(cls)

This is the first time I play with the __new__() special method, so it is possible I missed something, if so, sorry for the noise.

[qb-cea]

Typo in the code of the previous comment:

• __slots__ = ("new_attr",)
  + __slots__ = ("_new_attr",)

[pablogsal]

This behaviour is because "parent" descriptor ends calling:

 @classmethod
 def _from_parsed_parts(cls, drv, root, parts, init=True):
 self = object.__new__(cls)
 self._drv = drv
 self._root = root
 self._parts = parts
 if init:
 self._init()
 return self

and this calls object.__new__ and this call raises AttributeError: new_attr. Notice that object.__new__(cls) will not raise as this snippet shows:

>>>: class A:
...: def __new__(*args):
...: raise ZeroDivisionError()
...:

>> A()
---------------------------------------------------------------------------

ZeroDivisionError Traceback (most recent call last)
<python> in <module>()
----> 1 A()

<python> in __new__(*args)
1 class A:
2 def __new__(*args):
----> 3 raise ZeroDivisionError()
4

ZeroDivisionError:

>> object.__new__(A)
>> <main.A at 0x7f6239c17860>

[pablogsal]

Sorry, the exception of object.__new__(cls) is AttributeError: _drv

[pablogsal]

Antoine, is the reason of calling object.new(cls) in _from_parsed_parts that the code does not raise on creation?

[barneygale]

#100481 removes _from_parsed_parts() and adds a supported route for subclassing, so would resolve this issue I believe.

[barneygale]

Fixed in Python 3.12 / #102789 / 11c3020