[3.11] gh-135034: Normalize link targets in tarfile, add os.path.realpath(strict='allow_missing') (GH-135037)

[Yhg1s]

Addresses CVEs 2024-12718, 2025-4138, 2025-4330, and 2025-4517.
(cherry picked from commit 3612d8f)

Co-authored-by: Łukasz Langa [email protected]
Co-authored-by: Petr Viktorin [email protected]
Co-authored-by: Seth Michael Larson [email protected]
Co-authored-by: Adam Turner [email protected]
Co-authored-by: Serhiy Storchaka [email protected]

• Issue: Multiple tarfile extraction filter bypasses (filter="tar"/filter="data") #135034

---

📚 Documentation preview 📚: https://cpython-previews--135068.org.readthedocs.build/

[kulikjak]

After this update, I am getting the following test failure:

====================================================================== ERROR: test_realpath_limit_attack (test.test_tarfile.TestExtractionFilters.test_realpath_limit_attack) [fully_trusted] ---------------------------------------------------------------------- Traceback (most recent call last): File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/test/test_tarfile.py", line 3669, in test_realpath_limit_attackwith (self.subTest('fully_trusted'), File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/contextlib.py", line 137, in __enter__returnnext(self.gen) ^^^^^^^^^^^^^^ File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/test/test_tarfile.py", line 3458, in check_contextself.expected_paths =set(self.outerdir.glob('**/*')) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 958, in globfor p in selector.select_from(self): File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 411, in _select_fromfor starting_point inself._iterate_directories(parent_path, is_dir, scandir): File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 401, in _iterate_directoriesfor p inself._iterate_directories(path, is_dir, scandir): File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 401, in _iterate_directoriesfor p inself._iterate_directories(path, is_dir, scandir): File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 401, in _iterate_directoriesfor p inself._iterate_directories(path, is_dir, scandir): [Previous line repeated 14 more times] File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 395, in _iterate_directories entry_is_dir = entry.is_dir(follow_symlinks=False) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^OSError: [Errno 78] File name too long: '/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/build/amd64/build/test_python_22904æ/@test_22904_tmpæ-tardir/outerdir/dest/ddddddddddddddddddd dddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/dddddddddddddddddd ddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddd dddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/dddddddddddddddd ddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/lllllllllllllll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll llllllllllllllllllllllllllllllll'

Interestingly, it's just 3.11.13 that's failing - both 3.9.23 and 3.13.4 (and all other supported versions we are running in our internal buildbot) pass. 3.13 has a very different pathlib implementation, but 3.9 is pretty similar so I investigated the differences, and when I print the entires from here (_iterate_directories 3.9 / 3.11):

 with scandir(parent_path) as scandir_it: entries = list(scandir_it) 

I get very different results:
3.9:

[<DirEntry 'newfile'>, <DirEntry 'flag'>, <DirEntry 'dest'>] [<DirEntry 'a'>, <DirEntry 'flaglink'>, <DirEntry 'escape'>] 

3.11:

[<DirEntry 'dest'>, <DirEntry 'flag'>, <DirEntry 'newfile'>] [<DirEntry 'a'>, <DirEntry 'escape'>, <DirEntry 'flaglink'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>] [<DirEntry 'b'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>] [<DirEntry 'c'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>] [<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'd'>] [<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'e'>] [<DirEntry 'f'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>] [<DirEntry 'g'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>] [<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'h'>] [<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'i'>] [<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'j'>] [<DirEntry 'k'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>] [<DirEntry 'l'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>] [<DirEntry 'm'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>] [<DirEntry 'n'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>] [<DirEntry 'o'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>] [<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'p'>] [<DirEntry 'lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllll'>] 

Also, the scandir argument is different:
3.9: <built-in function scandir>
3.11: <function Path._scandir at 0x7fed9bd923e0>
so I guess that those implementations behave differently (the 3.11 one returns much more entires), which causes the failure?

This is on Oracle Solaris.

[encukou]

Ah! We got the same test failure in 3.10, so I hotfixed it there. Try applying the hack to 3.11: dff62a1

It's just a test failure: if the filter is fully_trusted, this archive unpacks to a very interesting directory tree -- one that this version of Path.glob can't handle. And we call glob in the tests to get data for the assertions.

[kulikjak]

Oh, that fixed it. Thanks!

(i didn't realize that this difference between 3.9 and 3.11 patch is what causes the issue)

[DimNik9]

Hello, sorry to ask directly here but i haven't understood how 3.10 & 3.11 are affected by these CVEs, since the description of each CVE states that ONLY versions after 3.12 are affected, since the extraction filters were first introduced in 3.12
Thank you in advance

[encukou]

yeah, looks like the CVE text needs an update :(
cc @sethmlarson
The feature was backported to 3.8.17, 3.9.17, 3.10.12, 3.11.4

[sethmlarson]

@encukou Gotcha, I didn't realize that filtering had been backported, I'll update the prose description in each document. The affectedness of the CVEs is correct thankfully :)

[sethmlarson]

Updated the CVE records and sent a correction to [email protected].