Add a --rerun option to the iOS testbed.

[freakboy3742]

Adds a --rerun option to the iOS testbed runner.

We've seen a small number of transient failures in the iOS buildbot - mostly in socket-related tests, due to socket availability. These tests pass on the next test, and can't be easily reproduced. The test runner has a --rerun option that may help with the resilience of these tests, and is enabled in the --slow/fast-ci configurations; this PR enables that option for the iOS test runner.

[freakboy3742]

!buildbot iOS

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @freakboy3742 for commit f9cca3e 🤖

The command will test the builders whose names match following regular expression: iOS

The builders matched are:

• iOS ARM64 Simulator PR

[ned-deily]

A good option to use

[miss-islington-app]

Thanks @freakboy3742 for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

[bedevere-app]

GH-122993 is a backport of this pull request to the 3.13 branch.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot iOS ARM64 Simulator 3.x has failed when building commit ce39d3c.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#builders/1380/builds/1068) and take a look at the build logs.
4. Check if the failure is related to this commit (ce39d3c) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#builders/1380/builds/1068

Failed tests:

• test_urllib2net

Failed subtests:

• test_ftp_timeout - test.test_urllib2net.TimeoutTest.test_ftp_timeout
• test_ftp - test.test_urllib2net.OtherNetworkTests.test_ftp
• test_ftp_no_timeout - test.test_urllib2net.TimeoutTest.test_ftp_no_timeout
• test_ftp_basic - test.test_urllib2net.TimeoutTest.test_ftp_basic
• test_ftp_default_timeout - test.test_urllib2net.TimeoutTest.test_ftp_default_timeout

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last): File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/support/socket_helper.py", line 249, in transient_internetyield File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 373, in test_ftp_timeout u = _urlopen_with_retry(self.FTP_HOST, timeout=60) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 29, in wrappedreturn _retry_thrice(func, exc, *args, **kwargs) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 25, in _retry_thriceraise last_exc File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 21, in _retry_thricereturn func(*args, **kwargs) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 189, in urlopenreturn opener.open(url, data, timeout) ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 489, in open response =self._open(req, data) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 506, in _open result =self._call_chain(self.handle_open, protocol, protocol +'_open', req) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 466, in _call_chain result = func(*args) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 1558, in ftp_openraise URLError(exp) from exp urllib.error.URLError: <urlopen error 500 OOPS: vsf_sysutil_bind> Traceback (most recent call last): Warning -- File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/libregrtest/run_workers.py", line 399, in run Warning -- mp_result = self._runtest(test_name) Warning -- File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/libregrtest/run_workers.py", line 359, in _runtest Warning -- retcode, tmp_files = self.run_tmp_files(worker_runtests, Warning -- ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ Warning -- stdout_file.fileno()) Warning -- ^^^^^^^^^^^^^^^^^^^^^ Warning -- File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/libregrtest/run_workers.py", line 295, in run_tmp_files Warning -- retcode = self._run_process(worker_runtests, Warning -- stdout_fd, tmp_dir) Warning -- File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/libregrtest/run_workers.py", line 177, in _run_process Warning -- popen = create_worker_process(runtests, output_fd, tmp_dir) Warning -- File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/libregrtest/worker.py", line 64, in create_worker_process Warning -- return subprocess.Popen(cmd, **kwargs) Warning -- ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^ Warning -- File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/subprocess.py", line 824, in __init__ Warning -- raise OSError( Warning -- errno.ENOTSUP, f"{sys.platform} does not support processes." Warning -- ) Warning -- OSError: [Errno 45] ios does not support processes. Traceback (most recent call last): File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 1548, in ftp_open fp, retrlen = fw.retrfile(file, type) ~~~~~~~~~~~^^^^^^^^^^^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 2443, in retrfile conn, retrlen =self.ftp.ntransfercmd(cmd) ~~~~~~~~~~~~~~~~~~~~~^^^^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/ftplib.py", line 353, in ntransfercmd host, port =self.makepasv() ~~~~~~~~~~~~~^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/ftplib.py", line 327, in makepasv untrusted_host, port = parse227(self.sendcmd('PASV')) ~~~~~~~~~~~~^^^^^^^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/ftplib.py", line 281, in sendcmdreturnself.getresp() ~~~~~~~~~~~~^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/ftplib.py", line 254, in getrespraise error_perm(resp) ftplib.error_perm: 500 OOPS: vsf_sysutil_bind Traceback (most recent call last): File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 1548, in ftp_open fp, retrlen = fw.retrfile(file, type) ~~~~~~~~~~~^^^^^^^^^^^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 2426, in retrfileraise URLError(f'ftp error: {reason}') from reason urllib.error.URLError: <urlopen error ftp error: 500 OOPS: vsf_sysutil_bind> Traceback (most recent call last): File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/support/socket_helper.py", line 249, in transient_internetyield File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 364, in test_ftp_no_timeout u = _urlopen_with_retry(self.FTP_HOST, timeout=None) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 29, in wrappedreturn _retry_thrice(func, exc, *args, **kwargs) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 25, in _retry_thriceraise last_exc File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 21, in _retry_thricereturn func(*args, **kwargs) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 189, in urlopenreturn opener.open(url, data, timeout) ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 489, in open response =self._open(req, data) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 506, in _open result =self._call_chain(self.handle_open, protocol, protocol +'_open', req) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 466, in _call_chain result = func(*args) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 1558, in ftp_openraise URLError(exp) from exp urllib.error.URLError: <urlopen error 500 OOPS: vsf_sysutil_bind> Traceback (most recent call last): File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 372, in test_ftp_timeoutwith socket_helper.transient_internet(self.FTP_HOST): ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/contextlib.py", line 162, in __exit__self.gen.throw(value) ~~~~~~~~~~~~~~^^^^^^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/support/socket_helper.py", line 264, in transient_internet filter_error(err) ~~~~~~~~~~~~^^^^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/support/socket_helper.py", line 237, in filter_error (("ConnectionRefusedError"in err.reason) or^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^TypeError: argument of type 'error_perm' is not a container or iterable Traceback (most recent call last): File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 261, in _test_urls f = urlopen(url, req, support.INTERNET_TIMEOUT) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 29, in wrappedreturn _retry_thrice(func, exc, *args, **kwargs) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 25, in _retry_thriceraise last_exc File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 21, in _retry_thricereturn func(*args, **kwargs) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 489, in open response =self._open(req, data) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 506, in _open result =self._call_chain(self.handle_open, protocol, protocol +'_open', req) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 466, in _call_chain result = func(*args) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 1558, in ftp_openraise URLError(exp) from exp urllib.error.URLError: <urlopen error <urlopen error ftp error: 500 OOPS: vsf_sysutil_bind>> Traceback (most recent call last): File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/support/socket_helper.py", line 249, in transient_internetyield File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 343, in test_ftp_basic u = _urlopen_with_retry(self.FTP_HOST) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 29, in wrappedreturn _retry_thrice(func, exc, *args, **kwargs) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 25, in _retry_thriceraise last_exc File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 21, in _retry_thricereturn func(*args, **kwargs) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 189, in urlopenreturn opener.open(url, data, timeout) ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 489, in open response =self._open(req, data) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 506, in _open result =self._call_chain(self.handle_open, protocol, protocol +'_open', req) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 466, in _call_chain result = func(*args) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 1558, in ftp_openraise URLError(exp) from exp urllib.error.URLError: <urlopen error 500 OOPS: vsf_sysutil_bind> Traceback (most recent call last): File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/support/socket_helper.py", line 249, in transient_internetyield File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 352, in test_ftp_default_timeout u = _urlopen_with_retry(self.FTP_HOST) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 29, in wrappedreturn _retry_thrice(func, exc, *args, **kwargs) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 25, in _retry_thriceraise last_exc File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 21, in _retry_thricereturn func(*args, **kwargs) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 189, in urlopenreturn opener.open(url, data, timeout) ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 489, in open response =self._open(req, data) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 506, in _open result =self._call_chain(self.handle_open, protocol, protocol +'_open', req) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 466, in _call_chain result = func(*args) File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 1558, in ftp_openraise URLError(exp) from exp urllib.error.URLError: <urlopen error 500 OOPS: vsf_sysutil_bind> Traceback (most recent call last): File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/urllib/request.py", line 2423, in retrfile conn, retrlen =self.ftp.ntransfercmd(cmd) ~~~~~~~~~~~~~~~~~~~~~^^^^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/ftplib.py", line 353, in ntransfercmd host, port =self.makepasv() ~~~~~~~~~~~~~^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/ftplib.py", line 327, in makepasv untrusted_host, port = parse227(self.sendcmd('PASV')) ~~~~~~~~~~~~^^^^^^^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/ftplib.py", line 281, in sendcmdreturnself.getresp() ~~~~~~~~~~~~^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/ftplib.py", line 254, in getrespraise error_perm(resp) ftplib.error_perm: 500 OOPS: vsf_sysutil_bind Traceback (most recent call last): File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 349, in test_ftp_default_timeoutwith socket_helper.transient_internet(self.FTP_HOST): ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/contextlib.py", line 162, in __exit__self.gen.throw(value) ~~~~~~~~~~~~~~^^^^^^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/support/socket_helper.py", line 264, in transient_internet filter_error(err) ~~~~~~~~~~~~^^^^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/support/socket_helper.py", line 237, in filter_error (("ConnectionRefusedError"in err.reason) or^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^TypeError: argument of type 'error_perm' is not a container or iterable Traceback (most recent call last): File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 342, in test_ftp_basicwith socket_helper.transient_internet(self.FTP_HOST, timeout=None): ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/contextlib.py", line 162, in __exit__self.gen.throw(value) ~~~~~~~~~~~~~~^^^^^^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/support/socket_helper.py", line 264, in transient_internet filter_error(err) ~~~~~~~~~~~~^^^^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/support/socket_helper.py", line 237, in filter_error (("ConnectionRefusedError"in err.reason) or^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^TypeError: argument of type 'error_perm' is not a container or iterable Traceback (most recent call last): File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/test_urllib2net.py", line 361, in test_ftp_no_timeoutwith socket_helper.transient_internet(self.FTP_HOST): ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/contextlib.py", line 162, in __exit__self.gen.throw(value) ~~~~~~~~~~~~~~^^^^^^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/support/socket_helper.py", line 264, in transient_internet filter_error(err) ~~~~~~~~~~~~^^^^^ File "/Users/buildbot/Library/Developer/XCTestDevices/DEF0AAE3-414C-4A0D-A16C-D561C37F8D16/data/Containers/Bundle/Application/028C1970-6491-46B7-A9E9-5B8076EA3A32/iOSTestbed.app/python/lib/python3.14/test/support/socket_helper.py", line 237, in filter_error (("ConnectionRefusedError"in err.reason) or^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^TypeError: argument of type 'error_perm' is not a container or iterable

[ned-deily]

Oops, it looks like regrtest by default uses a subprocess for the reruns. https://buildbot.python.org/#builders/1380/builds/1068

[freakboy3742]

Whelp... I guess we need --single-process as well. Unfortunately, that option doesn't exist in 3.13, so we've either got to backport #119728, or live without --rerun on the 3.13 branch. Any preferences @ned-deily?

[ned-deily]

It would be a RM call; to me, backporting a feature in the release candidate phase of 3.13 seems a stretch. Possibly in a maintenance release. But I think the right thing to do now is to live without it for 3.13.0.

It does look like there is some useful info in the buildbot traceback: 500 OOPS: vsf_sysutil_bind.

[freakboy3742]

Makes sense. I've opened #122994 to add the --single-process option on main; I guess the soon-to-be-answered question is whether the problem is just a very inconvenient, new form of transient error with FTP tests, or an iOS problem to be fixed...

[freakboy3742]

Looks like the vsf_sysutil_bind error is transient... but it can also trigger on macOS

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE RHEL7 LTO + PGO 3.13 has failed when building commit bfec6f3.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#builders/1461/builds/212) and take a look at the build logs.
4. Check if the failure is related to this commit (bfec6f3) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#builders/1461/builds/212

Failed tests:

• test_urllib2net

Failed subtests:

• test_ftp_timeout - test.test_urllib2net.TimeoutTest.test_ftp_timeout
• test_ftp - test.test_urllib2net.OtherNetworkTests.test_ftp
• test_ftp_no_timeout - test.test_urllib2net.TimeoutTest.test_ftp_no_timeout
• test_ftp_basic - test.test_urllib2net.TimeoutTest.test_ftp_basic
• test_ftp_default_timeout - test.test_urllib2net.TimeoutTest.test_ftp_default_timeout

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last): File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 372, in test_ftp_timeoutwith socket_helper.transient_internet(self.FTP_HOST): ...<2 lines>...self.assertEqual(u.fp.fp.raw._sock.gettimeout(), 60) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/contextlib.py", line 162, in __exit__self.gen.throw(value) ~~~~~~~~~~~~~~^^^^^^^ File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/support/socket_helper.py", line 264, in transient_internet filter_error(err) ~~~~~~~~~~~~^^^^^ File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/support/socket_helper.py", line 237, in filter_error (("ConnectionRefusedError"in err.reason) or^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^TypeError: argument of type 'error_perm' is not iterable Traceback (most recent call last): File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 349, in test_ftp_default_timeoutwith socket_helper.transient_internet(self.FTP_HOST): ...<6 lines>...self.assertEqual(u.fp.fp.raw._sock.gettimeout(), 60) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/contextlib.py", line 162, in __exit__self.gen.throw(value) ~~~~~~~~~~~~~~^^^^^^^ File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/support/socket_helper.py", line 264, in transient_internet filter_error(err) ~~~~~~~~~~~~^^^^^ File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/support/socket_helper.py", line 237, in filter_error (("ConnectionRefusedError"in err.reason) or^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^TypeError: argument of type 'error_perm' is not iterable Traceback (most recent call last): File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 1548, in ftp_open fp, retrlen = fw.retrfile(file, type) ~~~~~~~~~~~^^^^^^^^^^^^ File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 2426, in retrfileraise URLError(f'ftp error: {reason}') from reason urllib.error.URLError: <urlopen error ftp error: 500 OOPS: vsf_sysutil_bind> Traceback (most recent call last): File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/support/socket_helper.py", line 249, in transient_internetyield File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 364, in test_ftp_no_timeout u = _urlopen_with_retry(self.FTP_HOST, timeout=None) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 29, in wrappedreturn _retry_thrice(func, exc, *args, **kwargs) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 25, in _retry_thriceraise last_exc File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 21, in _retry_thricereturn func(*args, **kwargs) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 189, in urlopenreturn opener.open(url, data, timeout) ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 489, in open response =self._open(req, data) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 506, in _open result =self._call_chain(self.handle_open, protocol, protocol +'_open', req) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 466, in _call_chain result = func(*args) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 1558, in ftp_openraise URLError(exp) from exp urllib.error.URLError: <urlopen error 500 OOPS: vsf_sysutil_bind> Traceback (most recent call last): File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 361, in test_ftp_no_timeoutwith socket_helper.transient_internet(self.FTP_HOST): ...<6 lines>...self.assertIsNone(u.fp.fp.raw._sock.gettimeout()) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/contextlib.py", line 162, in __exit__self.gen.throw(value) ~~~~~~~~~~~~~~^^^^^^^ File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/support/socket_helper.py", line 264, in transient_internet filter_error(err) ~~~~~~~~~~~~^^^^^ File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/support/socket_helper.py", line 237, in filter_error (("ConnectionRefusedError"in err.reason) or^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^TypeError: argument of type 'error_perm' is not iterable Traceback (most recent call last): File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 1548, in ftp_open fp, retrlen = fw.retrfile(file, type) ~~~~~~~~~~~^^^^^^^^^^^^ File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 2443, in retrfile conn, retrlen =self.ftp.ntransfercmd(cmd) ~~~~~~~~~~~~~~~~~~~~~^^^^^ File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/ftplib.py", line 353, in ntransfercmd host, port =self.makepasv() ~~~~~~~~~~~~~^^ File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/ftplib.py", line 327, in makepasv untrusted_host, port = parse227(self.sendcmd('PASV')) ~~~~~~~~~~~~^^^^^^^^ File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/ftplib.py", line 281, in sendcmdreturnself.getresp() ~~~~~~~~~~~~^^ File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/ftplib.py", line 254, in getrespraise error_perm(resp) ftplib.error_perm: 500 OOPS: vsf_sysutil_bind Traceback (most recent call last): File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 261, in _test_urls f = urlopen(url, req, support.INTERNET_TIMEOUT) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 29, in wrappedreturn _retry_thrice(func, exc, *args, **kwargs) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 25, in _retry_thriceraise last_exc File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 21, in _retry_thricereturn func(*args, **kwargs) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 489, in open response =self._open(req, data) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 506, in _open result =self._call_chain(self.handle_open, protocol, protocol +'_open', req) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 466, in _call_chain result = func(*args) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 1558, in ftp_openraise URLError(exp) from exp urllib.error.URLError: <urlopen error <urlopen error ftp error: 500 OOPS: vsf_sysutil_bind>> Traceback (most recent call last): File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/support/socket_helper.py", line 249, in transient_internetyield File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 373, in test_ftp_timeout u = _urlopen_with_retry(self.FTP_HOST, timeout=60) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 29, in wrappedreturn _retry_thrice(func, exc, *args, **kwargs) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 25, in _retry_thriceraise last_exc File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 21, in _retry_thricereturn func(*args, **kwargs) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 189, in urlopenreturn opener.open(url, data, timeout) ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 489, in open response =self._open(req, data) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 506, in _open result =self._call_chain(self.handle_open, protocol, protocol +'_open', req) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 466, in _call_chain result = func(*args) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 1558, in ftp_openraise URLError(exp) from exp urllib.error.URLError: <urlopen error 500 OOPS: vsf_sysutil_bind> Traceback (most recent call last): File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/support/socket_helper.py", line 249, in transient_internetyield File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 352, in test_ftp_default_timeout u = _urlopen_with_retry(self.FTP_HOST) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 29, in wrappedreturn _retry_thrice(func, exc, *args, **kwargs) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 25, in _retry_thriceraise last_exc File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 21, in _retry_thricereturn func(*args, **kwargs) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 189, in urlopenreturn opener.open(url, data, timeout) ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 489, in open response =self._open(req, data) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 506, in _open result =self._call_chain(self.handle_open, protocol, protocol +'_open', req) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 466, in _call_chain result = func(*args) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 1558, in ftp_openraise URLError(exp) from exp urllib.error.URLError: <urlopen error 500 OOPS: vsf_sysutil_bind> Traceback (most recent call last): File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 342, in test_ftp_basicwith socket_helper.transient_internet(self.FTP_HOST, timeout=None): ...<2 lines>...self.assertIsNone(u.fp.fp.raw._sock.gettimeout()) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/contextlib.py", line 162, in __exit__self.gen.throw(value) ~~~~~~~~~~~~~~^^^^^^^ File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/support/socket_helper.py", line 264, in transient_internet filter_error(err) ~~~~~~~~~~~~^^^^^ File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/support/socket_helper.py", line 237, in filter_error (("ConnectionRefusedError"in err.reason) or^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^TypeError: argument of type 'error_perm' is not iterable Traceback (most recent call last): File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/support/socket_helper.py", line 249, in transient_internetyield File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 343, in test_ftp_basic u = _urlopen_with_retry(self.FTP_HOST) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 29, in wrappedreturn _retry_thrice(func, exc, *args, **kwargs) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 25, in _retry_thriceraise last_exc File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_urllib2net.py", line 21, in _retry_thricereturn func(*args, **kwargs) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 189, in urlopenreturn opener.open(url, data, timeout) ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 489, in open response =self._open(req, data) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 506, in _open result =self._call_chain(self.handle_open, protocol, protocol +'_open', req) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 466, in _call_chain result = func(*args) File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 1558, in ftp_openraise URLError(exp) from exp urllib.error.URLError: <urlopen error 500 OOPS: vsf_sysutil_bind> Traceback (most recent call last): File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/urllib/request.py", line 2423, in retrfile conn, retrlen =self.ftp.ntransfercmd(cmd) ~~~~~~~~~~~~~~~~~~~~~^^^^^ File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/ftplib.py", line 353, in ntransfercmd host, port =self.makepasv() ~~~~~~~~~~~~~^^ File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/ftplib.py", line 327, in makepasv untrusted_host, port = parse227(self.sendcmd('PASV')) ~~~~~~~~~~~~^^^^^^^^ File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/ftplib.py", line 281, in sendcmdreturnself.getresp() ~~~~~~~~~~~~^^ File "/home/buildbot/buildarea/3.13.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/ftplib.py", line 254, in getrespraise error_perm(resp) ftplib.error_perm: 500 OOPS: vsf_sysutil_bind

[ned-deily]

These errors may be related to the recent OS upgrade on the PSF pythontest.net test server.

[mhsmith]

I had been planning to use --slow-ci in the Android buildbot for consistency with the other platforms, but that won't be possible unless the --single-process option is available as well. So I'll submit a backport of that option, even if it does have to wait until 3.13.1.

[miss-islington-app]

Thanks @freakboy3742 for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

[bedevere-app]

GH-123367 is a backport of this pull request to the 3.13 branch.