Skip to content

Conversation

@barneygale
Copy link
Contributor

@barneygalebarneygale commented Jul 6, 2024

Handle errors from os.scandir() and ScandirIterator similarly, which lets us loop over directory entries with for.

$ ./python -m timeit -s 'import os''list(os.walk("."))' 5 loops, best of 5: 41.5 msec per loop 5 loops, best of 5: 40.1 msec per loop # --> 3.5% faster

Handle errors from `os.scandir()` and `ScandirIterator` similarly, which lets us loop over directory entries with `for`.
@barneygalebarneygale added the performance Performance or resource usage label Jul 6, 2024
@barneygalebarneygale changed the title GH-119186: Simplify os.walk() exception handlingGH-119169: Simplify os.walk() exception handlingJul 6, 2024
Copy link
Member

@serhiy-storchakaserhiy-storchaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

This is not so much optimization as simplification.

@barneygale
Copy link
ContributorAuthor

Thanks Serhiy

@barneygalebarneygale merged commit db00fee into python:mainJul 8, 2024
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL8 LTO + PGO 3.x has failed when building commit db00fee.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/568/builds/6911) and take a look at the build logs.
  4. Check if the failure is related to this commit (db00fee) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/568/builds/6911

Failed tests:

  • test_smtplib

Failed subtests:

  • test_with_statement_QUIT_failure - test.test_smtplib.SMTPSimTests.test_with_statement_QUIT_failure
  • test_with_statement - test.test_smtplib.SMTPSimTests.test_with_statement

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last): File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto-pgo/build/Lib/test/test_smtplib.py", line 1243, in test_with_statement_QUIT_failure smtp.noop() ~~~~~~~~~^^ File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto-pgo/build/Lib/smtplib.py", line 523, in noopreturnself.docmd("noop") ~~~~~~~~~~^^^^^^^^ File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto-pgo/build/Lib/smtplib.py", line 430, in docmdreturnself.getreply() ~~~~~~~~~~~~~^^ File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto-pgo/build/Lib/smtplib.py", line 403, in getreplyraise SMTPServerDisconnected("Connection unexpectedly closed") smtplib.SMTPServerDisconnected: Connection unexpectedly closed Traceback (most recent call last): File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto-pgo/build/Lib/test/test_smtplib.py", line 1233, in test_with_statement code, message = smtp.noop() ~~~~~~~~~^^ File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto-pgo/build/Lib/smtplib.py", line 523, in noopreturnself.docmd("noop") ~~~~~~~~~~^^^^^^^^ File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto-pgo/build/Lib/smtplib.py", line 430, in docmdreturnself.getreply() ~~~~~~~~~~~~~^^ File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto-pgo/build/Lib/smtplib.py", line 403, in getreplyraise SMTPServerDisconnected("Connection unexpectedly closed") smtplib.SMTPServerDisconnected: Connection unexpectedly closed

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL8 FIPS Only Blake2 Builtin Hash 3.x has failed when building commit db00fee.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/469/builds/8522) and take a look at the build logs.
  4. Check if the failure is related to this commit (db00fee) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/469/builds/8522

Failed tests:

  • test_smtplib

Failed subtests:

  • test_with_statement_QUIT_failure - test.test_smtplib.SMTPSimTests.test_with_statement_QUIT_failure
  • test_with_statement - test.test_smtplib.SMTPSimTests.test_with_statement

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last): File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/test/test_smtplib.py", line 1243, in test_with_statement_QUIT_failure smtp.noop() ~~~~~~~~~^^ File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/smtplib.py", line 523, in noopreturnself.docmd("noop") ~~~~~~~~~~^^^^^^^^ File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/smtplib.py", line 430, in docmdreturnself.getreply() ~~~~~~~~~~~~~^^ File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/smtplib.py", line 403, in getreplyraise SMTPServerDisconnected("Connection unexpectedly closed") smtplib.SMTPServerDisconnected: Connection unexpectedly closed Traceback (most recent call last): File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/test/test_smtplib.py", line 1233, in test_with_statement code, message = smtp.noop() ~~~~~~~~~^^ File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/smtplib.py", line 523, in noopreturnself.docmd("noop") ~~~~~~~~~~^^^^^^^^ File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/smtplib.py", line 430, in docmdreturnself.getreply() ~~~~~~~~~~~~~^^ File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/smtplib.py", line 403, in getreplyraise SMTPServerDisconnected("Connection unexpectedly closed") smtplib.SMTPServerDisconnected: Connection unexpectedly closed

noahbkim pushed a commit to hudson-trading/cpython that referenced this pull request Jul 11, 2024
Handle errors from `os.scandir()` and `ScandirIterator` similarly, which lets us loop over directory entries with `for`.
estyxx pushed a commit to estyxx/cpython that referenced this pull request Jul 17, 2024
Handle errors from `os.scandir()` and `ScandirIterator` similarly, which lets us loop over directory entries with `for`.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performancePerformance or resource usage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@barneygale@bedevere-bot@serhiy-storchaka