bpo-1635741: Port hashlib modules to multiphase init (PEP 489)#21818

koubaa · 2020-08-10T21:11:59Z

https://bugs.python.org/issue1635741

koubaa · 2020-08-12T01:57:44Z

@shihai1991 @vstinner @corona10 FYI. The macOS test failure looks to be due to the following error which seems unrelated to my change:

module 'test.support' has no attribute 'EnvironmentVarGuard'

shihai1991 · 2020-08-12T04:00:21Z

@shihai1991 @vstinner @corona10 FYI. The macOS test failure looks to be due to the following error which seems unrelated to my change:
module 'test.support' has no attribute 'EnvironmentVarGuard'

Hm, this failure have no relation with your PR. It's weird. Some failed test cases have updated: https://github.com/python/cpython/blob/master/Lib/test/test_selectors.py#L540

shihai1991 · 2020-08-12T04:12:01Z

The failed test cases have been fixed in 490c542
Can you do some git rebase operation? MAYBE this operation can solve your probleam.

Modules/_blake2/blake2b_impl.c

vstinner

This PR is a little bit too large. Would you mind to restrict it to blake2 which is already complex enough?

Once this PR is merged, you can write a PR for sha3 which is complex. Then we can finish with remaining modules like md5 and sha1. Or publish all PR are once.

Modules/_blake2/blake2b_impl.c

Modules/_blake2/blake2module.c

bedevere-bot · 2020-08-13T10:30:31Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

koubaa · 2020-08-13T12:17:26Z

@vstinner that's not a problem - I put each file in its own commit so I can easily submit separate PRs. I'll submit one for sha3, one for blake2, and one with the remaining ones. After breaking up and creating PRs for each I'll address your comments

koubaa · 2020-08-13T12:24:20Z

@vstinner

blake 2: bpo-1635741: port blake2 module to multi-phase init #21856
sha3: bpo-1635741: port sha3 module to multi-phase init #21855
everything else: I kept in this PR

Thanks for the review!

Modules/md5module.c

Modules/sha512module.c

koubaa · 2020-08-13T16:28:58Z

@vstinner I'm going on vacation so I'll come back to address you comment next week.

koubaa · 2020-08-22T00:23:21Z

The MacOS/Ubuntu CI has these test failures:

3 tests failed:
test_hashlib test_random test_statistics

Running locally (windows) those tests pass:

./PCBuild/amd64/python.exe -m test test_statistics test_hashlib test_random
0:00:00 Run tests sequentially
0:00:00 [1/3] test_statistics
0:00:08 load avg: 0.00 [2/3] test_hashlib
0:00:09 load avg: 0.00 [3/3] test_random
== Tests result: SUCCESS ==

I rebased on the latest master so I'm not sure what could be causing it.

**** EDIT ****
This was due to a use-after free which is now fixed.

Misc/NEWS.d/next/Core and Builtins/2020-08-10-16-11-32.bpo-1635741.O0d3ym.rst

Modules/md5module.c

Modules/sha1module.c

Modules/md5module.c

Modules/sha512module.c

Modules/md5module.c

Misc/NEWS.d/next/Core and Builtins/2020-08-10-16-11-32.bpo-1635741.O0d3ym.rst

Modules/md5module.c

Modules/sha1module.c

Modules/sha512module.c

Modules/md5module.c

vstinner

LGTM. Thanks for the multiple updates. These PR wasn't easy!

vstinner · 2020-09-04T08:54:45Z

Modules/sha512module.c


-if (Py_IS_TYPE((PyObject*)self, &SHA512type)){
-if ( (newobj=newSHA512object())==NULL)
+if (Py_IS_TYPE((PyObject*)self, st->sha512_type)){


Without defining_class, this test was wrong. So thanks for using defining_class :-)

vstinner

LGTM, but please fix the NEWS entry.

Misc/NEWS.d/next/Core and Builtins/2020-08-10-16-11-32.bpo-1635741.O0d3ym.rst

vstinner · 2020-09-06T10:09:59Z

Merged, thanks.

* origin/master: (1373 commits) bpo-1635741: Port mashal module to multi-phase init (python#22149) bpo-1635741: Port _string module to multi-phase init (pythonGH-22148) bpo-1635741: Convert _sha256 types to heap types (pythonGH-22134) bpo-1635741: Port the termios to multi-phase init (PEP 489) (pythonGH-22139) bpo-41732: add iterator to memoryview (pythonGH-22119) bpo-40744: Drop support for SQLite pre 3.7.3 (pythonGH-20909) bpo-41316: Make tarfile follow specs for FNAME (pythonGH-21511) bpo-41720: Add "return NotImplemented" in turtle.Vec2D.__rmul__(). (pythonGH-22092) bpo-1635741 port _curses_panel to multi-phase init (PEP 489) (pythonGH-21986) bpo-1635741: Port _overlapped module to multi-phase init (pythonGH-22051) bpo-1635741: Port _opcode module to multi-phase init (PEP 489) (pythonGH-22050) bpo-1635741 port zlib module to multi-phase init (pythonGH-21995) [doc] Add link to Generic in typing (pythonGH-22125) bpo-41513: Expand comments and add references for a better understanding (pythonGH-22123) bpo-1635741: Port _sha1, _sha512, _md5 to multiphase init (pythonGH-21818) closes bpo-41723: Fix an error in the py_compile documentation. (pythonGH-22110) [doc] Fix padding in some typing definitions (pythonGH-22114) Fix documented Python version for venv --upgrade-deps (pythonGH-22113) bpo-40318: Migrate to SQLite3 trace v2 API (pythonGH-19581) bpo-41687: Fix sendfile implementation to work with Solaris (python#22040) ...

…1818) Port the _sha1, _sha512, and _md5 extension modules to multi-phase initialization API (PEP 489).

koubaa requested a review from tiran as a code owner August 10, 2020 21:12

the-knights-who-say-ni added the CLA signed label Aug 10, 2020

bedevere-bot added the awaiting review label Aug 10, 2020

koubaa changed the title ~~Bpo 1635741 hashes~~bpo-1635741: Port hashlib modules to multiphase init (PEP 489)Aug 10, 2020

tiran self-assigned this Aug 11, 2020

vstinner reviewed Aug 12, 2020
View reviewed changes

Modules/_blake2/blake2b_impl.c Outdated Show resolvedHide resolved

koubaa force-pushed the bpo-1635741-hashes branch from 28c3f7b to 8491100Compare August 12, 2020 23:55

vstinner requested changes Aug 13, 2020
View reviewed changes

Modules/_blake2/blake2b_impl.c Outdated Show resolvedHide resolved
Modules/_blake2/blake2module.c Outdated Show resolvedHide resolved
Modules/_blake2/blake2module.c Outdated Show resolvedHide resolved

bedevere-bot added awaiting changes and removed awaiting review labels Aug 13, 2020

koubaa force-pushed the bpo-1635741-hashes branch from 5ddff05 to fddf3ceCompare August 13, 2020 12:22

vstinner reviewed Aug 13, 2020
View reviewed changes

Modules/md5module.c Outdated Show resolvedHide resolved
Modules/md5module.c Outdated Show resolvedHide resolved
Modules/md5module.c Outdated Show resolvedHide resolved
Modules/sha512module.c Outdated Show resolvedHide resolved
Modules/sha512module.c Outdated Show resolvedHide resolved

koubaa force-pushed the bpo-1635741-hashes branch from fddf3ce to 617660dCompare August 21, 2020 23:43

vstinner reviewed Aug 28, 2020
View reviewed changes

vstinner reviewed Aug 29, 2020
View reviewed changes

Modules/md5module.c Outdated Show resolvedHide resolved

koubaa added 7 commits August 31, 2020 16:15

port sha1
4ae1a3f

port sha512
b389566

port md5
8add6b1

blurb
3ff18f7

address comments
9388486

remove unneeded comment
e0c0feb

fix use-after-free
6c7f435

koubaa added 4 commits August 31, 2020 16:15

add traverse/clear/free module methods
37e2282

resolve comments
0384e6b

pep7
2cfcb12

fix dealloc function
4bcb57f

koubaa force-pushed the bpo-1635741-hashes branch from 02ffcf6 to 4bcb57fCompare August 31, 2020 21:15

vstinner reviewed Sep 1, 2020
View reviewed changes

update based on PR review
3cd5d6e

vstinner reviewed Sep 2, 2020
View reviewed changes

Modules/md5module.c Outdated Show resolvedHide resolved

koubaa added 2 commits September 2, 2020 18:01

use defining_class
0244bcf

remove comment
b66a965

vstinner approved these changes Sep 4, 2020
View reviewed changes

bedevere-bot added awaiting merge and removed awaiting changes labels Sep 4, 2020

vstinner approved these changes Sep 4, 2020
View reviewed changes

Misc/NEWS.d/next/Core and Builtins/2020-08-10-16-11-32.bpo-1635741.O0d3ym.rst Outdated Show resolvedHide resolved

fix readme
df69769

vstinner merged commit 63f102f into python:masterSep 6, 2020

bedevere-bot removed the awaiting merge label Sep 6, 2020

koubaa deleted the bpo-1635741-hashes branch September 11, 2020 02:24

xzy3 pushed a commit to xzy3/cpython that referenced this pull request Oct 18, 2020
bpo-1635741: Port _sha1, _sha512, _md5 to multiphase init (pythonGH-2…
5644536
…1818) Port the _sha1, _sha512, and _md5 extension modules to multi-phase initialization API (PEP 489).

kylotanmannequin mentioned this pull request Sep 19, 2022
Py_Finalize() doesn't clear all Python objects at exit #44470
Closed

Uh oh!

bpo-1635741: Port hashlib modules to multiphase init (PEP 489)#21818

bpo-1635741: Port hashlib modules to multiphase init (PEP 489) #21818

Uh oh!

Conversation

koubaa commented Aug 10, 2020• edited by bedevere-bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

koubaa commented Aug 12, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shihai1991 commented Aug 12, 2020

Uh oh!

shihai1991 commented Aug 12, 2020

Uh oh!

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bedevere-bot commented Aug 13, 2020

Uh oh!

koubaa commented Aug 13, 2020

Uh oh!

koubaa commented Aug 13, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

koubaa commented Aug 13, 2020

Uh oh!

koubaa commented Aug 22, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

vstinnerSep 4, 2020

Choose a reason for hiding this comment

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vstinner commented Sep 6, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

koubaa commented Aug 10, 2020•
edited by bedevere-bot
Loading

koubaa commented Aug 12, 2020•
edited
Loading

koubaa commented Aug 22, 2020•
edited
Loading