Skip to content

Conversation

@rvagg
Copy link
Member

Discovered while messing with some infra switching things around to python3. On Linux, if you call configure directly with a python3 (not letting it find python for you) and have LC_ALL=C set, as we do in some of our infra, apparently open() defaults to ascii. This is a problem for icu version detection because there's a © at the top of deps/icu-small/source/common/unicode/uvernum.h which it reads to find major version number.

$ LC_ALL=C python3 ./configure.py Traceback (most recent call last): File "./configure.py", line 1660, in <module> configure_intl(output) File "./configure.py", line 1498, in configure_intl for line in open(uvernum_h).readlines(): File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode return codecs.ascii_decode(input, self.errors)[0] UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 3: ordinal not in range(128) 

This change fixes it and makes it work for Python 2 & 3. Although I don't really know what I'm doing with the conversion back to ascii for later compatibility so someone who knows better should check.

@rvaggrvagg requested review from cclauss and srl295October 21, 2019 11:38
@nodejs-github-botnodejs-github-bot added the build Issues and PRs related to build files or the CI. label Oct 21, 2019
@nodejs-github-bot
Copy link
Collaborator

@targos
Copy link
Member

This is similar to #30023

@rvagg
Copy link
MemberAuthor

indeed it is! but in this case the return value needs to mix with non-utf8 strings and python2 borks at that, hence the str(x).

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

Copy link
Contributor

@cclausscclauss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Minor nit: for line in open(uvernum_h).readlines(): leaves the filehandle open so the preferred approach is:

withopen(uvernum_h) asin_file: forlineinin_file:

because the with open() as context manager will automate the filehandle close().

@rvagg
Copy link
MemberAuthor

ok, have tried that approach, ptal @cclauss

@nodejs-github-bot
Copy link
Collaborator

@cclauss
Copy link
Contributor

I have already approved. For those who want to dig in...
https://www.python.org/dev/peps/pep-0538/ and https://www.python.org/dev/peps/pep-0540/

@rvaggrvaggforce-pushed the rvagg/python3-icu-utf8 branch from 40c91e1 to 88ecfeeCompareOctober 23, 2019 00:23
@nodejs-github-bot
Copy link
Collaborator

rvagg added a commit that referenced this pull request Oct 23, 2019
PR-URL: #30047 Reviewed-By: Christian Clauss <[email protected]> Reviewed-By: Sam Roberts <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: David Carlier <[email protected]>
@rvagg
Copy link
MemberAuthor

Landed in 779d7ef

@rvaggrvagg closed this Oct 23, 2019
@rvaggrvagg deleted the rvagg/python3-icu-utf8 branch October 23, 2019 23:29
targos pushed a commit that referenced this pull request Oct 24, 2019
PR-URL: #30047 Reviewed-By: Christian Clauss <[email protected]> Reviewed-By: Sam Roberts <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: David Carlier <[email protected]>
@targostargos mentioned this pull request Nov 5, 2019
targos pushed a commit that referenced this pull request Nov 8, 2019
PR-URL: #30047 Reviewed-By: Christian Clauss <[email protected]> Reviewed-By: Sam Roberts <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: David Carlier <[email protected]>
targos pushed a commit that referenced this pull request Nov 10, 2019
PR-URL: #30047 Reviewed-By: Christian Clauss <[email protected]> Reviewed-By: Sam Roberts <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: David Carlier <[email protected]>
@targostargos mentioned this pull request Nov 10, 2019
targos pushed a commit that referenced this pull request Nov 11, 2019
PR-URL: #30047 Reviewed-By: Christian Clauss <[email protected]> Reviewed-By: Sam Roberts <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: David Carlier <[email protected]>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

buildIssues and PRs related to build files or the CI.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants

@rvagg@nodejs-github-bot@targos@cclauss@sam-github@devnexen@richardlau