Skip to content

Conversation

@jaraco
Copy link
Member

@jaracojaraco commented Oct 1, 2022

@woutdenolf
Copy link

woutdenolf commented Oct 3, 2022

Starts to be rather cumbersome to support python>=3.7

importsystry: fromimportlib.metadataimportentry_pointsexceptImportError: fromimportlib_metadataimportentry_pointsifsys.version_info>= (3,10): _entry_points=entry_points(group=namespace) else: try: _entry_points=entry_points().get(namespace, []) exceptAttributeError: _entry_points=entry_points().select(group=namespace)

@jaraco
Copy link
MemberAuthor

jaraco commented Oct 3, 2022

That's one way to do it. Other options include:

  • Rely on importlib_metadata >= 3.6 for Python < 3.10. As a general rule, you should adopt importlib_metadata any time there's behavior from a future Python version but your application supports older Python versions.
  • Use backports.entry_points_selectable (provides forward compatibility to Python 2.7+).
  • Bypass entry_points altogether and re-implement its behavior using more fundamental interfaces (flake8 took this approach; I don't recommend it).

@woutdenolf
Copy link

Thanks for the feedback!

Rely on importlib_metadata >= 3.6 for Python < 3.10

Many projects just need importlib.metadata so they have importlib-metadata; python_version < '3.8'. Once python 3.7 gets dropped it should become simpler.

Python 3.10 still supports entry_points().select(group=namespace) so using entry_points(group=namespace) is not mandatory.

tsibley added a commit to nextstrain/cli that referenced this pull request Oct 4, 2022
The biggest impact of this change is on Python 3.7 where we were seeing CI test failures.¹ Those now pass again. The story is complicated. Let's start with the facts. The recent release of importlib-metadata 5.0.0 removed a previously deprecated entry points API.² This API is used by flake8. flake8 4.0.0 onwards declares (more or less): importlib-metadata <4.3; python_version < "3.8" …while prior versions, flake8 3.9.2 and earlier, declared: importlib-metadata; python_version < "3.8" Note that flake8 3.9.2 and earlier still require in practice the upper bound on the importlib-metadata version, they just don't express it in package metadata. (This is what's known as foreshadowing…) Meanwhile, Sphinx 4.4.0 onwards declares (more or less): importlib-metadata >=4.4; python_version < '3.10' Note that on Python 3.7 and earlier, the latest versions of both flake8 and Sphinx have conflicting version requirements for importlib-metadata. So when Pip on 3.7 goes to install both, it has to resolve the issue by downgrading either flake8 or Sphinx until it finds a dependency solution. For whatever reason, it choose to keep the newer versions of Sphinx and importlib-metadata and downgrade flake8. So it walks back in flake8 versions until it eventually finds 3.9.2 without the importlib-metadata version pin. Pip is satisfied, but although Pip can't know it, flake8 won't work with importlib-metadata 5.0.0. By pinning flake8 ≥4.0.0, we're hinting to Pip that only those versions accurately declare their dependencies given the current situation. Pip ultimately can use this better dependency information solve the importlib-metadata conflict by walking back to a version of Sphinx ≤4.4.0 (which it turns out, don't use importlib-metadata at all). The flake8 breakage doesn't arise on Python 3.6 because support for that Python version was dropped by importlib-metadata ≥4.9.0, so the removed API in 5.0.0 isn't seen. Without our flake8 pin, Pip still resolves the conflict by walking flake8 back to 3.9.2 and keeping the latest Sphinx plus importlib-metadata 4.8.3. While that importlib-metadata version is declared as unsupported in later version of flake8, it seems to work fine in practice. ¯\_(ツ)_/¯ On Python ≥3.8, the importlib-metadata conflict doesn't exist, presumably because flake8 uses the importlib.metadata standard library instead. We'll still build our production docs on RTD with the latest version of Sphinx, because we use Python 3.10 there, and the downgrading of Sphinx is limited to 3.7. (If some of this is making your head ring but also ringing some bells, this isn't the first time importlib-metadata's development choices and flake8's development choices have had fallout for us.³) ¹ python/importlib_metadata#409 (comment) ² python/importlib_metadata#405 ³ "dev: Ignore a deprecation warning generated by flake8's usage of importlib_metadata" (1a04901)
tsibley added a commit to nextstrain/cli that referenced this pull request Oct 5, 2022
The biggest impact of this change is on Python 3.7 where we were seeing CI test failures.¹ Those now pass again. The story is complicated. Let's start with the facts. The recent release of importlib-metadata 5.0.0 removed a previously deprecated entry points API.² This API is used by flake8. flake8 4.0.0 onwards declares (more or less): importlib-metadata <4.3; python_version < "3.8" …while prior versions, flake8 3.9.2 and earlier, declared: importlib-metadata; python_version < "3.8" Note that flake8 3.9.2 and earlier still require in practice the upper bound on the importlib-metadata version, they just don't express it in package metadata. (This is what's known as foreshadowing…) Meanwhile, Sphinx 4.4.0 onwards declares (more or less): importlib-metadata >=4.4; python_version < '3.10' Note that on Python 3.7 and earlier, the latest versions of both flake8 and Sphinx have conflicting version requirements for importlib-metadata. So when Pip on 3.7 goes to install both, it has to resolve the issue by downgrading either flake8 or Sphinx until it finds a dependency solution. For whatever reason, it choose to keep the newer versions of Sphinx and importlib-metadata and downgrade flake8. So it walks back in flake8 versions until it eventually finds 3.9.2 without the importlib-metadata version pin. Pip is satisfied, but although Pip can't know it, flake8 won't work with importlib-metadata 5.0.0. By pinning flake8 ≥4.0.0, we're hinting to Pip that only those versions accurately declare their dependencies given the current situation. Pip ultimately can use this better dependency information solve the importlib-metadata conflict by walking back to a version of Sphinx ≤4.4.0 (which it turns out, don't use importlib-metadata at all). The flake8 breakage doesn't arise on Python 3.6 because support for that Python version was dropped by importlib-metadata ≥4.9.0, so the removed API in 5.0.0 isn't seen. Without our flake8 pin, Pip still resolves the conflict by walking flake8 back to 3.9.2 and keeping the latest Sphinx plus importlib-metadata 4.8.3. While that importlib-metadata version is declared as unsupported in later version of flake8, it seems to work fine in practice. ¯\_(ツ)_/¯ On Python ≥3.8, the importlib-metadata conflict doesn't exist, presumably because flake8 uses the importlib.metadata standard library instead. We'll still build our production docs on RTD with the latest version of Sphinx, because we use Python 3.10 there, and the downgrading of Sphinx is limited to 3.7. (If some of this is making your head ring but also ringing some bells, this isn't the first time importlib-metadata's development choices and flake8's development choices have had fallout for us.³) ¹ python/importlib_metadata#409 (comment) ² python/importlib_metadata#405 ³ "dev: Ignore a deprecation warning generated by flake8's usage of importlib_metadata" (1a04901)
pvital added a commit to instana/python-sensor that referenced this pull request Mar 8, 2023
importlib_metadata package removed deprecated entry_point interfaces on version 5.0.0 [1] , and this change impacts celery >= 5.2.7 [2] running with python 3.7.X (it doesn't impact >= 3.8). For this reason, we control celery and importlib_metadata versions on python 3.7 environments. [1] python/importlib_metadata#405 [2] celery/celery#7783 Signed-off-by: Paulo Vital <paulo.vital@ibm.com>
Ferenc- pushed a commit to instana/python-sensor that referenced this pull request Mar 8, 2023
importlib_metadata package removed deprecated entry_point interfaces on version 5.0.0 [1] , and this change impacts celery >= 5.2.7 [2] running with python 3.7.X (it doesn't impact >= 3.8). For this reason, we control celery and importlib_metadata versions on python 3.7 environments. [1] python/importlib_metadata#405 [2] celery/celery#7783 Signed-off-by: Paulo Vital <paulo.vital@ibm.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@jaraco@woutdenolf