bpo-28411: Remove PyInterpreterState.modules.

[ericsnowcurrently]

The status quo for sys.modules is broken. While importlib (and a few other places) use sys.modules directly, the C-API (including PyImport_Import(), etc.) use PyInterpreterState.modules directly. Since assigning to sys.modules does not update PyInterpreterState.modules, the two can get out of sync. The workaround, which we use in the test suite, is complicated. This patch resolves the issue by getting rid of PyInterpreterState.modules and making sys.modules authoritative in the C-API.

Note that this code was landed earlier (PR #1638), but reverted due to problems when a bogus value (e.g. []) is assigned to sys.modules (see bpo-31404), which is very much a corner case. I've broken that original patch up and already landed the majority. The remaining piece here is to actually drop PyInterpreterState.modules. This PR should not be merged until the problem described in bpo-31404 is resolved.

https://bugs.python.org/issue28411

[vstinner]

I added [WIP] to your PR title for this reason: "This PR should not be merged until the problem described in bpo-31404 is resolved."

[ericsnowcurrently]

Thanks.

[brettcannon]

@ericsnowcurrently It looks like bpo-31404 was resolved, so can the WIP label be removed?

[ericsnowcurrently]

FYI, I'm going to revisit this PR in the near future.

[brettcannon]

Removing the import team from reviewing as this is still a WIP. We can be added back when it's ready to go.

[matrixise]

Hi @ericsnowcurrently

Do you continue to work on this PR?

Thanks

[alimcmaster1]

@matrixise -> think we should be good to close. No response from @ericsnowcurrently

[ericsnowcurrently]

Does the problem described in bpo-31404 still exist with this branch applied?

[alimcmaster1]

Unsure - I couldn’t repro on Linux and don’t have a windows machine handy to check.

We can still keep the issue open - but might make sense to close this PR if you are no longer are interested in working on this. Unless you/others disagree?

[github-actions]

This PR is stale because it has been open for 30 days with no activity.

[github-actions]

This PR is stale because it has been open for 30 days with no activity.

[ericsnowcurrently]

I'm fine with closing this. I'll probably circle back to it at some point.