concurrent.futures.InterpreterPoolExecutor

[crusaderky]

BPO	37293
Nosy	@ericsnowcurrently, @applio, @crusaderky, @jakirkham

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2019-06-15.13:15:17.301>
labels = ['extension-modules', 'type-feature', '3.9']
title = 'concurrent.futures.InterpreterPoolExecutor'
updated_at = <Date 2020-08-05.21:42:09.359>
user = 'https://github.com/crusaderky'

bugs.python.org fields:

activity = <Date 2020-08-05.21:42:09.359>
actor = 'jakirkham'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Extension Modules']
creation = <Date 2019-06-15.13:15:17.301>
creator = 'crusaderky'
dependencies = []
files = []
hgrepos = []
issue_num = 37293
keywords = []
message_count = 3.0
messages = ['345681', '346225', '346226']
nosy_count = 4.0
nosy_names = ['eric.snow', 'davin', 'crusaderky', 'jakirkham']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue37293'
versions = ['Python 3.9']

[crusaderky]

As one of the logical consequences to PEP-554, it would be neat to have a concurrent.futures.InterpreterPoolExecutor.

I wrote the initial code at https://github.com/crusaderky/subinterpreters_tests - currently missing unit tests and pickle5 buffers support.

If everybody is happy with the design, I'll start working on a PR as soon as the GIL becomes per-interpreter (it's kinda pointless before that).

[ericsnowcurrently]

FWIW, performance benefits when subinterpreters stop sharing the GIL are not the only benefit. In fact, PEP-554 is specifically written to avoid that consideration, focusing on the benefits of the concurrency model (i.e. CSP). So I wouldn't call this pointless even without per-interpreter GIL. :)

(Of course, this *would* be blocked on acceptance and implementation of PEP-554.)

[ericsnowcurrently]

@davin, we've spoken before about something similar for multiprocessing, IIRC. :)

[jakirkham]

Think this can be closed now that PR ( #124548 ) is merged

[ericsnowcurrently]

Duplicate of gh-124694. (or rather, I didn't see this one when I created the other one.)