Skip to content

gh-139540: Fix executor deallocation crash when JIT compilation fails#139952

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ashm-dev wants to merge 3 commits into from
Closed

gh-139540: Fix executor deallocation crash when JIT compilation fails #139952

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3a2d34b
Track executor object with GC
ashm-dev Oct 11, 2025
963bcf0
Create 2025-10-11-16-14-36.gh-issue-139549.9BGTLP.rst
ashm-dev Oct 11, 2025
db344b2
Fix memory leak in JIT executor on compilation failure
ashm-dev Oct 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions Misc/NEWS.d/next/Core_and_Builtins/2025-10-11-16-14-36.gh-issue-139549.9BGTLP.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
Fix memory leak in JIT executor on compilation failure

The executor object passed to the TIER1_TO_TIER2 macro was not
decremented if the Tier 2 code returned with an exception set but
without a fatal error (i.e., not returning NULL).

This change moves the Py_DECREF call inside the TIER1_TO_TIER2
macro to ensure the executor's reference count is always decremented
after its execution, regardless of the outcome.
1 change: 1 addition & 0 deletions Python/bytecodes.c
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -2972,6 +2972,7 @@ dummy_func(
assert(tstate->current_executor == NULL);
assert(executor != tstate->interp->cold_executor);
tstate->jit_exit = NULL;
tstate->current_executor = (PyObject *)executor;
Copy link
Member

@picnixzpicnixzOct 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it just to hold a reference to the executor?

TIER1_TO_TIER2(executor);
}
}
Expand Down
5 changes: 5 additions & 0 deletions Python/ceval.c
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -1146,6 +1146,11 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int
assert(frame->owner == FRAME_OWNED_BY_INTERPRETER);
/* Restore previous frame and exit */
tstate->current_frame = frame->previous;
#ifdef _Py_TIER2
if (tstate->current_executor != NULL){
tstate->current_executor = NULL;
}
#endif
return NULL;
}
#ifdef _Py_TIER2
Expand Down
5 changes: 2 additions & 3 deletions Python/ceval_macros.h
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -351,12 +351,12 @@ _PyFrame_SetStackPointer(frame, stack_pointer)

/* Tier-switching macros. */

#define TIER1_TO_TIER2(EXECUTOR) \
#define TIER1_TO_TIER2(EXECUTOR) \
Copy link
Member

@picnixzpicnixzOct 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this changed? please revert.

do{\
OPT_STAT_INC(traces_executed); \
Copy link
Member

@picnixzpicnixzOct 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this changed? please revert.

next_instr = _Py_jit_entry((EXECUTOR), frame, stack_pointer, tstate); \
frame = tstate->current_frame; \
stack_pointer = _PyFrame_GetStackPointer(frame); \
Py_DECREF(EXECUTOR); \
Copy link
Member

@picnixzpicnixzOct 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why should we decref it here?

if (next_instr == NULL){\
next_instr = frame->instr_ptr; \
JUMP_TO_LABEL(error); \
Expand DownExpand Up@@ -413,4 +413,3 @@ check_periodics(PyThreadState *tstate){
}
return 0;
}

6 changes: 6 additions & 0 deletions Python/generated_cases.c.h
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion Python/optimizer.c
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -1202,6 +1202,7 @@ make_executor_from_uops(_PyUOpInstruction *buffer, int length, const _PyBloomFil
if (executor == NULL){
return NULL;
}
_PyObject_GC_TRACK(executor);

/* Initialize exits */
_PyExecutorObject *cold = _PyExecutor_GetColdExecutor();
Expand DownExpand Up@@ -1257,7 +1258,6 @@ make_executor_from_uops(_PyUOpInstruction *buffer, int length, const _PyBloomFil
return NULL;
}
#endif
_PyObject_GC_TRACK(executor);
return executor;
}

Expand DownExpand Up@@ -1507,6 +1507,7 @@ _PyExecutor_GetColdExecutor(void)
if (cold == NULL){
Py_FatalError("Cannot allocate core JIT code");
}
_PyObject_GC_TRACK(cold);
Copy link
Member

@picnixzpicnixzOct 11, 2025•
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is incorrect. The executor is immortal for now.

No this was correct, my bad. I didn't read the function correctly. The problem is that we're setting it immortal afterwards and I'm not sure it will be happy.

Copy link
ContributorAuthor

@ashm-devashm-devOct 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, thanks for the correction.

The intent was to track it for consistency immediately after allocation. My assumption is the GC correctly handles a tracked object that later becomes immortal.

Is that a safe assumption, or should we explicitly _PyObject_GC_UNTRACK it just before immortalization?

Copy link
Member

@picnixzpicnixzOct 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That I don't know. The problem here is whether we want an immortal cold executor or not. That's why I wanted to wait for Mark's output here, and didn't write a PR already.

((_PyUOpInstruction *)cold->trace)->opcode = _COLD_EXIT;
#ifdef _Py_JIT
cold->jit_code = NULL;
Expand Down
3 changes: 3 additions & 0 deletions Python/pystate.c
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -1717,6 +1717,9 @@ PyThreadState_Clear(PyThreadState *tstate)
Py_CLEAR(tstate->async_gen_finalizer);

Py_CLEAR(tstate->context);
#ifdef _Py_TIER2
Py_CLEAR(tstate->current_executor);
#endif

#ifdef Py_GIL_DISABLED
// Each thread should clear own freelists in free-threading builds.
Expand Down
Loading