`Py_EnterRecursiveCall()`/`_Py_EnterRecursiveCall()` return value misused in a number of places

[OTheDev]

Using grep, I identified 3 instances in main where code checks if Py_EnterRecursiveCall(...) < 0 (which just returns _Py_EnterRecursiveCall(...)) or _Py_EnterRecursiveCall(...) < 0.

1. Py_EnterRecursiveCall() documentation only guarantees that a nonzero value is returned in the event of an error.
2. The actual implementation can't return a negative value:

cpython/Include/internal/pycore_ceval.h

Lines 130 to 138 in 7644935

	static inline int _Py_EnterRecursiveCallTstate(PyThreadState *tstate,
	const char *where){
	return (_Py_MakeRecCheck(tstate) && _Py_CheckRecursiveCall(tstate, where));
	}
	static inline int _Py_EnterRecursiveCall(const char *where){
	PyThreadState *tstate = _PyThreadState_GET();
	return _Py_EnterRecursiveCallTstate(tstate, where);
	}

Linked PRs

• gh-101965: Fix usage of Py_EnterRecursiveCall return value in _bisectmodule.c #101966

[erlend-aasland]

Thanks for the report, Owain! In this case, I think we can close it as obsolete bco. Irit's patch #102078. Thanks for your interest in improving CPython :)

[OTheDev]

@erlend-aasland Thanks! However, Irit's patch does not fix the instances in the bisect module. Will it be addressed there?

[erlend-aasland]

@erlend-aasland Thanks! However, Irit's patch does not fix the instances in the bisect module. Will it be addressed there?

Well observed! Check with Irit on her PR. I'm guessing it is out of scope for her PR. In that case we'll reopen this and you can update your PR to target the remaining issues.