Segmentation Fault in combination with Distributed

[schlichtanders]

Affects: PythonCall

Describe the bug
When trying to combine PythonCall with Pluto, I get Segfaults which seem to be due to a combination with Distributed. But not really sure yet.

It already took a while to extract the Segmentation fault error message - want to share t here

329] signal (11.1): Segmentation fault
in expression starting at none:1
_PyEval_EvalFrameDefault at /home/jolin_user/.julia/environments/v1.10/.CondaPkg/env/lib/libpython3.11.so.1.0 (unknown line)
_PyEval_Vector at /home/jolin_user/.julia/environments/v1.10/.CondaPkg/env/lib/libpython3.11.so.1.0 (unknown line)
PyEval_EvalCode at /home/jolin_user/.julia/environments/v1.10/.CondaPkg/env/lib/libpython3.11.so.1.0 (unknown line)
builtin_exec at /home/jolin_user/.julia/environments/v1.10/.CondaPkg/env/lib/libpython3.11.so.1.0 (unknown line)
cfunction_vectorcall_FASTCALL_KEYWORDS at /home/jolin_user/.julia/environments/v1.10/.CondaPkg/env/lib/libpython3.11.so.1.0 (unknown line)
PyObject_CallObject at /home/jolin_user/.julia/packages/PythonCall/Nr75f/src/C/pointers.jl:303 [inlined]
macro expansion at /home/jolin_user/.julia/packages/PythonCall/Nr75f/src/Core/Py.jl:132 [inlined]
pycallargs at /home/jolin_user/.julia/packages/PythonCall/Nr75f/src/Core/builtins.jl:220
unknown function (ip: 0xffff691d316f)
_jl_invoke at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/gf.c:2895 [inlined]
ijl_apply_generic at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/gf.c:3077
pycall#21 at /home/jolin_user/.julia/packages/PythonCall/Nr75f/src/Core/builtins.jl:243
pycall at /home/jolin_user/.julia/packages/PythonCall/Nr75f/src/Core/builtins.jl:233 [inlined]
_#11 at /home/jolin_user/.julia/packages/PythonCall/Nr75f/src/Core/Py.jl:357 [inlined]
Py at /home/jolin_user/.julia/packages/PythonCall/Nr75f/src/Core/Py.jl:357 [inlined]
pyexec at /home/jolin_user/.julia/packages/PythonCall/Nr75f/src/Core/builtins.jl:1326
pyexec at /home/jolin_user/.julia/packages/PythonCall/Nr75f/src/Core/builtins.jl:1325
unknown function (ip: 0xffff69be5067)
_jl_invoke at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/gf.c:2895 [inlined]
ijl_apply_generic at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/gf.c:3077
jl_apply at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
do_call at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/interpreter.c:126
eval_value at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/interpreter.c:223
eval_stmt_value at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/interpreter.c:174 [inlined]
eval_body at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/interpreter.c:635
jl_interpret_toplevel_thunk at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/interpreter.c:775
jl_toplevel_eval_flex at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/toplevel.c:934
ijl_toplevel_eval_in at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/toplevel.c:985
eval at ./boot.jl:385
jfptr_eval_25809.3 at /usr/local/julia/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/gf.c:2895 [inlined]
ijl_apply_generic at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/gf.c:3077
jl_apply at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
jl_f__call_latest at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/builtins.c:812
_jl_invoke at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/gf.c:2895 [inlined]
ijl_apply_generic at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/gf.c:3077
jl_apply at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
do_apply at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/builtins.c:768
invokelatest#2 at ./essentials.jl:892
_jl_invoke at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/gf.c:2895 [inlined]
ijl_apply_generic at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/gf.c:3077
jl_apply at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
do_apply at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/builtins.c:768
invokelatest at ./essentials.jl:889
_jl_invoke at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/gf.c:2895 [inlined]
ijl_apply_generic at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/gf.c:3077
jl_apply at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
do_apply at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/builtins.c:768
110 at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:287
run_work_thunk at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:70
109 at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:287
unknown function (ip: 0xffff6c0f92b7)
_jl_invoke at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/gf.c:2895 [inlined]
ijl_apply_generic at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/gf.c:3077
jl_apply at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
start_task at /cache/build/default-aws-aarch64-ci-0-2/julialang/julia-release-1-dot-10/src/task.c:1238
Allocations: 20773219 (Pool: 20743047; Big: 30172); GC: 24

using plain julia 1.10.5

[schlichtanders]

So I can reproduce this and it happens when I send a python code object from the one julia process to the other. (supported by setting JULIA_PYTHONCALL_PICKLE=dill).

This was not failing on previous versions of PythonCall... not precisely sure which change made the difference.

EDIT: I just realized that I am sending from PythonCall v0.9.19 to v0.9.23. That difference might be the reason

[cjdoris]

Can you reproduce the issue with the latest version of PythonCall installed everywhere?

[schlichtanders]

I updated everything and indeed the error is gone. Closing this issue respectively. Hope it is going to help others in case they find it by searching on github.