Repository navigation
thread safety #882
Description
If Julia's Threads.nthreads() > 1, we may want to do some more work to ensure thread safety. In particular:
- Call
PyEval_InitThreads()in__init__(this is only needed for Python ≤ 3.6, and is a no-op in Python ≥ 3.7). - Acquire the GIL in the PyObject
finalizer, by callingPyGILState_Ensure(returns anenum, i.e. aCintin Julia) andPyGILState_Release. This is because the Julia GC may be called from threads other than the main thread, so we need to ensure that we hold Python's GIL before decref-ing.
We might also expose an API to acquire the GIL, but in general I would recommend that user code should only call Python from the main thread.
See this forum post and #881 for a possible test case.
Activity
marius311 commentedon Feb 25, 2021
Thanks, yea this sounds like it could be the cause. I tried your suggestion and it deadlocks. My guess is you have to do something like here. Playing with it, although I'm not an expert.
stevengj commentedon Feb 25, 2021
I hope we don't have to do something as complicated as this code in FFTW.jl.
See the discussion in JuliaMath/FFTW.jl#141 where we ran into deadlocks during GC because FFTW was trying to acquire a mutex lock during plan destruction.
stevengj commentedon Feb 25, 2021
Hmm, unfortunately I think we may be in the same situation as FFTW, because Python can call back to Julia. That is, the following may occur:
So, we may need something like the FFTW.jl solution after all.
marius311 commentedon Feb 25, 2021
Thanks, need to think about this and read that code, but one thing I dont totally follow is that in solution in my PR, the PyObject finalizer never tries to acquire the GIL. So are we fine?
PallHaraldsson commentedon Feb 26, 2021
For some uses PythonCall.jl [EDIT: renamed from Python.jl] may be preferable? I'm not yet clear about the pros and cons, or why that new package needed, but at least it has some GIL-related code, so you could either use that package or maybe it's helpful to look at the code to fix here:
https://github.com/cjdoris/PythonCall.jl/blob/6bf7575247b0354009ae8f97387817f60bf45442/src/gil.jl
secondaryfox commentedon Feb 10, 2022
I'm trying to run pytorch code from julia's pycall. I know that the pytorch code does not require GIL so I would like to run multiple of this code with julia threads -> pycall -> without GIL. Is this automatically achieved or should I do anything extra?
mirkobunse commentedon Jul 15, 2022
Users of PyCall can use a global lock that every call to Python has to acquire.
This approach works at least if Julia calls Python without any call-back to Julia. I assume that it also works with a call-back, as long as this call-back does not call Python again.
EDIT
I have verified the above approach with the following example, where we intend to fill the large array
awith random numbers fromnumpy.random.rand().The following code, which does not acquire a lock, segfaults:
However, we can acquire the lock to achieve the desired effect.
Of course, this locking mechanism requires all Python calls to acquire the lock. And acquiring the lock limits the degree to which our program parallelizes. In the above example, there is indeed no parallelism at all because the lock has to be acquired for each, complete iteration of the loop.
the-noble-argon commentedon Sep 13, 2022
@mirkobunse
I recently tried doing what you suggested with a simple JSON-writing exercise (I know Julia has the same feature, this is just a minimal example)
However, I get this error:
mirkobunse commentedon Sep 14, 2022
@the-noble-argon
Thanks for reporting! I figured that the error occurs because your threads are still running when Julia exits (EDIT: to be more clear, my pylock is not the cause of this error). A solution is to add two lines to your script:
This will make your code wait for all threads before Julia exits.
the-noble-argon commentedon Sep 14, 2022
@mirkobunse
I think that solves it (which is nice, because I was sure the pylock solution should work). Interestingly enough, this script I wrote runs fine on linux even without the wait at the end. When I saw some .dll files buried in the error message, I was wondering if it was a Python-Windows problem; sure enough, when I ran it on Ubuntu, everything was fine. I can't vouch for Mac but I'd assume it would run. For me this was actually fine, because I plan on deploying my solution to a linux machine anyway, it's just that my work-dev machine is a windows box.
Anyway, this is a huge leap in usability from when I tried to use PyCall in a multithreaded environment a couple of years ago. Whoever did this, thanks a lot!
the-noble-argon commentedon Sep 16, 2022
Unfortunately, I spoke too soon. I realized that on the linux command line version, I forgot to enable multiple threads. I'm really trying to do this in a way that is safe. I'm putting all my python objects as global constants to they don't get unintentionally garbage collected, and I'm implementing the lock functionality. Here is my latest script
I'm able to get crashes in both windows and linux. For windows:
For linux, I have:
mirkobunse commentedon Sep 16, 2022
Sorry, I'm not able to fix this example. Using
@syncand replacingwaitwithfetchseems to bring some stability, i.e., the example works more often than it breaks. Since these changes are no definite fixes, however, I will not further elaborate on them.the-noble-argon commentedon Sep 17, 2022
Yeah, this is a tough one that's hard to track down. I got it to a point in #1006 where I could still use @async on PyCall and @threads for the multithreaded Julia code, I wasn't able to crash that. I think there is something about @Spawn that really trips up a running PyCall task. Only being able to use @threads while PyCall isn't running isn't ideal, but at least that's better than Python, and @threads got better with Julia 1.8. Honestly, I mostly use PyCall for IO related tasks on Azure, so doing an @async on that will likely be just as good as multithreading, if we could use @Spawn with impunity, but I guess I can settle on @threads.
the-noble-argon commentedon Sep 19, 2022
@mirkobunse I found a solution that might interest you. I think garbage collection was being triggered in another thread (which is why the segfault happened indeterministically) and probably nuked something running in PyCall. The solution is a bit kludgy, but I fixed this by modifying the pylock() function to temporarily disable GC in the same manner that the lock/unlock is being performed under the hood.
This of course validates yet again the importance of this issue with respect to protecting PyCall tasks from GC calls from other threads.
3 remaining items
martinmestre commentedon Aug 16, 2023
Hi, I am obtaining segmentation fault when trying to use multithreading with NOMAD.jl where my loss functions are in Python and I am using PyCall.jl :
The error says:
Caught seg fault in thread 0Has anyone had a similar problem?
the-noble-argon commentedon Aug 16, 2023
Multiple times. When calling Python in a multithreaded environment you have to use the pylock() pattern we figured out in the discussion above. Issue #1006 also has this workaround. First, you have to build a PyLock function that locks the python process and disables the garbage collector while the Python code is running.
You apply the pylock function around your PyCall code.
martinmestre commentedon Aug 17, 2023
Thank you! I will try as you suggest.
martinmestre commentedon Aug 17, 2023
Hi @the-noble-argon , when I do
the first time that χ² appears in the editor is marked as not used. Does pylock introduce a new scope?
stevengj commentedon Aug 17, 2023
Yes, because it's a standard
doconstruct in Julia, which is sugar for an anonymous function. Trymartinmestre commentedon Aug 17, 2023
Thanks @stevengj , now it starts running but doesn't use the 4 threads I set with
julia -t 4and it stops with the following error:Is that a known problem?
the-noble-argon commentedon Aug 17, 2023
This doesn't look like the kinds of errors I get when PyCall segfaults on me, in such cases I get very long messages. I'm not sure this is a PyCall.jl issue. Can you try calling your loss function repeatedly in a multithreaded loop without NOMAD.jl and see if you get any more informative errors? Is there a way to track what NOMAD.jl uses as input to your loss function and build a multithreaded for loop for that?
Before trying this however, we may need to take a step backward. You should be aware that a python process can't be run in a multithreaded manner (blame Python's GIL which I hear they're trying to remove yet again). If the pylock() pattern is properly used, it will prevent any python code from running in parallel and prevent any garbage collection while python code is running. So if most of your computational effort is spent running your Python loss function, multithreading will achieve little more than causing you anguish. You may need to rewrite your loss function in Julia if you want your loss function to multithread.
martinmestre commentedon Aug 17, 2023
Thanks! Good to know what you say so that I stop trying with this path. The Python loss function is by orders of manitude that most time consuming part of my code. For a future project I will re-write it in julia. If I use distributed processes, will I have the same problem with Python's GIL ?
I still do not understand why Python can't work in real multithreading, I have used the standard libray within Python.
the-noble-argon commentedon Aug 17, 2023
I think that should work. Distributed processes have their own memory spaces managed by the O.S. so you don't run into the nasty kinds of problems of multiple threads changing things in the same process memory space. You can have as many Python processes running on one machine as you want (obviously within reason).
Python was never meant for high performance computing, it was meant to glue many (potentially high performance) packages together. Many Python packages do multithreading under the hood because they are written in high performance multithreaded languages. It may surprise you, but most data science isn't actually executed in Python, it's executed with Python libraries written in languages like C++. The core aim of many Python developers is to spend as little compute time in Python as possible, but have most compute done in high-performance libraries.
I use a lot of Python, but what I DONT use it for is optimization, especially if I have to write the objective/loss function myself. These functions get called over and over again, so there is a huge benefit from having the loss function compiled (which Python doesn't do unless you dabble in Numba/Cython, but those are finicky about the data types you can use). Because of Julia's flexibility, ease of writing, and JIT compiling nature, it is hands-down the best language I've ever used for optimizing custom objective functions. If this wasn't enough, Julia also has some really good automatic differentiation tools that analyze your code and writes the derivatives for you (if your loss function is differentiable and entirely written in Julia). When I write Julia/Python combinations, I use PyCall to call the third party APIs/SDKs (because everyone writes APIs/SDKs for Python) and Julia for the heavy number crunching.
martinmestre commentedon Aug 17, 2023
Yes, you are right. The loss function is in python because of a long story but at present I write everything in Julia. Anyway I will have to call some coordinate transformations from Astropy (Python api). Can I do automatic differentiation throgh julia code that calls PyCall or PythonCall ?
the-noble-argon commentedon Aug 17, 2023
No, you can't differentiate through anything that isn't Julia, you would have to write your own differentiation rule for anything that leaves the Julia environment. I learned that the hard way when trying to do a maximum-likelihood estimate of a gamma distribution, which called the incomplete gamma function which called an R function written in Fortran under the hood. But I COULD maximize a normal distribution, because the erf implementation was in Julia. Go figure.
martinmestre commentedon Aug 17, 2023
Thanks a lot!
mirkobunse commentedon Aug 29, 2023
If differentiation is your main concern, you might want to try Python's JAX package. It has a numpy wrapper, which can be used just like regular numpy, and which makes most numpy functions automatically differentiable. You just need to import this wrapper
instead of importing the original numpy and use automatic differentiation
to differentiate (almost) any numpy-based function.
Translating your original numpy code to JAX code might be easier than translating everything to Julia.
yuyichao commentedon Oct 18, 2026
Should be fixed by #1097 now.