Repository navigation
Problem with interrupting 'parallelFor()' execution on Windows machines #79
Description
Hello, and thank you for creating this package for R. I encountered a potential issue with the parallelFor() function on Windows machines when a C++ function is interrupted by the user from R. Below is a minimal example to reproduce the issue. I tried using the sourceCpp() function and building an R package with the same C++ code. The results were identical. The number of selected threads did not influence the result either.
I ran this code on three machines:
- Intel Core i7-2720QM, Linux Mint 22, R version 4.3.3
- Intel Core i7-4790K, Windows 10 x64 (build 19045), R version 4.1.3
- AMD Ryzen 5 5625U, Windows 10 x64 (build 19045), R version 4.4.1
library(Rcpp)
sourceCpp(code = '
#include <Rcpp.h>
#include <RcppThread.h>
#include <thread>
#include <chrono>
using namespace Rcpp;
// [[Rcpp::depends(RcppThread)]]
// [[Rcpp::export]]
bool TestUserInterrupt(int n_threads = 1L) {
RcppThread::Rcout << "Main thread ID: " << std::this_thread::get_id()
<< std::endl;
RcppThread::parallelFor(0, 5 * 10 * n_threads, [](int i) {
RcppThread::Rcout << std::this_thread::get_id() << "|";
std::this_thread::sleep_for(std::chrono::milliseconds(100));
}, n_threads);
RcppThread::Rcout << std::endl << "Processing complete." << std::endl;
return true;
}
')
TestUserInterrupt(1)On the Linux machine, everything worked as expected (the "Error: C++ call interrupted by the user" message was printed to the console, and the function could be rerun).
On the Windows machines, however, when the function was interrupted, the output to the console stopped without displaying the "Error: C++ call interrupted by the user" message. A subsequent call to the function resulted in no output: R was busy, and the interruption was not possible without restarting the R session. I am not sure if it is relevant, but when using multiple threads (e.g., 4), the output was sometimes shuffled (e.g., "... 4|||53|4||2|352|||4 ...").
One more remark. Initially I encountered a problem when I tried to integrate RcppThread into another R package. In that case, R crashed when the execution was interrupted by the user from R.
Activity
[-]Problem with interrupting 'parallelfor()' execution on Windows machines[/-][+]Problem with interrupting 'parallelFor()' execution on Windows machines[/+]aitap commentedon Apr 8, 2025
To save some time debugging: this doesn't happen on
Rterm.exe, but it does happen onRgui.exe. The second call toTestUserInterrupt(1)hangs in:...which suggests that the mutex in the
RMonitorinstance is left locked after R is interrupted. Why would that happen whenRcppThreadcarefully wraps its own checks inR_ToplevelExec?RcppThread/inst/include/RcppThread/RMonitor.hpp
Lines 127 to 138 in 3d47c8a
That's because printing in Rgui requires processing GUI events, which performs its own interrupt checks:
Moving the interrupt check before printing will make the problem less likely to happen, but it will leave a window of opportunity for the user to press interrupt after the check but before printing is complete:
RcppThread/inst/include/RcppThread/ThreadPool.hpp
Lines 249 to 251 in 3d47c8a
I'm not exactly sure, but a complete solution may involve printing from
R_ToplevelExecor using Rcpp'sunwindProtectto make sure that the destructors will fire. A good question for R-package-devel.AndreySamokhin commentedon Apr 17, 2025
I stand corrected, my initial example is not perfect as it works correctly when the
RcppThread::Rcout ...line is commented out. I have prepared another example that demonstrates the original issue more clearly.On both Windows and Linux, R crashes when the function is manually interrupted from R. The only difference is that on Linux, the
"Error: C++ call interrupted by the user"message is displayed just before the crash, whereas on Windows, this message does not appear.aitap commentedon Apr 18, 2025
That's another problem. In
RcppThread::parallelFor, when thewait()method throws aUserInterruptException, the thread pool keeps working, accessing the local variables belonging to the main thread, while the stack of the main thread is being unwound to the exception handler, destroying the local variables in the process.Even without the crash due to local variable access, you can try to interrupt a
parallelForand see that R remains busy consuming CPU time for a long time after the main thread returns to the prompt. Moreover, submitting a new task to the pool won't work:I think one possible solution here is to catch the exception in
ThreadPool::wait, stop the pool, wait until the threads actually stop executing their jobs, and then rethrow the exception.tnagler commentedon Jan 14, 2026
Many thanks to both of you and sorry for the long delay before responding/fixing the issue. Both mentioned issues should be fixed by #82, at least they were for me on a local windows install.
@AndreySamokhin The shuffled output mentioned in your first post is expected.
RcppThread::Rcout << std::this_thread::get_id() << "|";is actually making two separate calls toRcout- one for the number, one for the|. This can be avoided by, e.g.,std::ostringstream msg; msg << std::this_thread::get_id() << "|"; RcppThread::Rcout << msg.str();fix early termination/interaction with Rgui (tnagler#82, fixes tnagle…