Native consumers of the libcuopt-mathopt wheel segfault: solver_settings_t is split across mathopt and client

[ramakrishnap-nv]

Description

cuopt_cli as shipped in the libcuopt-mathopt wheel segfaults immediately on startup. The same binary built from the same commit into the libcuopt wheel works.

$ pip install libcuopt_mathopt_cu13-26.10.0-py3-none-manylinux_2_28_x86_64.whl
$ .../site-packages/libcuopt_mathopt/bin/cuopt_cli --help
Segmentation fault (core dumped)     # exit 139, zero bytes of output

$ .../site-packages/libcuopt/bin/cuopt_cli --help
Usage: cuopt_cli [--help] [--version] ...   # exit 0, 96 lines

Found while installing each component wheel on its own in a bare container, against the wheels from the latest green build.yaml run on main (run 36580941794, x86_64, cu13).

It is the binary, not the environment

With both wheels installed in the same container, and the mathopt binary forced onto the metapackage's libraries:

libcuopt/bin/cuopt_cli                   exit=0    lines=96
libcuopt_mathopt/bin/cuopt_cli           exit=139  lines=0
mathopt CLI + LD_LIBRARY_PATH=libcuopt/lib64   exit=139

The two are genuinely different compilations, not the same binary with different metadata:

	build-id	.text	.rodata
libcuopt/bin/cuopt_cli	b9c1fdb1...	3100a14cc7034fe1	cf1b26601c8b39d2
libcuopt_mathopt/bin/cuopt_cli	924272b9...	724578ddda193f99	b2b4a865cc6f3d2e

libcuopt_client.so has the same build-id in both wheels; libcuopt_mathopt.so and cuopt_cli differ.

Ruled out

• Unresolved libraries. ldd reports nothing missing. Every entry resolves, including libcuopt_client.so from the sibling wheel via $ORIGIN/../../libcuopt_client/lib64.
• Lazy binding. LD_BIND_NOW=1 still segfaults, so every symbol resolves; the crash is in execution, not loading.
• A missing component. Installing libcuopt-routing alongside makes no difference.
• Arguments. --help and no arguments both crash, before any output.
• ABI mismatch between CLI and engine. A matched pair from the same build job crashes just as a mismatched pair does.
• RUNPATH corruption from patchelf. Both binaries carry the same RUNPATH entries and near-identical LOAD segments.
• Duplicate OpenMP runtimes. Both wheels vendor libgomp-34324e08.so.1.0.0 under one soname, so only one instance loads.

What is known about the crash

Thread 1 "cuopt_cli" received signal SIGSEGV
#0  __memcpy_avx_unaligned ()
#1  0x000062a0875c245d in ?? ()
#2  0x000062a0875c0c3c in ?? ()
#3  __libc_start_call_main (main=0x62a0875c0320, argc=2, ...)

Two frames inside cuopt_cli between main and a memcpy, with no output produced, so it is early in main. The shipped binary is stripped, so the frames have no names; a debug build is needed to go further.

Why nothing caught it

The component wheels are built and published but never installed or imported in CI. Noted previously in #1635 (comment). A job that installs each wheel in a clean environment and runs cuopt_cli --help would catch this directly.

Impact

cuopt_cli is the reason libcuopt-mathopt declares a console entry point. As shipped, a standalone pip install libcuopt-mathopt provides a CLI that cannot start.

[ramakrishnap-nv]

Root cause, and it is broader than cuopt_cli

Any native program linking libcuopt_mathopt.so from the component wheel crashes, not just the CLI. The C API examples from the docs, built against the wheel's own headers, both segfault:

$ gcc -I.../libcuopt_mathopt/include -I.../libcuopt_client/include simple_lp_example.c \
      -lcuopt_mathopt -lcuopt_client
$ ./simple_lp_example
Segmentation fault    # same for simple_milp_example.c

The same sources against the libcuopt metapackage solve correctly (Termination status: Optimal (1), objective -0.200000).

With symbols, the crash is exact:

#2 test_simple_lp ()                 simple_lp_example.c:148
#1 cuOptSetFloatParameter ()         <- libcuopt_mathopt.so
#0 solver_settings_t<int,double>::set_parameter<double>(std::string const&, double)
                                     <- libcuopt_client.so     SIGSEGV

A C API entry point in mathopt calls a solver_settings_t method that binds into the client.

solver_settings_t<int,double> is split across two shared libraries

Library	set_parameter<double>	solver_settings_t()
libcuopt_client.so	exports	--
libcuopt_mathopt.so (component wheel)	--	exports
libcuopt_mathopt.so (metapackage)	--	exports

One library constructs the object, the other mutates it. Both mathopt builds export the same set, and the client is byte-identical between the two wheels (build-id 5133d1c7...), so the difference that decides crash-or-not lies in the two mathopt builds, which differ in build-id, .text and .rodata despite identical sources and build scripts.

That makes this an ODR/layout hazard rather than a packaging mistake: the two halves of one class agree in the metapackage build and disagree in the component build.

Corrections to my original report

• The crash is not before main. A minimal native program that links the same libraries and calls cuOptGetVersion prints from main and returns 26.10.0 cleanly. The doc examples produce no output only because they print nothing before their first cuOpt* call.
• Load order is not the cause. LD_PRELOADing libcuopt_client.so so it initialises first does not help.
• The library is fine when driven from Python: load_library() succeeds and a ctypes call to cuOptGetVersion returns 26.10.0. Only the native path through cuOptSetFloatParameter crashes, which is why the Python suite never caught it.

Separately: the wheel is not usable as an SDK

libcuopt_mathopt.so's RUNPATH has no entry for the sibling client wheel:

$ORIGIN:$ORIGIN/../lib64:$ORIGIN/../../rapids_logger/lib64:$ORIGIN/../../librmm/lib64:...:$ORIGIN/../../libcuopt_mathopt_cu13.libs

so a linked program fails with libcuopt_client.so: cannot open shared object file until LD_LIBRARY_PATH is set by hand. The sibling entries were added to the executables only, on the reasoning that load_library() preloads the client -- which holds for Python consumers but not for anyone linking the library from C or C++.

[ramakrishnap-nv]

Minimal reproducer

Stepping through the C API one call at a time isolates it. Identical source, identical probe, two wheels:

Call	libcuopt metapackage	libcuopt-mathopt wheel
cuOptGetVersion	rc=0	rc=0
cuOptCreateProblem	rc=0	rc=0
cuOptGetNumVariables	rc=0, nv=2	rc=0, nv=2
cuOptCreateSolverSettings	rc=0	rc=0
cuOptSetFloatParameter	rc=0	SIGSEGV
cuOptSolve	rc=0	--
cuOptGetObjectiveValue	obj=-1.000000	--

Everything up to the settings mutator succeeds on both, including building a real problem and reading its variable count back. The crash is on the first call that reaches solver_settings_t::set_parameter, which binds into libcuopt_client.so while cuOptCreateSolverSettings -- which succeeds -- is served by mathopt.

cuOptSolverSettings st = NULL;
cuOptCreateSolverSettings(&st);              /* rc=0, both wheels   */
cuOptSetFloatParameter(st, CUOPT_TIME_LIMIT, 5.0);  /* SIGSEGV on the component wheel */

So the settings object is constructed by one library and mutated by another, and the two disagree only in the component build. That is consistent with the split reported above: mathopt exports the constructor, the client exports set_parameter<double>.

Worth noting cuOptCreateProblem and cuOptGetNumVariables both work, so the problem-construction half of the C API is fine. Only the settings path crosses the boundary.

[ramakrishnap-nv]

The split is real, and it is the same split in both builds

solver_settings_t<int, double> has its members partitioned across two shared libraries:

Library	Members it defines
libcuopt_client.so	set_parameter, get_parameter, get_parameter_names, get_{int,float,bool}_parameters, get_pdlp_settings, get_mip_settings, dump_parameters_to_file, ...
libcuopt_mathopt.so	the constructor, add_initial_solution, add_initial_mip_solution, get_initial_*, has_initial_*, get_pdlp_warm_start_data, set_pdlp_warm_start_data

Roughly: the CPU-side parameter registry lives in the client, the GPU-side initial-solution and warm-start state lives in mathopt. One object, two owners.

Why the metapackage does not fail

Not because the split is absent there -- it is identical. Measured on both mathopt builds:

                         UND set_parameter<double>   DEFINED
libcuopt_mathopt.so (metapackage)        1               0
libcuopt_mathopt.so (component wheel)    1               0

Both call out to the client's copy; neither has its own. The exported symbol sets are identical, and both bake the same CUDA architectures (75-real,80-real,86-real,90a-real,100f-real,120a-real). libcuopt_client.so is byte-identical between the two wheels (build-id 5133d1c7...).

So the same call lands in the same client function in both cases. It succeeds in one and segfaults in the other, which leaves only the object being passed.

What the crash actually is

=> set_parameter<double>+73:  mov  0x8(%r14),%rdx
   set_parameter<double>+77:  cmp  0x8(%rbx),%rdx
   set_parameter<double>+81:  jne  set_parameter<double>+64

A loop comparing std::string lengths -- the parameter-name lookup walking the registry -- with %r14 pointing at unmapped memory (si_addr = 0x6078b2c47040, a plausible heap address, so it has walked off the end rather than hit a null).

The client's set_parameter is reading the registry container at an offset that does not match the object mathopt constructed. The two halves disagree about the layout of the class they share.

What is not yet established

Why the layouts diverge. Source, build scripts, compile flags and CUDA architectures all match between the two mathopt builds, yet their .text and .rodata differ. Each wheel configures the whole C++ tree in its own job and resolves its own build-time dependency versions, so a nightly librmm/libraft/CCCL picked up by one job and not the other would be enough, if any of those types appear in solver_settings_t. That is a hypothesis, not a measurement.

The fix worth making

The layout divergence is the trigger, but the defect is the design: one class whose storage is written by two independently-built shared libraries has no ABI contract holding it together, so it will keep breaking whenever the builds drift. Options, roughly in order of robustness:

1. Give the class one owner. Move all of solver_settings_t into cuopt_client. It is problem/settings description, which is the client's half of the boundary this split already draws, and the same conclusion Split the cuopt Python package into per-solver packages for VRP and LP #2005 reached about DataModel and SolverSettings on the Python side. The GPU-side warm-start members are the obstacle.
2. Hide the GPU half behind an opaque pointer, exactly as fix(client): hold the GPU variant alternatives behind an opaque pointer #1940 did for the MIP/LP GPU solution variants. The layout then stops depending on any CUDA-side type and becomes identical in every library that sees the header.
3. Keep the split but pin the ABI -- a compile-time assertion on sizeof(solver_settings_t) and key member offsets, checked in both libraries. Cheap, and would have turned this segfault into a build failure, but it does not remove the hazard.

(2) looks closest to the pattern already used in this codebase for the same shape of problem. (3) is worth doing regardless, since it converts a silent runtime crash into a loud build error.