Makefile / configure.ac: realpath is not available on macOS

Bug report

Makefile.pre.in and configure.ac are using the realpath command for some targets. For example WASI HOSTRUNNER uses realpath --relative-to to get the working directory relative to the srcdir root directory. macOS seems to lack realpath, breaking WASI tests, see #95828 (comment)

For WASI HOSTRUNNER we need to replace realpath somehow. Or find a different approach to fix testing with OOT builds. The HOSTRUNNER for WASI is complicated because we have to make srcdir available to WASI runtime environment. --mapdir /::$(srcdir) maps the srcdir to root / inside the runtime environment. wasmtime sets cwd to /, so our logic to find pybuilddir.txt does not work. That's why the HOSTRUNNER sets --env PYTHONPATH=/$(shell realpath --relative-to $(abs_srcdir) $(abs_builddir))/$(shell cat pybuilddir.txt):/Lib, which resolves to --env PYTHONPATH=/builddir/wasi/build/lib.wasi-wasm32-3.12:/Lib on my system. The prefix depends on the OOT builddir. The suffix is read from pybuilddir.txt, which is generated at the end of the build process.

Linked PRs

GH-95952: Require setting HOSTRUNNER when cross compiling for WASI #138203

macOS does not have a realpath command (at least not on macOS 12). There is a manual page, but that doesn't mention a relative-to option (it basically describes a command-line wrapper around realpath(3)).

UPDATE: The beta for the next major release of macOS does include a realpath(1) command, but that doesn't have the --relative-to option and hence isn't usable for your use case.

@brettcannon

@brettcannon, is this still relevant?

@brettcannon

@brettcannon, is this still relevant?

I have no idea (I certainly don't need it as I generate a shell script to avoid such issues), so I'm going to close it as unnecessary.

configure.ac still contains code using realpath options not supported on macOS:

dnl TODO: support other WASI runtimes
    dnl wasmtime starts the proces with "/" as CWD. For OOT builds add the
    dnl directory containing _sysconfigdata to PYTHONPATH.
    [WASI/*], [HOSTRUNNER='wasmtime run --env PYTHONPATH=/$(shell realpath --relative-to $(abs_srcdir) $(abs_builddir))/$(shell cat pybuilddir.txt):/Lib --mapdir /::$(srcdir) --'],

@ronaldoussoren

I believe @ronaldoussoren is referring to:

cpython/configure.ac

Lines 1417 to 1420 in 939fc6d

    
           dnl TODO: support other WASI runtimes 
        
           dnl wasmtime starts the proces with "/" as CWD. For OOT builds add the 
        
           dnl directory containing _sysconfigdata to PYTHONPATH. 
        
           [WASI/*], [HOSTRUNNER='wasmtime run --env PYTHONPATH=/$(shell realpath --relative-to $(abs_srcdir) $(abs_builddir))/$(shell cat pybuilddir.txt):/Lib --mapdir /::$(srcdir) --'],

I personally think that should be changed to error out if the host runner is not specified for WASI. With Tools/wasm/wasi.py (and the old Tools/wasm/wasm_build.py), there's enough help in setting that value externally. Otherwise we perhaps should have a --host-runner option to ./configure?

@ronaldoussoren

I believe @ronaldoussoren is referring to:

That's correct.

cpython/configure.ac

Lines 1417 to 1420 in 939fc6d

dnl TODO: support other WASI runtimes

dnl wasmtime starts the proces with "/" as CWD. For OOT builds add the

dnl directory containing _sysconfigdata to PYTHONPATH.

[WASI/*], [HOSTRUNNER='wasmtime run --env PYTHONPATH=/$(shell realpath --relative-to $(abs_srcdir) $(abs_builddir))/$(shell cat pybuilddir.txt):/Lib --mapdir /::$(srcdir) --'],

I personally think that should be changed to error out if the host runner is not specified for WASI. With Tools/wasm/wasi.py (and the old Tools/wasm/wasm_build.py), there's enough help in setting that value externally. Otherwise we perhaps should have a --host-runner option to ./configure?

I don't have an opinion on that, WASI/WebAsm support is interesting from the distance but I've never looked at it beyond reading articles ("how cool, CPython in a browser").

There are three workarounds for this on MacOS:

Install coreutils if you're a homebrew user
https://codereview.stackexchange.com/questions/242050/realpath-substitute-with-relative-to-and-relative-base-support has a shell script implementation of realpath with --relative-to support
Implementing realpath in Python should be easy enough (and Apple's compiler ships with Python 3, which means anyone playing with this should have Python 3 available)

For (3) something like this should work (I may have gotten the two arguments backwards):

HOSTRUNNER='python3 -c "import sys, pathlib; print(pathlib.Path(sys.argv[1]).resolve().relative_to(sys.argv[2]), walk_up=True))" "$(abs_builddir)" "$(abs_srcdir)"'

Just dropping automatically settings HOSTRUNNER for WASI is probably easier ;-)

pythonGH-95952: Require setting `HOSTRUNNER` when cross compiling for…

GH-95952: Require setting `HOSTRUNNER` when cross compiling for WASI (#…

Fixed this for 3.15.

pythonGH-95952: Require setting `HOSTRUNNER` when cross compiling for…

Navigation Menu

Repository navigation

Makefile / configure.ac: realpath is not available on macOS #95952

Bug report

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	dnl TODO: support other WASI runtimes
	dnl wasmtime starts the proces with "/" as CWD. For OOT builds add the
	dnl directory containing _sysconfigdata to PYTHONPATH.
	[WASI/*], [HOSTRUNNER='wasmtime run --env PYTHONPATH=/$(shell realpath --relative-to $(abs_srcdir) $(abs_builddir))/$(shell cat pybuilddir.txt):/Lib --mapdir /::$(srcdir) --'],

Makefile / configure.ac: realpath is not available on macOS #95952

Description

Bug report

Linked PRs

Activity

ronaldoussoren commented on Aug 15, 2022

erlend-aasland commented on Nov 30, 2023

brettcannon commented on Dec 1, 2023

ronaldoussoren commented on Dec 1, 2023

brettcannon commented on Dec 1, 2023

ronaldoussoren commented on Dec 2, 2023

ronaldoussoren commented on Dec 27, 2023

brettcannon commented on Sep 5, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions