forkserver requires a writeable temp file system

Bug report

Bug description:

We run containers in our k8s without any writeable filesystem (readOnlyRootFilesystem=True), including no writeable temp file system.

The following code therefore raises:

>>> tempfile._get_default_tempdir()
Traceback (most recent call last):
  File "<python-input-1>", line 1, in <module>
    tempfile._get_default_tempdir()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.14/tempfile.py", line 222, in _get_default_tempdir
    raise FileNotFoundError(_errno.ENOENT,
                            "No usable temporary directory found in %s" %
                            dirlist)
FileNotFoundError: [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/']

Unfortunately, multiprocessing defaults to forkserver (in this case started by granian, see emmett-framework/granian#905):

cpython/Lib/multiprocessing/context.py

Lines 326 to 339 in 5d3f02a

    
           _concrete_contexts = { 
        
               'fork': ForkContext(), 
        
               'spawn': SpawnContext(), 
        
           } 
        
           if reduction.HAVE_SEND_HANDLE: 
        
               _concrete_contexts['forkserver'] = ForkServerContext() 
        
           # bpo-33725: running arbitrary code after fork() is no longer reliable 
        
           # on macOS since macOS 10.14 (Mojave). Use spawn by default instead. 
        
           # gh-84559: We changed everyones default to a thread safeish one in 3.14. 
        
           if reduction.HAVE_SEND_HANDLE and sys.platform != 'darwin': 
        
               _default_context = DefaultContext(_concrete_contexts['forkserver']) 
        
           else: 
        
               _default_context = DefaultContext(_concrete_contexts['spawn'])

And then fails if it started on such a read only filesystem.

[INFO] Starting granian (main PID: 1)
[INFO] Listening at: http://0.0.0.0:8009
Traceback (most recent call last):
  File "/opt/app/venv/bin/granian", line 10, in <module>
    sys.exit(cli.entrypoint())
             ~~~~~~~~~~~~~~^^
  File "/opt/app/venv/lib/python3.14/site-packages/granian/cli.py", line 606, in entrypoint
    cli(auto_envvar_prefix='GRANIAN')
    ~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app/venv/lib/python3.14/site-packages/click/core.py", line 1524, in __call__
    return self.main(*args, **kwargs)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/opt/app/venv/lib/python3.14/site-packages/click/core.py", line 1445, in main
    rv = self.invoke(ctx)
  File "/opt/app/venv/lib/python3.14/site-packages/click/core.py", line 1308, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app/venv/lib/python3.14/site-packages/click/core.py", line 877, in invoke
    return callback(*args, **kwargs)
  File "/opt/app/venv/lib/python3.14/site-packages/granian/cli.py", line 600, in cli
    server.serve()
    ~~~~~~~~~~~~^^
  File "/opt/app/venv/lib/python3.14/site-packages/granian/server/mp.py", line 513, in serve
    super().serve(spawn_target, target_loader, wrap_loader)
    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app/venv/lib/python3.14/site-packages/granian/server/common.py", line 760, in serve
    serve_method(spawn_target, target_loader)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app/venv/lib/python3.14/site-packages/granian/server/common.py", line 588, in _serve
    self.startup(spawn_target, target_loader)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app/venv/lib/python3.14/site-packages/granian/server/common.py", line 494, in startup
    self._spawn_workers(spawn_target, target_loader)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app/venv/lib/python3.14/site-packages/granian/server/common.py", line 346, in _spawn_workers
    wrk.start()
    ~~~~~~~~~^^
  File "/opt/app/venv/lib/python3.14/site-packages/granian/server/common.py", line 72, in start
    self.inner.start()
    ~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.14/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
                  ~~~~~~~~~~~^^^^^^
  File "/usr/lib/python3.14/multiprocessing/context.py", line 306, in _Popen
    return Popen(process_obj)
  File "/usr/lib/python3.14/multiprocessing/popen_forkserver.py", line 35, in __init__
    super().__init__(process_obj)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/usr/lib/python3.14/multiprocessing/popen_fork.py", line 20, in __init__
    self._launch(process_obj)
    ~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/usr/lib/python3.14/multiprocessing/popen_forkserver.py", line 51, in _launch
    self.sentinel, w = forkserver.connect_to_new_process(self._fds)
                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/usr/lib/python3.14/multiprocessing/forkserver.py", line 89, in connect_to_new_process
    self.ensure_running()
    ~~~~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.14/multiprocessing/forkserver.py", line 166, in ensure_running
    address = connection.arbitrary_address('AF_UNIX')
  File "/usr/lib/python3.14/multiprocessing/connection.py", line 82, in arbitrary_address
    return os.path.join(util.get_temp_dir(),
                        ~~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.14/multiprocessing/util.py", line 216, in get_temp_dir
    base_tempdir = _get_base_temp_dir(tempfile)
  File "/usr/lib/python3.14/multiprocessing/util.py", line 171, in _get_base_temp_dir
    base_tempdir = tempfile.gettempdir()
  File "/usr/lib/python3.14/tempfile.py", line 312, in gettempdir
    return _os.fsdecode(_gettempdir())
                        ~~~~~~~~~~~^^
  File "/usr/lib/python3.14/tempfile.py", line 305, in _gettempdir
    tempdir = _get_default_tempdir()
  File "/usr/lib/python3.14/tempfile.py", line 222, in _get_default_tempdir
    raise FileNotFoundError(_errno.ENOENT,
                            "No usable temporary directory found in %s" %
                            dirlist)
FileNotFoundError: [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/workdir']

Given that it already checks for some other problems (the SEND_HANDLE) and defaults to spawn if there are problems, it would be nice to also check for read only tempfiles and default to spawn in that case.

e.g. something long the lines of

try:
     tempfile.gettempdir()
     CAN_RUN_FORKERSERVER = True
except FileNotFoundError: # read only filesystem
    CAN_RUN_FORKERSERVER = False

And then use that similarly to HAVE_SEND_HANDLE.

CPython versions tested on:

3.14

Operating systems tested on:

Linux

Linked PRs

Maybe using /dev/shm as a fallback base dir when tempfile.gettempdir() fails is also a good solution?

@gpshead

I think it is overcomplicate in this case. We can default to spawn in this case I guess or document this and force the user to set spawn as the default start method.

cc @gpshead

adding a check and falling back to spawn in that case makes sense to me.

pythongh-155717: use `spawn` as the default start method for read-onl…

gh-155717: use `spawn` as the default start method for read-only file…

gh-155717: skip `TMPDIR` writeability test if process user is root (#…

This has been fixed in 3.16 and I'm waiting for the backports to land on 3.14 and 3.15. I don't know if I should actually fix this on 3.13 since the default start method is still 'fork'. Would it make sense to forbid switching to 'forkserver' even if the user explicitly requests it? For instance:

Start Python on a read-only filesystem
Do some operations
Spawn a process with a context with a 'forkserver' start method but for which TMPDIR is writeable

Detecting whether 'forkserver' will not be broken at runtime (and not just at startup time) may require more work that I'm not sure it's really necessary. We can however extend the docs about this.

i'd say just leave it as a fix that lands back to 3.14 and not worry about 3.13.

[3.15] gh-155717: use `spawn` as the default start method for read-on…

@picnixz

The performance regression suite I run against CPython builds picked up a cold-start regression from #155827: import multiprocessing got much slower, even in programs that never start a process. Something as simple as this pays the full cost:

from multiprocessing import cpu_count

workers = cpu_count()

This isn't specific to any platform, I can reproduce it locally too, but I validate these changes on AWS Lambda, where cold starts make it easy to measure. There (x86_64, 1024 MB, PGO+LTO build), import multiprocessing went from about 65 ms to 115 ms (+50 ms, about +75%).

Most of the extra time comes from imports, not from the writability check itself. import multiprocessing now also loads multiprocessing.util (which imports subprocess) and tempfile (which imports shutil, re, random and the compression modules). Locally, these imports cost about 9 ms, while the check itself takes about 0.2 ms.

Programs that do start processes pay this too, because each new interpreter imports multiprocessing again: the forkserver, the resource tracker and every spawn worker. On Lambda, where CPU is limited, starting a single Process was about 20 ms slower.

The read-only behavior can be kept without doing this work at import time, by choosing the default context the first time it's needed:

DefaultContext resolves the default lazily, and the writability check only runs then.
util.py imports tempfile locally where it's used, as before gh-155717: use spawn as the default start method for read-only filesystems #155827.

On the same Lambda setup, compared with the code before #155827:

	#155827	#155827 + lazy default context
`import multiprocessing`	+50.2 ms (+75%)	no significant change
starting one `Process`	+23.0 ms (+7.5%)	no significant change

A read-only temporary directory still gives spawn, a writable one still gives forkserver, and the multiprocessing and concurrent.futures test suites pass on Linux and macOS. I also added a test that importing multiprocessing doesn't resolve the default context.

This is complementary to #158410: with only the lazy change, the FileNotFoundError for a missing temp directory just moves from the import to the first use, so both are needed.

Since the 3.14 backport (#155886) is still open, it would be nice to sort this out before it lands. Happy to send a PR. cc @picnixz @gpshead

@gpshead

I think that's because the contexts are created at import time IIRC

That was my first guess too! But when I looked closer, creating the contexts turned out to be cheap, and that was already happening before #155827. Most of the time goes into picking the default at import time, since that's what now brings in multiprocessing.util and tempfile.

no worries, its mostly import time for tempfile and transitive deps. fix coming.

Thanks @gpshead! Happy to run the same benchmarks against your fix once it's up, including the Lambda cold-start ones.

sure, give #159254 a whirl. it's basically deferring the imports until first need.

22. sure, give #159254 a whirl. it's basically deferring the imports until first need.

Nice! I'll take a look. Thanks for the quick turnaround.

gh-155717: Determine multiprocessing's default start method lazily (G…

pythongh-155717: Reword the NEWS entry for the default start method c…

gh-155717: Reword the NEWS entry for the default start method change (G…

pythongh-155717: use `spawn` as the default start method for read-onl…

[3.14] pythongh-155717: Determine multiprocessing's default start met…

pythongh-155717: Reword the NEWS entry for the default start method c…

[3.15] gh-155717: Determine multiprocessing's default start method la…

[3.14] gh-155717: use `spawn` as the default start method for read-on…

thanks everyone!

Navigation Menu

Repository navigation

forkserver requires a writeable temp file system #155717

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

2 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	_concrete_contexts = {
	'fork': ForkContext(),
	'spawn': SpawnContext(),
	}
	if reduction.HAVE_SEND_HANDLE:
	_concrete_contexts['forkserver'] = ForkServerContext()

	# bpo-33725: running arbitrary code after fork() is no longer reliable
	# on macOS since macOS 10.14 (Mojave). Use spawn by default instead.
	# gh-84559: We changed everyones default to a thread safeish one in 3.14.
	if reduction.HAVE_SEND_HANDLE and sys.platform != 'darwin':
	_default_context = DefaultContext(_concrete_contexts['forkserver'])
	else:
	_default_context = DefaultContext(_concrete_contexts['spawn'])

forkserver requires a writeable temp file system #155717

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

aisk commented on Aug 13, 2026

picnixz commented on Aug 13, 2026

gpshead commented on Aug 14, 2026

picnixz commented on Aug 16, 2026

gpshead commented on Aug 18, 2026

leandrodamascena commented on Oct 12, 2026

2 remaining items

leandrodamascena commented on Oct 12, 2026

gpshead commented on Oct 12, 2026

leandrodamascena commented on Oct 12, 2026

gpshead commented on Oct 13, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions