_pickle crashes when read() returns a bytes subclass for large payloads

Crash report

Loading a pickle containing a large string (>= 1MB) using a custom file-like object that returns a bytes subclass from read() causes a deterministic segmentation fault in CPython 3.15.

PoC

import pickle
import gc

# GC-tracked heap type
class MyBytes(bytes):
    def __init__(self, *a): 
        self.tag = "meta"

class ReaderSub:
    def __init__(self, data): 
        self.d = data
        self.p = 0
        
    def read(self, n):
        c = self.d[self.p:self.p + n]
        self.p += len(c)
        return MyBytes(c)
        
    def readline(self):
        i = self.d.find(b'\n', self.p)
        line = self.d[self.p:i+1] if i >= 0 else self.d[self.p:]
        self.p = len(self.d) if i < 0 else i + 1
        return line

doc = pickle.dumps({'v': 'B' * (3 * 1024 * 1024)})

obj = pickle.load(ReaderSub(doc))
print("Successfully loaded.")

Code Output

Running on Python 3.15.0b1+:

realloc(): invalid old size

Running with Python 3.14.7:

Successfully loaded.

Root Cause

The crash occurs in Modules/_pickle.c, specifically within the new chunked read loop in _Unpickler_ReadFromFile (around line 1445).

The loop does this:

data = _Pickle_FastCall(self->read, len);
/* ... */
if (_PyBytes_Resize(&data, cursize) < 0)

_PyBytes_Resize is being called directly on the foreign object returned by self->read().
Because MyBytes is a subclass of bytes, it passes PyBytes_Check(). However, because it is a Python-level subclass, it is a heap type and is therefore tracked by the Garbage Collector.

This means the actual allocation includes a PyGC_Head before the object data. When _PyBytes_Resize calls PyObject_Realloc(v, PyBytesObject_SIZE + newsize), the pointer v is an interior pointer (offset by 32 bytes from the true malloc address). Reallocating an interior pointer corrupts the heap and immediately crashes the interpreter.

CPython versions tested on:

3.15

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.15.0rc2 (main, Sep 29 2026, 15:02:39) [Clang 22.1.3 ]

Linked PRs

@vstinner

cc @vstinner
I guess _PyBytes_IsMutable needs PyBytes_CheckExact?

_PyBytes_Resize is being called directly on the foreign object returned by self->read().
Because MyBytes is a subclass of bytes, it passes PyBytes_Check(). However, because it is a Python-level subclass, it is a heap type and is therefore tracked by the Garbage Collector.

Oh. It seems like it's a bad idea to accept bytes subclass in _PyBytes_Resize(). For example, if it's refcount is greater than 1, _PyBytes_Resize() returns a bytes object, not an instance of the subclass.

And so, yes, _pickle should not call _PyBytes_Resize() if PyBytes_CheckExact() is false.

@drakeo338

Well, _PyBytes_Resize is documented, so it might be better to keep the documented behaviour (“creating a new bytes object and destroying the old one, only more efficiently” -- of course, without the optimization in this case).

_PyBytes_Resize() returns a bytes object, not an instance of the subclass.

IMO, that's fine. BytesSubclass(...)[:] and most other operations will do the same thing.

@drakeo338, instead of your fix in pickle.c, could you add a !PyBytes_CheckExact(v) to _PyBytes_ResizeKeepOnError (just before the !_PyObject_IsUniquelyReferenced)?

Navigation Menu

Repository navigation

_pickle crashes when read() returns a bytes subclass for large payloads #158841

Crash report

PoC

Code Output

Root Cause

CPython versions tested on:

Operating systems tested on:

Output from running 'python -VV' on the command line:

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

_pickle crashes when read() returns a bytes subclass for large payloads #158841

Description

Crash report

PoC

Code Output

Root Cause

CPython versions tested on:

Operating systems tested on:

Output from running 'python -VV' on the command line:

Linked PRs

Activity

encukou commented on Oct 5, 2026

vstinner commented on Oct 5, 2026

encukou commented on Oct 8, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions