ZstdDecompressor and LZMADecompressor raise on repeated decompress(b'', max_length=0) calls with output pending #160121

Bug description

When a compression.zstd.ZstdDecompressor has decompressed output waiting (because an earlier max_length cut it short), calling decompress with max_length=0 works 15 times and fails on the 16th:

>>> from compression import zstd
>>> d = zstd.ZstdDecompressor()
>>> d.decompress(zstd.compress(b"\x00"), max_length=0)
b''
>>> for _ in range(15):
...     d.decompress(b"", max_length=0)
...
>>> d.decompress(b"", max_length=0)
Traceback (most recent call last):
  File "<python-input-4>", line 1, in <module>
    d.decompress(b"", max_length=0)
_zstd.ZstdError: Unable to decompress Zstandard data: Operation made no progress over multiple calls, due to output buffer being full

Every one of those calls is a legitimate "buffer this, give me nothing yet" request, and needs_input is False throughout, so I would expect b'' each time with no error. The decompressor is not actually broken afterwards: a following decompress(b"", -1) still returns b"\x00" with eof set, so this is an error raised on a valid call rather than lost data.

LZMADecompressor fails the same sequence, on the fourth call rather than the sixteenth:

>>> import lzma
>>> d = lzma.LZMADecompressor()
>>> d.decompress(lzma.compress(b"\x00"), max_length=0)
b''
>>> for _ in range(3):
...     d.decompress(b"", max_length=0)
...
>>> d.decompress(b"", max_length=0)
Traceback (most recent call last):
  File "<python-input-4>", line 1, in <module>
    d.decompress(b"", max_length=0)
_lzma.LZMAError: Insufficient buffer space

That is the mechanism described in #160000 (liblzma's LZMA_BUF_ERROR leaking through when the output buffer is empty) showing up with input buffered rather than on a fresh decompressor; I expect the fix for that issue covers it, but it seemed worth recording the second shape since the fresh-decompressor test alone would not catch a regression here. BZ2Decompressor and zlib._ZlibDecompressor return b'' for any number of such calls.

The zstd mechanism: decompress_lock_held calls ZSTD_decompressStream even when max_length is 0, so with a zero-size output buffer: Modules/_zstd/decompressor.c lines 229-233. libzstd counts consecutive ZSTD_decompressStream calls that consume no input and produce no output (ZSTD_NO_FORWARD_PROGRESS_MAX, which is 16) and returns ZSTD_error_noForwardProgress_destFull once the limit is hit, as protection against callers looping forever. The simplest fix is probably to return b'' without calling into libzstd at all when max_length == 0, since there is nothing such a call can do except (with new input) append to the internal input buffer, which happens outside this loop anyway.

Found by running the Hypothesis tests from #159960 under Atheris, with the _zstd module compiled with -fsanitize=fuzzer-no-link. libFuzzer hit it within about twenty seconds, whereas plain Hypothesis had not found it in many runs, because it needs sixteen consecutive max_length=0 values; the zstd reproducer above is the shrunk result. Both backends are now pinned in that PR as test_max_length_zero_with_pending_output under expectedFailure.

CPython versions tested on

CPython main (39bc958), libzstd 1.5.7, liblzma 5.8.4

Operating systems tested on

Linux

cc @emmatyping