Skip to content

Fix decoder integer ranges, string ends and termination, and JSON bools - #79

Merged
garethpotter merged 1 commit into
mainfrom
fix-decoder-ranges-and-json-bools
Nov 6, 2026
Merged

garethpotter merged 1 commit into
mainfrom
fix-decoder-ranges-and-json-bools

Conversation

@garethpotter

@garethpotter garethpotter commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes to the binary and text decoders found by the server protocol tests from #78, plus JSON bools in the text encoder. Re-enables the #78 tests these fix.

Binary decoder (ThingSetBinaryDecoder.cpp)

  • 8- and 16-bit integers: the largest value of each type was rejected, because the check was i < UINT8_MAX rather than <= (so 255, 65535, 127 and 32767 failed). The signed decoders also had no lower bound, so -129 decoded into an int8_t as 127. Both bounds are now checked.
  • Strings into a char buffer: a string exactly as long as the buffer was accepted, then the terminator was written one byte past the end. A string must now be shorter than the buffer.

Text decoder (ThingSetTextDecoder.hpp/.cpp)

  • Integers: there was no range check at all, so 256 decoded into a uint8_t as 0. A new decodeInteger<T>() parses with strtoll/strtoull and fails if the value doesn't fit T, if a negative is given for an unsigned type, or if the token isn't a number (previously true decoded as 0). int64_t used strtol, which is 32 bits on 32-bit targets such as Zephyr. Non-integer numbers still truncate (50.6 into an int32_t gives 50), as thingset-node-c does.
  • Lists and maps ending in a bool, null or string: the end of a list or map is found by looking for ] or } at the read position, but only the number decoders moved that position past their value. So a list ending in true, null or a string couldn't be closed; in practice a function with a bool parameter couldn't be called in text mode (!Exec/xVoidParams [true] gave A0). Every single-value decode now leaves the read position at the end of its token, and the end checks skip whitespace (so [ 1 , 2 ] works) and stay within the buffer.
  • Strings into a char buffer: no terminator was written, so the result depended on what was in the buffer before (this is why TextDecoder.DecodeString failed intermittently). The terminator is now written, and the string must leave room for it.

Text encoder

Bools are written as true/false instead of 1/0.

Tests

  • Re-enabled from Add server protocol tests ported from thingset-node-c #78: BinaryUpdateOutOfRange, TextUpdateOutOfRange, TextExecVoidWithParams, BinaryUpdateTextGetBool, plus the maximum-value cases in the serde tables.
  • New TextDecoder tests: bool literals in a list, lists ending in a string or null, whitespace before ], out-of-range and negative-into-unsigned integers, a non-number into an integer, and a string with no room for its terminator.
  • New BinaryDecoder tests: a decoded string is terminated, and one with no room for the terminator is rejected without writing past the buffer.
  • TextEncoder: the bool tests now expect true/false.

Two existing tests were wrong:

  • TextDecoder.DecodeInt64 decoded -9223372036854775808 into a uint64_t and only passed because negatives used to wrap. It now uses int64_t, as its name and expected value (INT64_MIN) intend.
  • TextRecords.EncodeAndDecodeSimpleRecord gave the decoder the whole 1024-byte buffer, not just the encoded part, so uninitialised bytes after the JSON were parsed too. It failed depending on which tests ran before it (on macOS, in the full suite). It now passes encoder.getEncodedLength().

The full suite passes: 234 tests, 10 runs out of 10, with gcc in the CI container, and 206 tests, 3 runs out of 3, on macOS. The Zephyr build is left to CI.

🤖 Generated with Claude Code

Binary decoder:
- the 8- and 16-bit integer decoders rejected the largest value of each
  type (value < max instead of <=), and the signed ones had no lower
  bound, so -129 decoded into an int8_t as 127
- decoding a string into a char buffer accepted one exactly the size of
  the buffer and then wrote the terminator past its end

Text decoder:
- integers are range checked, negatives are rejected for unsigned
  types, and a token that is not a number fails instead of decoding
  as 0; int64_t now parses with strtoll rather than strtol
- each single value (including bools, null and strings, which did not
  before) leaves the read position at the end of its token, so a list
  or map whose last value is one of these can be closed. This made
  functions with a bool parameter impossible to call in text mode
- the end of a list or map is found after skipping whitespace
- decoding a string into a char buffer now writes the terminator, and
  requires room for it

Text encoder: bools are written as true/false rather than 1/0.

Tests: re-enables the server protocol tests these fix and adds decoder
tests. Corrects TextDecoder.DecodeInt64, which decoded into a uint64_t,
and TextRecords.EncodeAndDecodeSimpleRecord, which decoded the whole
buffer including uninitialised bytes after the encoded JSON, so it
failed depending on which tests ran before it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@garethpotter
garethpotter force-pushed the fix-decoder-ranges-and-json-bools branch from f31d209 to 4b5d28e Compare November 6, 2026 13:20
@garethpotter
garethpotter changed the base branch from server-protocol-tests to main November 6, 2026 13:20
@garethpotter garethpotter changed the title Fix decoder integer ranges, text list/map ends and JSON bools Fix decoder integer ranges, string ends and termination, and JSON bools Nov 6, 2026
@ethan-duckett-brill

Copy link
Copy Markdown
Collaborator

Looks good to me!

@garethpotter
garethpotter merged commit 7ad61c6 into main Nov 6, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants