Skip to content

feat: strict anydata validation with payload schema context - #50

Merged
lllyfeng merged 3 commits into
yang-central:mainfrom
Deepya1502:pr/strict-anydata-validation
Oct 8, 2026
Merged

lllyfeng merged 3 commits into
yang-central:mainfrom
Deepya1502:pr/strict-anydata-validation

Conversation

@Deepya1502

@Deepya1502 Deepya1502 commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Make strict payload-schema validation for non-empty anydata content opt-in across JSON, XML, CBOR, and Protocol Buffers codecs.
  • By default, parse anydata content using the enclosing schema when no payload schema context is provided.
  • Reject primitive and null anydata values with BAD_ELEMENT, as required by RFC 7951 §5.5.
  • Recursively validate successfully parsed nested payload documents and rebase validation paths beneath the enclosing anydata node.
  • Add codec coverage, payload schemas, and documentation for supported anydata shapes.

The branch is rebased onto the current yang-central/yangkit main.

Non-empty anydata content now requires a matching payload schema context
resolved via AnydataValidationSupport. On resolution failure the outer
anydata node is created without a payload value and an operation-failed
record is added; the enclosing document schema is never used as a
fallback, and content is never silently discarded.

Primitive anydata payloads (string/number/boolean/null/array) produce
ErrorTag.BAD_ELEMENT records per libyang's structured-input rule for
anydata, while preserving Yangkit's validation result format.

Validating the enclosing document recursively validates the nested
payload document, returning standard yangkit validation records for
mandatory, must, cardinality, unique, type, and leafref constraints
with paths rooted beneath the enclosing anydata node.

Applies to the JSON, XML, CBOR, and Protocol Buffers data codecs.
Updates tests and module READMEs accordingly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@lllyfeng lllyfeng left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Deepya1502, welcome, and thanks for this substantial contribution! Recursive validation of anydata payloads, with paths rebased under the anydata node, is something we've wanted for a while. Using the payload document's own schema context when serializing is also a real bug fix.

I checked out the branch, merged it with current main, and ran the full build. Everything compiles and all tests pass. CI hasn't started yet because the workflow needs approval for first-time contributors; I'll approve that separately so the checks can run.

My main concern is backward compatibility:

  1. A missing payload schema should not be an error by default. RFC 7950 §7.10 describes anydata as data "for which the data model is not known at module design time", and says knowing it later is possible but "not required". With this PR, any non-empty anydata parsed without a resolver produces an Error-level operation-failed record, and the content is dropped. I confirmed this breaks existing callers: the original JsonCodecNotificationsTest from main fails 2/5 on this branch (yp:datastore-contents and yp:value from YANG-Push). The PR only passes because it changes that test to supply a resolver. Could we make the strict behavior opt-in? For example, add AnydataValidationOptions#requirePayloadSchema(boolean) (default false). When it's off, keep today's fallback to the enclosing schema context. When it's on, use the new operation-failed behavior. Then please revert the change to JsonCodecNotificationsTest, so it keeps guarding the default behavior.

  2. Rejecting primitive anydata is fine. RFC 7951 §5.5 says anydata is encoded like a container (a name/object pair), so rejecting strings, numbers, booleans, null and arrays in JSON and CBOR is spec-compliant. Please mention it in the PR description as a behavior change so we can call it out in the release notes.

  3. Please split out the unrelated changes. The XML codec recursion refactor, addExistingListKey, and the unique null-schema-path fix look useful, but they're independent of anydata and don't have their own tests. Separate PRs would make them much easier to review and bisect.

Details are in the inline comments. Thanks again, happy to help iterate on this!

Comment thread yangkit-data-api/README.md Outdated
Comment thread yangkit-data-api/README.md Outdated
Make missing payload schemas an opt-in strict error, restore notification compatibility tests, and assert rebased validation paths. Remove unrelated XML and unique-schema-path changes from this PR.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@lllyfeng

lllyfeng commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Thanks @Deepya1502 for the quick turnaround, and for splitting the unrelated changes into #51, #52 and #53. That makes review much easier.

The opt-in requirePayloadSchema switch and the restored JsonCodecNotificationsTest look good. The default fallback also fixes the earlier IETFCasesTest.validateApp6Example CI failure, and CI is green now.

A few remaining items:

  1. PR description: please update it. It still says a payload schema is required. Please note that strict mode is opt-in, and that by default parsing falls back to the enclosing schema. Please also call out the behavior change: primitive and null anydata values are now rejected (BAD_ELEMENT), per RFC 7951 §5.5.
  2. JSON null: rejecting it as BAD_ELEMENT is fine. It's consistent with the review.
  3. AnyDataDataImpl#buildAnydataPath: could you explain why the parent's path is skipped when parentData.getContext().getParent() == null? Please add a short code comment as well.
  4. Non-blocking nit: strict mode is only detected via instanceof AnydataValidationOptions, so custom AnydataValidationContextResolver implementations can't turn it on. Maybe expose it on the resolver interface instead?

Merge order: #50 first. Then please rebase #51, #52 and #53, since they conflict with #50 and with each other.

@Deepya1502 Deepya1502 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the review comments are addressed, please review.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Deepya1502

Copy link
Copy Markdown
Contributor Author

Thanks for the feedback @lllyfeng. Updated the PR description, I added a short comment in  AnyDataDataImpl#buildAnydataPath  explaining that a root parent has no enclosing path to prepend.

I also updated  AnydataValidationContextResolver  with a default  isRequirePayloadSchema()  method, overridden it in  AnydataValidationOptions , and changed schema resolution to use the interface method. This allows custom resolvers to enable strict mode while preserving the default fallback behavior and lambda compatibility. Added a JSON codec regression test for a custom strict resolver; the targeted test passed.

@lllyfeng lllyfeng left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Deepya1502, all review points are addressed and CI is green. Approving and merging.

@lllyfeng
lllyfeng merged commit 8525020 into yang-central:main Oct 8, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants