Repository navigation
fix(domain): match IDN TLDs against IANA list in IDNA-encoded form - #524
Open
rahul05ranjan wants to merge 1 commit into
Open
rahul05ranjan wants to merge 1 commit into
rahul05ranjan wants to merge 1 commit into
Conversation
With consider_tld=True, the TLD was looked up in the IANA list as written. The list stores internationalized TLDs in their ASCII (IDNA-encoded) form, so domains like пример.рф were rejected even though their punycode form xn--e1afmkfd.xn--p1ai was accepted. Encode the TLD before the lookup (falling back to the raw value when IDNA encoding fails, so unencodable TLDs are still rejected exactly like before). hostname() and url() pass consider_tld through to domain(), so they pick up the fix as well. Fixes python-validators#522 Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
With
consider_tld=True, the TLD was looked up in the IANA list as written in the input. The IANA list stores internationalized TLDs in their IDNA-encoded ASCII form (e.g.XN--P1AI), so a domain likeпример.рфwas rejected even though the identical punycode form was accepted:This encodes the TLD with IDNA before the IANA lookup. When IDNA encoding of the TLD fails, the raw value is kept so the lookup still rejects it, matching the previous behavior for unencodable/unknown TLDs.
hostname()andurl()passconsider_tldthrough todomain(), so they pick up the fix as well.Tests
Added regression cases to
tests/test_domain.py:пример.рфand trailing-dot form withrfc_1034=Trueпример.испытание(a non-IANA IDN TLD that encodes fine, so it must still be rejected after encoding)Verification
The 18 failures are pre-existing on current master (all in
tests/crypto_addresses/test_eth_address.py, verified by stashing this change and re-running: same 18 failed, 1095 passed — this PR adds 3 passing tests). All domain/hostname/url tests pass, including the new IDN TLD cases.Fixes #522