Repository navigation
fix: read a number grouped with an apostrophe, such as 1'239.09, under de-CH - #127
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A number grouped with an apostrophe —
1'239.09,1'234'567.89, as Swiss formats write it — is now profiled and imported as a number under a culture that groups that way (de-CH,fr-CH,it-CH), typed with either apostrophe. Before, the profile called it text even withde-CHinAnalysisOptions.Cultures, while the import read it.Why it was text.
CouldBeNumeric, the filter that spares the parse for values no tried culture could read, let through digits,.,,, signs, the exponent and three spaces, but no apostrophe. It now lets'and’through as well. A value with an apostrophe under a culture that does not group with one still fails the parse, so the default cultures read every file as before.Either apostrophe. Which one
de-CHgroups with depends on the runtime's ICU data:'(U+0027) on one system,’(U+2019) on another. .NET matches only the culture's own separator, so a file typed with the ASCII apostrophe would read on one machine and not on the next. For a culture that groups with an apostrophe, a value holding the other one is parsed from a copy that uses the culture's own. The copy sits on the stack and is made only when needed. The rule lives inNumberReading.TryParse, which the import now calls for integers and decimals, so the profile and the import read a value the same way.Performance. The first version did the group check and the apostrophe lookup twice for every decimal: once for the integer attempt and again for the decimal one. That made analysis of the 5M-row csv fixture about 5% slower. The profiler now does the lookup once per culture and the group check once per value, as before. Measured as analysis of the whole file, min of 5 alternating runs against
mainon a busy machine, so provisional:docs/operations.mdnow says which look-alike separators a culture reads: an ordinary space for a no-break-space culture, which .NET already did and a test now pins, and either apostrophe.Closes #126