Skip to content

Fix for a bug in unicode surrogate pair handling#58

Open
bogbert wants to merge 1 commit into
json-parser:masterfrom
bogbert:master
Open

Fix for a bug in unicode surrogate pair handling#58
bogbert wants to merge 1 commit into
json-parser:masterfrom
bogbert:master

Conversation

@bogbert

@bogbert bogbert commented Aug 26, 2016

Copy link
Copy Markdown

eg: the JSON string "\ud841\udf31" should contain the character with the unicode scalar value U+20731, not U+10731

eg: the JSON string "\ud841\udf31" should contain the character with the unicode scalar value U+20731, not U+10731
@LB-- LB-- added this to the v1.2.0 milestone Aug 16, 2021
@DimitriPapadopoulos

Copy link
Copy Markdown
Contributor

@LB-- I'm wondering whether this should be added to the v1.1.1 milestone. It looks like Unicode strings are not read correctly.

@bogbert It would be great if a test could be added. File tests/valid-0012.json already contains UTF-16 surrogate pairs as far as I can understand. If the bug does not show with the existing examples, could you add one this 𠜱 example?

"CJK Unified Ideograph-20731 (U+20731)": "\ud841\udf31",

@LB-- LB-- modified the milestones: v1.2.0, v1.1.1 Nov 6, 2021
@LB--

LB-- commented Nov 6, 2021

Copy link
Copy Markdown
Member

I've added some basic string comparison tests in 174e5a7 as a good starting point for this. It seems like this change does in fact fix the issue, but I'll want to get more comparisons in the tests first to make sure nothing else gets broken.

@DimitriPapadopoulos

DimitriPapadopoulos commented Dec 29, 2021

Copy link
Copy Markdown
Contributor

See for example:

I have forked the bogbert/masterbranch into Dimitri/Papadopoulos/bogbert to rebase it and attempt to add more tests.

This was referenced Dec 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants