Skip to content

THRIFT-6462: Decode JSON escapes above U+7FFF in the JavaME TJSONProtocol - #4061

Merged
Jens-G merged 1 commit into
apache:masterfrom
Jens-G:THRIFT-6462
Oct 28, 2026
Merged

Jens-G merged 1 commit into
apache:masterfrom
Jens-G:THRIFT-6462

Conversation

@Jens-G

@Jens-G Jens-G commented Oct 28, 2026

Copy link
Copy Markdown
Member

THRIFT-6462

readJSONString in lib/javame/src/org/apache/thrift/protocol/TJSONProtocol.java assembled the four hex digits of an escape into a short, so the escapes of U+8000 to U+FFFF became negative code units. The surrogate branches cast the value back to char and decoded pairs correctly. The plain-character branch, however, passed the negative value to toUTF8(int), which wrote only its low byte. The read did not fail; the string came back with a different character.

This change keeps the code unit as a char, as the Java library does since THRIFT-4408 (bd3ba6f).

New test lib/javame/test/TestJSONUnicodeEscapes.java, which the lib-javame CI job runs with the other test/Test*.java:

escape of master this branch
U+0041, U+00E9, U+07FF, U+0800, U+7FFF read correctly read correctly
U+8000, U+E000 U+0000 the character
U+D7FF, U+FFFF U+FFFD the character
U+9A41 U+0041 the character
a surrogate pair (U+1F600) read correctly read correctly
escapes between plain characters wrong characters in place read correctly

The tests were built and run the way the lib-javame job runs them (JDK 17, thrift:jammy): all five lib/javame tests pass. On master the new test fails 6 of 12 checks.

  • Apache Jira ticket: THRIFT-6462
  • Single commit
  • Code change, so the commit carries no CI-skip marker.

🤖 Generated with Claude Code

…ocol

Client: javame

readJSONString assembled the four hex digits of an escape into a short,
so the escapes of U+8000 to U+FFFF became negative code units. The
surrogate branches cast the value back to char, but the plain-character
branch passed it to toUTF8(int), which wrote only the low byte, and the
string read back with a different character. Keep the code unit as a
char, as the Java library does since THRIFT-4408.

TestJSONUnicodeEscapes reads the escapes of characters below and above
U+8000, a surrogate pair, and escapes between plain characters.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mergeable mergeable Bot added the java Pull requests that update Java code label Oct 28, 2026
@Jens-G
Jens-G merged commit 02e29d7 into apache:master Oct 28, 2026
109 checks passed
@Jens-G
Jens-G deleted the THRIFT-6462 branch October 28, 2026 21:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

java Pull requests that update Java code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant