Skip to content

Fix exception thrown when streaming certain XML values via XmlReader - #4757

Merged
cheenamalhotra merged 2 commits into
dotnet:mainfrom
sharpjs:dev/sharpjs/xml-encoding
Oct 8, 2026
Merged

cheenamalhotra merged 2 commits into
dotnet:mainfrom
sharpjs:dev/sharpjs/xml-encoding

Conversation

@sharpjs

@sharpjs sharpjs commented Sep 26, 2026

Copy link
Copy Markdown
Contributor

Description

This PR fixes an exception thrown when streaming certain xml values.

When an xml value streams from the database server, it streams as BOM-less UTF-16LE, according to my experiments. When SqlClient constructs an XmlReader to present that value to the caller (for example, in sqlDataReader.GetXmlReader(…)), SqlClient does not specify the encoding. Instead, the default encoding detection logic applies. That logic detects UTF-16LE when the stream begins with a BOM or a <; otherwise that logic assumes UTF-8. Because an xml value can be an XML fragment, there exist xml value BOM-less UTF-16LE streams that the detection logic will detect wrongly as UTF-8. One such value is:

foo<bar/>

Some SqlClient code paths that return an XmlReader instance apply a SqlStream wrapper that synthesizes a BOM, ensuring that the detection logic detects UTF-16LE. Other code paths do not, and the detection logic is free to detect UTF-8 incorrectly. In that case, the returned XmlReader throws an exception on the first call to xmlReader.Read(), as the reader becomes confused by what it sees as a NUL every second character. That is the bug.

This PR alters XmlReader construction to set the encoding to UTF-16LE (Encoding.Unicode) explicitly, thus directly bypassing the encoding detection logic and avoiding the bug.

Once the fix is in place, the SqlStream BOM-synthesizing behavior might be removable. In the interest of focus, that is left for a possible future PR.

This PR carries no API changes and no documentation impact.

Issues

Closes #4756.

Testing

This PR modifies the following test methods in DataReaderStreamsTest (manual tests, set 2) to test for this fix:

  • GetFieldValueAsync_OfXmlReader
  • GetFieldValue_OfXmlReader
  • GetXmlReader

Specifically, this PR prefixes the root-level text foo to test xml values so that an XmlReader constructed without an explicit encoding will encounter the pathological case in the encoding detection logic.

This closes a bug where UTF-16 XML streams not starting with '<' could
be detected incorrectly as UTF-8, causing exceptions on reading.
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@sharpjs

sharpjs commented Sep 26, 2026

Copy link
Copy Markdown
Contributor Author

@dotnet-policy-service agree

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The fix is focused and adequately covers synchronous, asynchronous, sequential, and buffered reader paths; the documentation comment is non-blocking.

Review effort: Balanced
Findings: 1 Low severity

Open (1)
What changed in this PR

Fixes XML fragment streaming by explicitly decoding SQL XML streams as UTF-16LE.

Changes:

  • Supplies an Encoding.Unicode parser context when creating XML readers.
  • Expands regression tests to cover fragments beginning with text.
File Description
SqlTypeWorkarounds.cs Explicitly configures UTF-16LE decoding.
DataReaderStreamsTest.cs Adds fragment-focused coverage and serialization support.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

This change was suggested by Copilot review:
dotnet#4757 (comment)
Copilot AI balanced review requested due to automatic review settings October 1, 2026 23:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The focused encoding fix is consistent with SQL XML streams and covers affected sync, async, and sequential-access paths.

Review effort: Balanced
Findings: None

Resolved since last review (1)

@mdaigle

mdaigle commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).
1 pipeline(s) were filtered out due to trigger conditions.

@codecov

codecov Bot commented Oct 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 64.66%. Comparing base (671d010) to head (539e565).
⚠️ Report is 24 commits behind head on main.

❗ There is a different number of reports uploaded between BASE (671d010) and HEAD (539e565). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (671d010) HEAD (539e565)
CI-SqlClient 1 0
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4757      +/-   ##
==========================================
- Coverage   71.96%   64.66%   -7.31%     
==========================================
  Files         291      286       -5     
  Lines       45110    68405   +23295     
==========================================
+ Hits        32462    44231   +11769     
- Misses      12648    24174   +11526     
Flag Coverage Δ
CI-SqlClient ?
PR-SqlClient-Project 64.66% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mdaigle
mdaigle requested a review from apoorvdeshmukh October 7, 2026 20:50
@sharpjs

sharpjs commented Oct 8, 2026

Copy link
Copy Markdown
Contributor Author

I assume this PR is still in review and not waiting on any action from me. If I am mistaken, just let me know. Thanks!

@cheenamalhotra
cheenamalhotra merged commit 5eaa932 into dotnet:main Oct 8, 2026
211 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Exception thrown when streaming certain XML values via XmlReader

5 participants