Skip to content

Bug: case=False is not working correctly in the string accessor - #11747

Merged
headtr1ck merged 3 commits into
pydata:mainfrom
novain-shibu:fix-str-case-insensitive-literal
Oct 20, 2026
Merged

headtr1ck merged 3 commits into
pydata:mainfrom
novain-shibu:fix-str-case-insensitive-literal

Conversation

@novain-shibu

Copy link
Copy Markdown
Contributor

Description

I found two issues in the string accessor while testing different string dtypes against pandas.

1. str.replace() ignores case=False when regex=False

For example, replacing "HELLO" with "bye" in ["Hello World"] should give "bye World" when case=False. Instead, the original string remains unchanged, with no error.

This happens with regular strings, object-dtype arrays, and StringDType arrays. The problem is that the regex=False path uses Python's str.replace(), which does not support case-insensitive replacement.

2. str.contains() crashes with case=False on some dtypes

When case=False and regex=False are used together, str.contains() raises an AttributeError on object-dtype and StringDType arrays. Normal Unicode string arrays work.

The issue comes from passing the pattern, which is a plain string, to StringAccessor as though it were an array.

Proposed fix

  • For replace(), use re.sub() with re.IGNORECASE when case=False and regex=False. Escape the search pattern so it is treated literally, keep the replacement unchanged so backreferences are not interpreted, and preserve the existing n limit.
  • For contains(), compare the strings element by element using casefold(), with .upper() for bytes.
  • Add an entry to whats-new.rst.

Tests

I added four tests covering case-insensitive replacement across dtypes and the affected contains() and replace() cases for object and StringDType arrays.

The tests fail before the fix and pass after it. The 228 existing string-accessor tests also pass before and after the changes (run on 14 October).

I haven't rerun the tests after today's rebase because I don't have a test environment on this laptop. CI will need to confirm the results on the rebased changes.

Related PR

headtr1ck opened #11746 today, which also modifies contains() and replace() as part of using np.strings in the string accessor. However, it leaves the affected case=False logic unchanged, so these bugs remain.

Since both PRs touch the same functions, there may be a small conflict or rebase needed, depending on which one merges first. I'm happy to help resolve it.

I found these issues using the same approach as #11722: testing the same inputs across different dtypes and comparing the results with pandas, then investigating the differences.

One related issue I haven't addressed is that flags= is also ignored when regex=False.

Checklist

  • Tests added
  • User visible changes (including notable bug fixes) are documented in whats-new.rst

AI Disclosure

  • This PR contains AI-generated content.
    • I have tested any AI-generated content in my PR.
    • I take responsibility for any AI-generated content in my PR.
      Testing: The tests were run before the latest rebase; they have not been rerun afterward.
      Tools: Claude helped me investigate the behaviour, reason about the fixes, and develop the tests.

[PR opened by Claude Code on behalf of Novain Shibu]

str.replace ignored case=False when regex=False and returned the input
unchanged. str.contains with case=False, regex=False raised
AttributeError on object-dtype and StringDType arrays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Novain Shibu <georgeshibu380@gmail.com>
@welcome

welcome Bot commented Oct 16, 2026

Copy link
Copy Markdown

Thank you for opening this pull request! It may take us a few days to respond here, so thank you for being patient.
If you have questions, some answers may be found in our contributing guidelines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Novain Shibu <georgeshibu380@gmail.com>

@headtr1ck headtr1ck left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of minor points but looks good, thanks!

I assume case was only ever supposed to be used for regex, but I guess it doesn't hurt to just support it :)

Comment thread doc/whats-new.rst Outdated
the array. Previously, e.g. ``da.cumulative("time").argmax()`` returned indices
shifted by the number of missing values in front of the array (:issue:`11336`).
By `Michael Niklas <https://github.com/headtr1ck>`_.
- ``DataArray.str.replace`` now respects ``case=False`` when ``regex=False``;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- ``DataArray.str.replace`` now respects ``case=False`` when ``regex=False``;
- :py:meth:`DataArray.str.replace` now respects ``case=False`` when ``regex=False``;

and others as well

Comment thread xarray/tests/test_accessor_str.py Outdated
id="StringDType",
marks=pytest.mark.skipif(
not hasattr(np, "dtypes") or not hasattr(np.dtypes, "StringDType"),
reason="requires numpy>=2",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our minimum numpy version is already 2.2, the skip is not required anymore.

Comment thread xarray/tests/test_accessor_str.py Outdated
assert_equal(result, xr.DataArray([False, False, True], dims=["x"]))

result = da.str.replace("o", "_", case=False, regex=False)
assert list(result.values) == ["F__", "xYz", "f__.__"]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this test also use assert_equal?

Use :py:meth: roles in the whats-new entry, remove the numpy>=2 skipif
(minimum numpy is 2.2), and compare the replace result with assert_equal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Novain Shibu <georgeshibu380@gmail.com>
@novain-shibu

Copy link
Copy Markdown
Contributor Author

Thanks for the review! I’ve made all three changes in 9b28876:

  • Used :py:meth: roles for both methods in whats-new
  • Removed the numpy>=2 skip
  • Switched the last check to assert_equal and made sure the expected dtype matches the input

@headtr1ck headtr1ck added the plan to merge Final call for comments label Oct 19, 2026
@headtr1ck
headtr1ck merged commit 6fa6881 into pydata:main Oct 20, 2026
59 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

plan to merge Final call for comments

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants