Skip to content

pr: preserve non-UTF-8 input filenames - #15488

Open
limadog9 wants to merge 1 commit into
uutils:mainfrom
limadog9:fix/pr-non-utf8-filenames-12997
Open

limadog9 wants to merge 1 commit into
uutils:mainfrom
limadog9:fix/pr-non-utf8-filenames-12997

Conversation

@limadog9

Copy link
Copy Markdown
Contributor

Keep filename bytes through argument preprocessing, metadata lookup, and input reads. Non-UTF-8 operands are no longer silently discarded in favor of standard input.

Add regression tests for existing and missing filenames and merged input. All 87 pr integration tests pass.

Closes #12997.

@limadog9
limadog9 force-pushed the fix/pr-non-utf8-filenames-12997 branch from 14a29e3 to 9a11cbb Compare October 18, 2026 19:46
Comment thread src/uu/pr/src/pr.rs Outdated
.find_position(|x| x.to_str().is_some_and(|x| e_regex.is_match(x.trim())));
if let Some((pos, value)) = expand_tabs_option {
arguments[pos] = format!("{}\t8", value.trim());
arguments[pos] = format!("{}\t8", value.to_str().unwrap().trim()).into();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we avoid this unwrap? e.g. keep the &str from find_position

Comment thread src/uu/pr/src/pr.rs Outdated
|s| s.as_str(),
)
.to_string();
paths[0].to_string_lossy().into_owned()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the header will show U+FFFD here, GNU prints the raw bytes
all the new tests use -t, so could you please add one without it?

Comment thread tests/by-util/test_pr.rs Outdated
.arg(name)
.arg("right")
.succeeds()
.stdout_contains("left")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stdout_contains is weak here, could you please check the merged line?

@sylvestre

Copy link
Copy Markdown
Contributor

please also check the header of a non-UTF-8 file, see the inline comments. thanks

@limadog9
limadog9 force-pushed the fix/pr-non-utf8-filenames-12997 branch 3 times, most recently from 2d736aa to 056a65c Compare October 19, 2026 11:36
@github-actions

Copy link
Copy Markdown

GNU testsuite comparison:

Skipping an intermittent issue tests/tail/symlink (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/tail/follow-symlink is no longer failing!

Comment thread tests/by-util/test_pr.rs Outdated
}

#[test]
#[cfg(target_os = "linux")]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why only linux here when the header test runs on every unix except apple? could the gates match?

@limadog9
limadog9 force-pushed the fix/pr-non-utf8-filenames-12997 branch from 056a65c to a108039 Compare October 24, 2026 12:04
Keep filename bytes through argument preprocessing, metadata lookup, and input reads. Non-UTF-8 operands are no longer silently discarded in favor of standard input.

Add regression tests for existing and missing filenames and merged input. All 87 pr integration tests pass.

Closes uutils#12997.
@sylvestre
sylvestre force-pushed the fix/pr-non-utf8-filenames-12997 branch from a108039 to 09e05a5 Compare October 25, 2026 09:41

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(pr): when the file argument is $'\xFF' it does not error out

2 participants