Skip to content

stat: preserve non-UTF-8 filenames in error messages - #14755

Open
ddy314 wants to merge 2 commits into
uutils:mainfrom
ddy314:fix/stat-invalid-utf8-diagnostics
Open

ddy314 wants to merge 2 commits into
uutils:mainfrom
ddy314:fix/stat-invalid-utf8-diagnostics

Conversation

@ddy314

@ddy314 ddy314 commented Sep 20, 2026

Copy link
Copy Markdown

Preserve filenames in stat errors without lossy UTF-8 conversion, using the existing locale-aware shell escaping. The diagnostics now use GNU-compatible octal byte escapes. Fixes #14685.

Also fix the shared shell quoter when a filename contains both a single quote and invalid bytes: dollar-quoted escapes require single-quote segments. Add regression coverage and explicitly gate the byte-based stat test to Unix.

Validation: 42 stat tests, 14 quoting tests, and 350 related utility integration tests passed (2 ignored); clippy and rustfmt passed. Thirty stderr/exit-code comparisons against GNU stat 9.11 matched across C/C.UTF-8, default/-L/-f and five filename cases. The full selected-feature test command also passes with the host NO_COLOR setting removed for the color tests.

AI assistance: Codex helped implement, test, and respond to review under the account owner's direction.

Copilot AI lite review requested due to automatic review settings September 20, 2026 14:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

Review effort: Lite
Findings: 1 High severity · 1 Medium severity

Open (2)
What changed in this PR

This PR updates stat error reporting to preserve non-UTF-8 filenames by quoting the original OS string rather than a lossy UTF-8 conversion (fixing #14685).

Changes:

  • Use OsStr::quote() output in stat error paths to avoid U+FFFD replacing invalid bytes.
  • Add an integration test covering missing non-UTF-8 filenames across default, -L, and -f modes.
File Description
tests/​by-util/​test_stat.rs Adds a regression test ensuring diagnostics keep non-UTF-8 bytes when a path is missing.
src/​uu/​stat/​src/​stat.rs Switches relevant error messages to use quoted original OsStr instead of lossy UTF-8 strings.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tests/by-util/test_stat.rs
Comment thread tests/by-util/test_stat.rs Outdated
@github-actions

Copy link
Copy Markdown

GNU testsuite comparison:

Congrats! The gnu test tests/csplit/csplit is no longer failing!
Congrats! The gnu test tests/du/max-depth is no longer failing!
Congrats! The gnu test tests/fold/fold is no longer failing!
Congrats! The gnu test tests/misc/just-created is no longer failing!
Congrats! The gnu test tests/ptx/word-regex-loop is no longer failing!

Comment thread tests/by-util/test_stat.rs Outdated
.args(&args)
.arg(name)
.fails_with_code(1)
.stderr_contains("$'missing-\\xFF'");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the expected output is not correct. With GNU stat I get the following output:

$ stat $'missing-\xFF'
stat: cannot statx 'missing-'$'\377': No such file or directory

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right: preserving the byte with Quotable still gave a different format. b36f315 switches these errors to the existing locale-aware shell quoter and expects octal escapes. I also fixed its single-quote/invalid-byte edge case. Thirty full stderr and exit-code comparisons against GNU stat 9.11 match across C/C.UTF-8, default/-L/-f and five filename cases. The 42 stat tests, 14 quoting tests and 350 related utility tests pass (2 ignored).

Copilot AI review requested due to automatic review settings September 21, 2026 12:29
always_quote: bool,
dirname: bool,
size_hint: usize,
encoding: crate::i18n::UEncoding,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please import UEncoding instead of the full path 3 times

encoding: crate::i18n::UEncoding,
) -> Self {
let (mut quotes, must_quote) = initial_quoting(reference, dirname, always_quote, true);
let invalid = match encoding {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this rescans every name (ls calls this per entry). could you only do it when initial_quoting picked Quotes::Double?

}

#[test]
fn test_invalid_bytes_with_single_quote() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could use check_names_raw_both like the other tests here, no?

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Address the redundant validation scan and add the required WASI test ignore.

Review effort: Lite
Findings: None

Resolved since last review (2)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(stat): invalid unicode file not escaped, they replaced with U+FFFD

4 participants