Skip to content

csv: support non-ASCII Unicode dialect characters #8408

Description

@coderabbitai

Summary

Add support for single non-ASCII Unicode code points in CSV dialect character attributes, matching CPython behavior.

Rationale

csv dialect options such as delimiter, quotechar, and escapechar accept exactly one Unicode character in CPython. RustPython currently stores these attributes as u8 in crates/stdlib/src/csv.rs, so valid one-character values outside the byte range (for example, ) are rejected.

Affected area

  • crates/stdlib/src/csv.rs
  • CSV dialect parsing, validation, reader, and writer configuration

Required changes

Replace the byte-only representation and parsing path for dialect characters with a representation that preserves a full Unicode code point, then propagate that support through the CSV reader and writer implementation. Keep existing validation that rejects empty and multi-character values, while reporting conversion/representation failures accurately.

Acceptance criteria

  • delimiter, quotechar, and escapechar accept a single non-ASCII Unicode code point such as .
  • Empty and multi-character strings remain rejected according to CPython-compatible behavior.
  • Reader and writer behavior correctly handles the supported Unicode dialect characters.
  • Relevant RustPython CSV tests cover successful non-ASCII dialect-character usage and invalid lengths.

Backlinks

Requested by @hyoinandout.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions