Skip to content

Commit 64e1001

Browse files
shps951023shps951023
committed
Add inferred-schema async Serde exports with .NET parity
1 parent 436f1bf commit 64e1001

10 files changed

Lines changed: 375 additions & 38 deletions

File tree

‎README.md‎

Lines changed: 17 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -137,11 +137,23 @@ let count = MiniExcel::save_as_with_schema_async_with_cancellation(
137137
).await?;
138138
```
139139

140-
The schema is required so empty streams and one-pass producers remain deterministic. Rows cross a
141-
bounded channel and are disk-spooled before the blocking constant-memory writer runs. Producer
142-
errors, cancellation, dropped futures, validation failures, and destination races leave an existing
143-
target byte-identical or a missing target absent. `with_overwrite_file(true)` enables atomic
144-
replacement. The returned count excludes the header.
140+
Serde streams can instead infer their schema from the first row:
141+
142+
```rust
143+
let count = MiniExcel::save_as_serialized_async(
144+
"book.xlsx",
145+
rows,
146+
&WriteOptions::new().with_sheet_name("Async"),
147+
).await?;
148+
```
149+
150+
Dynamic streams require an explicit schema so empty streams remain deterministic. Serde streams
151+
infer final field names and order after destination preflight; an empty Serde stream requires
152+
`with_print_header(false)`. `save_as_serialized_async_with_cancellation()` adds explicit
153+
cancellation. Rows cross a bounded channel and are disk-spooled before the blocking
154+
constant-memory writer runs. Producer errors, cancellation, dropped futures, validation failures,
155+
and destination races leave an existing target byte-identical or a missing target absent.
156+
`with_overwrite_file(true)` enables atomic replacement. The returned count excludes the header.
145157

146158
## Borrowed Readers And Writers
147159

‎README.zh-CN.md‎

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -135,10 +135,23 @@ let count = MiniExcel::save_as_with_schema_async_with_cancellation(
135135
).await?;
136136
```
137137

138-
显式 schema 使空 stream 与 one-pass producer 的行为保持确定。Row 通过 bounded channel，
139-
并在 blocking constant-memory writer 运行前落盘 spool。Producer error、cancellation、
140-
drop future、validation failure 与 destination race 都会使已有 target 字节不变，或使缺失
141-
target 继续不存在。`with_overwrite_file(true)` 启用原子 replacement。返回 count 不含 header。
138+
Serde stream 可改为从第一行推断 schema：
139+
140+
```rust
141+
let count = MiniExcel::save_as_serialized_async(
142+
"book.xlsx",
143+
rows,
144+
&WriteOptions::new().with_sheet_name("Async"),
145+
).await?;
146+
```
147+
148+
Dynamic stream 需要显式 schema，以保持空 stream 的确定性。Serde stream 会在 destination
149+
preflight 后从第一行推断最终 field name 与顺序；空 Serde stream 需要
150+
`with_print_header(false)`。`save_as_serialized_async_with_cancellation()` 提供显式
151+
cancellation。Row 通过 bounded channel，并在 blocking constant-memory writer 运行前落盘
152+
spool。Producer error、cancellation、drop future、validation failure 与 destination race
153+
都会使已有 target 字节不变，或使缺失 target 继续不存在。`with_overwrite_file(true)` 启用
154+
原子 replacement。返回 count 不含 header。
142155

143156
## 借用 Reader 与 Writer
144157

‎docs/compatibility.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ The latest `calamine 0.36` and `rust_xlsxwriter 0.97` require Rust 1.88. The MVP
5656
| `InsertSheet` append/replace | `insert()` / `insert_with_schema()` / `insert_serialized()` / borrowed reader-to-writer variants | Path APIs are atomic; separate borrowed streams require an empty sink and preserve package behavior without atomic commit |
5757
| Async Insert producer | `insert_with_schema_async*()` | Optional `async` feature; bounded producer channel with blocking XLSX work on a dedicated thread |
5858
| Async path query | `query_async*()` / `query_as_async*()` | Optional `async` feature; bounded dynamic/Serde streams, cooperative cancellation, blocking XLSX workers |
59-
| Async explicit-schema export | `save_as_with_schema_async*()` | Optional `async` feature; bounded row producer, atomic destination, cooperative cancellation |
59+
| Async dynamic/Serde export | `save_as_with_schema_async*()` / `save_as_serialized_async*()` | Optional `async` feature; explicit or first-row-inferred schema, bounded producer, atomic destination, cooperative cancellation |
6060
| Async basic template path output | `save_as_template_async*()` | Optional `async` feature; scalar/list renderer, atomic destination, cooperative cancellation |
6161
| Per-sheet visibility | `WriteOptions::with_sheet_visibility()` | Visible, hidden, and very hidden; first visible sheet is active |
6262
| `overwriteFile` | `WriteOptions::with_overwrite_file()` | Defaults to `false`; existing paths require explicit opt-in |
@@ -191,7 +191,7 @@ The contract covers only the current common surface: dynamic/typed path queries,
191191
| Addressed JSONL/Markdown/manifest RAG export | Rust research extension | No |
192192
| Async Insert producer | Implemented behind optional feature | Rust cancellation tests; not shared internals |
193193
| Async dynamic/typed path query | Implemented behind optional feature | Rust parity, cancellation, error, and cleanup tests |
194-
| Async explicit-schema path export | Implemented behind optional feature | Rust success, rollback, cancellation, and cleanup tests; pinned .NET async-enumerable baseline |
194+
| Async dynamic/Serde path export | Implemented behind optional feature | Rust explicit/inferred schema, rollback, cancellation, and cleanup tests; pinned .NET async-enumerable baseline |
195195
| Async basic template path output | Implemented behind optional feature | Rust success, rollback, cancellation, and cleanup tests; scoped pinned .NET async template baseline |
196196
| DataReader and broader stream ownership | Deferred | No |
197197
| Append worksheet to existing `.xlsx` workbook | Implemented and atomically committed | Rust tests; shared parity contract not yet extended |

‎docs/compatibility.zh-CN.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ Rust MVP 在统一的 `MiniExcel` facade 后实现最小但实用的 MiniExcel
5656
| `InsertSheet` append/replace | `insert()` / `insert_with_schema()` / `insert_serialized()` / borrowed reader-to-writer variants | Path API 为原子操作；独立 borrowed stream 要求空 sink，并在无原子 commit 的情况下保持相同 package 行为 |
5757
| Async Insert producer | `insert_with_schema_async*()` | 可选 `async` feature；bounded producer channel，XLSX 工作在专用 blocking thread |
5858
| Async path query | `query_async*()` / `query_as_async*()` | 可选 `async` feature；bounded 动态/Serde stream、协作式 cancellation、blocking XLSX worker |
59-
| Async 显式 schema export | `save_as_with_schema_async*()` | 可选 `async` feature；bounded row producer、原子 destination、协作式 cancellation |
59+
| Async dynamic/Serde export | `save_as_with_schema_async*()` / `save_as_serialized_async*()` | 可选 `async` feature；显式或首行推断 schema、bounded producer、原子 destination、协作式 cancellation |
6060
| Async 基础 template path output | `save_as_template_async*()` | 可选 `async` feature；scalar/list renderer、原子 destination、协作式 cancellation |
6161
| 每表 visibility | `WriteOptions::with_sheet_visibility()` | visible、hidden、very hidden；第一个 visible sheet 为 active |
6262
| `overwriteFile` | `WriteOptions::with_overwrite_file()` | 默认 `false`；已有路径需要显式允许覆盖 |
@@ -191,7 +191,7 @@ Rust workflow 会在 Linux 和 Windows 上运行 Rust 契约。其 .NET parity j
191191
| 带地址 JSONL/Markdown/manifest RAG 导出 | Rust 研究扩展 | 否 |
192192
| Async Insert producer | 已通过可选 feature 实现 | Rust cancellation test；不共享内部行为 |
193193
| Async 动态/类型化 path query | 已通过可选 feature 实现 | Rust parity、cancellation、error 与 cleanup 测试 |
194-
| Async 显式 schema path export | 已通过可选 feature 实现 | Rust success、rollback、cancellation 与 cleanup 测试；固定 .NET async-enumerable 基准 |
194+
| Async dynamic/Serde path export | 已通过可选 feature 实现 | Rust 显式/推断 schema、rollback、cancellation 与 cleanup 测试；固定 .NET async-enumerable 基准 |
195195
| Async 基础 template path output | 已通过可选 feature 实现 | Rust success、rollback、cancellation 与 cleanup 测试；scoped 固定 .NET async template 基准 |
196196
| DataReader 与更广泛的 stream ownership | 延后 | 否 |
197197
| 向现有 `.xlsx` workbook 追加 worksheet | 已实现并原子提交 | Rust 测试；共享 parity contract 尚未扩展 |

‎docs/dotnet-feature-gaps.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ This report compares observable public capabilities in the local checkouts below
88

99
| Project | Revision |
1010
| --- | --- |
11-
| MiniExcel-Rust | working tree based on `14ad5ec` (`0.3.0`) |
11+
| MiniExcel-Rust | working tree based on `436f1bf` (`0.3.0`) |
1212
| MiniExcel .NET | `b9a76d7af62142e0e38545b6905b01a06e8d160e` |
1313

1414
The comparison uses the .NET public APIs, their controlling implementations, and focused tests under the sibling `../MiniExcel` checkout. Rust status is based on the public `MiniExcel` facade, options, integration tests, and [compatibility boundary](compatibility.md).
@@ -31,8 +31,8 @@ Rust already implements dynamic and Serde-typed XLSX path queries, inclusive A1
3131
| Named tables | Implemented | Dynamic/typed path queries, byte queries, and borrowed-reader visitors use table metadata headers and bounds with case-insensitive table-name matching. |
3232
| DataReader and DataTable | Different by design | Rust exposes iterators and borrowed visitors instead of .NET tabular interfaces. A Rust-native Arrow/record-batch adapter is deferred until a concrete integration requires it and does not block parity completion. |
3333
| Caller-owned streams | Partial | Borrowed synchronous dynamic/typed/structured visitors, metadata reads, dynamic/schema/typed/multi-sheet writers, and separate reader-to-writer Insert are implemented with leave-open semantics. Borrowed lazy iterators, borrowed async streams, and template streams remain unsupported. |
34-
| Async and cancellation | Partial | Optional runtime-neutral dynamic/Serde path queries, explicit-schema path export, basic template path output, and explicit-schema Insert support cooperative cancellation and atomic publication where applicable. Advanced template streams, inferred/typed async write sources, borrowed async I/O, and progress callbacks remain unsupported. ZIP and filesystem work remains blocking on dedicated workers. |
35-
| General save inputs | Partial | Export from general objects/enumerables, dictionaries, `DataTable`, `IDataReader`, and async enumerables, with progress. Rust accepts dynamic or same-type Serde slices and reports per-sheet row counts. |
34+
| Async and cancellation | Partial | Optional runtime-neutral dynamic/Serde path queries, explicit-schema dynamic and inferred-schema Serde path export, basic template path output, and explicit-schema Insert support cooperative cancellation and atomic publication where applicable. Advanced template streams, borrowed async I/O, and progress callbacks remain unsupported. ZIP and filesystem work remains blocking on dedicated workers. |
35+
| General save inputs | Partial | Export from general objects/enumerables, dictionaries, `DataTable`, `IDataReader`, and async enumerables, with progress. Rust accepts dynamic or same-type Serde slices, bounded dynamic/Serde async streams, and reports per-sheet row counts. |
3636
| Multi-sheet export | Partial | Rust creates ordered visible, hidden, and very-hidden worksheets, but does not yet accept heterogeneous Serde row types in one call. |
3737
| Existing-workbook operations | Implemented | Rust atomically appends, strictly replaces, renames, changes visibility, reorders, and performs .NET-style source-workbook copy-and-add while preserving unrelated package parts and worksheet identity. Rename preserves formula text; visibility rejects hiding the final visible sheet; reorder remaps active/view/local-name indices; copy-and-add preserves the source and atomically publishes a separate destination. |
3838
| Templates | Partial | Rust fills path/byte templates with scalars, arrays, conditional blocks, validated multirow groups, and `$=` formulas; path output also has a cancellable async wrapper. Streams, nested/logical conditions, grouped/conditional formulas, parametrized sheets, formula-reference translation, and formula calculation remain unsupported. Stale calcChain metadata is removed and full recalculation requested. |
@@ -52,7 +52,7 @@ Rust already implements dynamic and Serde-typed XLSX path queries, inclusive A1
5252
| --- | --- | --- |
5353
| Public read/write boundary | `miniexcel/src/facade.rs`, `miniexcel/src/options.rs` | `src/MiniExcel.OpenXml/Api/OpenXmlImporter.cs`, `OpenXmlExporter.cs` |
5454
| Async query | `MiniExcel::query_async*` and `query_as_async*`; Rust focused parity/cancellation/error/cleanup tests | `OpenXmlImporter.QueryAsync`; `MiniExcelOpenXmlImporterAsyncTests` |
55-
| Async export | `MiniExcel::save_as_with_schema_async*`; Rust focused rollback/cancellation/cleanup tests | `OpenXmlExporter.ExportAsync`; async-enumerable exporter tests |
55+
| Async export | `MiniExcel::save_as_with_schema_async*` and `save_as_serialized_async*`; Rust explicit/inferred schema, rollback/cancellation/cleanup tests | `OpenXmlExporter.ExportAsync`; `SaveAsByAsyncEnumerable` and empty async-enumerable tests |
5656
| Async template | `MiniExcel::save_as_template_async*`; Rust focused rollback/cancellation/cleanup tests | `OpenXmlTemplater.SaveAsByTemplateAsync`; scoped basic/cancellation tests |
5757
| Template conditions | Enumerable-cell `@if`/`@elseif`/`@else` blocks; Rust sync/async branch/error/style tests | `TestIEnumerableConditional` |
5858
| Template groups | `@group`/`@header`/`@endgroup` multirow blocks; Rust sync/async order/error/style tests | `GroupTemplateTest`; `TestIEnumerableGrouped` |
@@ -71,7 +71,7 @@ The .NET APIs marked with `Async` also have generated synchronous counterparts t
7171

7272
## Suggested Implementation Order
7373

74-
1. **Broader async export/stream APIs**: extend runtime-neutral cancellation to typed/inferred producers and borrowed streams without presenting blocking ZIP work as async I/O.
74+
1. **Borrowed async streams and progress**: add caller-owned async reader/writer integration and progress only where ownership, cancellation, and blocking-I/O semantics remain explicit.
7575
2. **Advanced templates and collection mapping**: add formula/merge-aware groups, richer conditional expressions, parametrized sheets, and deterministic collection layouts through separate compatibility milestones.
7676
3. **Selected-sheet cloning**: not part of .NET `CopyAndAddSheet`; add only for a concrete Rust use case with a relationship-closure cloning contract.
7777

‎docs/dotnet-feature-gaps.zh-CN.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88

99
| 项目 | 版本基线 |
1010
| --- | --- |
11-
| MiniExcel-Rust | 基于 `14ad5ec` 的工作区（`0.3.0`） |
11+
| MiniExcel-Rust | 基于 `436f1bf` 的工作区（`0.3.0`） |
1212
| MiniExcel .NET | `b9a76d7af62142e0e38545b6905b01a06e8d160e` |
1313

1414
比对依据包括同级目录 `../MiniExcel` 中的 .NET 公开 API、控制实现和聚焦测试，以及 Rust 的公开 `MiniExcel` 门面、选项、集成测试和[兼容性边界](compatibility.zh-CN.md)。
@@ -31,8 +31,8 @@ Rust 已支持动态及 Serde 强类型 XLSX 路径查询、闭区间 A1 范围
3131
| 命名表格 | 已实现 | 动态/类型化 path query、byte query 与 borrowed-reader visitor 使用 table metadata header 和 bounds，并按 table name 大小写不敏感匹配。 |
3232
| DataReader 与 DataTable | 设计不同 | Rust 使用 iterator 与 borrowed visitor，不复制 .NET tabular interface。只有具体集成需要时才考虑 Rust-native Arrow/record-batch adapter；此项不阻塞 parity 完成。 |
3333
| 调用方提供的流 | 部分实现 | 已实现借用的同步动态/类型化/structured visitor、metadata 读取、动态/schema/类型化/多表 writer，以及独立 reader-to-writer Insert，并保持 leave-open。借用 lazy iterator、borrowed async stream 和 template stream 仍不支持。 |
34-
| 异步与取消 | 部分实现 | 可选、runtime-neutral 的动态/Serde path query、显式 schema path export、基础 template path output 与显式 schema Insert 已支持协作式 cancellation，并在适用位置提供原子发布。高级 template stream、推断/类型化 async write source、borrowed async I/O 与 progress callback 仍不支持；ZIP 和 filesystem 工作仍在专用 blocking worker 上执行。 |
35-
| 通用保存输入 | 部分实现 | 从普通对象/可枚举对象、字典、`DataTable`、`IDataReader` 和异步枚举导出，并报告进度。Rust 接受动态行或同类型 Serde 切片，并返回每张工作表的行数。 |
34+
| 异步与取消 | 部分实现 | 可选、runtime-neutral 的动态/Serde path query、显式 schema dynamic 与推断 schema Serde path export、基础 template path output 与显式 schema Insert 已支持协作式 cancellation，并在适用位置提供原子发布。高级 template stream、borrowed async I/O 与 progress callback 仍不支持；ZIP 和 filesystem 工作仍在专用 blocking worker 上执行。 |
35+
| 通用保存输入 | 部分实现 | 从普通对象/可枚举对象、字典、`DataTable`、`IDataReader` 和异步枚举导出，并报告进度。Rust 接受动态行或同类型 Serde 切片、bounded dynamic/Serde async stream，并返回每张工作表的行数。 |
3636
| 多工作表导出 | 部分实现 | Rust 可按输入顺序创建 visible、hidden 和 very-hidden 工作表，但尚不能在一次调用中接受异构 Serde 行类型。 |
3737
| 修改现有工作簿 | 已实现 | Rust 可原子 append、严格 replace、rename、修改 visibility、reorder，并执行 .NET 风格的 source-workbook copy-and-add，同时保留无关 package part 与 worksheet identity。Rename 保留 formula 文本；visibility 拒绝隐藏最后一张 visible sheet；reorder remap active/view/local-name index；copy-and-add 保留 source 并原子发布独立 destination。 |
3838
| 模板 | 部分实现 | Rust 可使用 scalar、array、conditional block、validated multirow group 与 `$=` formula 填充 path/byte template；path output 也提供 cancellable async wrapper。Stream、nested/logical condition、grouped/conditional formula、参数化 sheet、formula-reference translation 与公式计算仍不支持。Stale calcChain metadata 会被删除并要求 full recalculation。 |
@@ -52,7 +52,7 @@ Rust 已支持动态及 Serde 强类型 XLSX 路径查询、闭区间 A1 范围
5252
| --- | --- | --- |
5353
| 公开读写边界 | `miniexcel/src/facade.rs`、`miniexcel/src/options.rs` | `src/MiniExcel.OpenXml/Api/OpenXmlImporter.cs`、`OpenXmlExporter.cs` |
5454
| Async query | `MiniExcel::query_async*` 与 `query_as_async*`；Rust focused parity/cancellation/error/cleanup 测试 | `OpenXmlImporter.QueryAsync`；`MiniExcelOpenXmlImporterAsyncTests` |
55-
| Async export | `MiniExcel::save_as_with_schema_async*`；Rust focused rollback/cancellation/cleanup 测试 | `OpenXmlExporter.ExportAsync`；async-enumerable exporter 测试 |
55+
| Async export | `MiniExcel::save_as_with_schema_async*` 与 `save_as_serialized_async*`；Rust 显式/推断 schema、rollback/cancellation/cleanup 测试 | `OpenXmlExporter.ExportAsync`；`SaveAsByAsyncEnumerable` 与 empty async-enumerable 测试 |
5656
| Async template | `MiniExcel::save_as_template_async*`；Rust focused rollback/cancellation/cleanup 测试 | `OpenXmlTemplater.SaveAsByTemplateAsync`；scoped basic/cancellation 测试 |
5757
| Template condition | Enumerable-cell `@if`/`@elseif`/`@else` block；Rust sync/async branch/error/style 测试 | `TestIEnumerableConditional` |
5858
| Template group | `@group`/`@header`/`@endgroup` multirow block；Rust sync/async order/error/style 测试 | `GroupTemplateTest`；`TestIEnumerableGrouped` |
@@ -71,7 +71,7 @@ Rust 已支持动态及 Serde 强类型 XLSX 路径查询、闭区间 A1 范围
7171

7272
## 建议实现顺序
7373

74-
1. **更广泛 async export/stream API**：把 runtime-neutral cancellation 扩展到类型化/推断 producer 和 borrowed stream，不把 blocking ZIP 工作描述成 async I/O。
74+
1. **Borrowed async stream 与 progress**：仅在 ownership、cancellation 与 blocking-I/O 语义清晰时增加 caller-owned async reader/writer integration 和 progress。
7575
2. **高级模板与 collection mapping**：通过独立兼容里程碑增加 formula/merge-aware group、更丰富 conditional expression、参数化 sheet 和确定性 collection layout。
7676
3. **选定 worksheet cloning**：不属于 .NET `CopyAndAddSheet`；仅在有具体 Rust 使用场景时，通过 relationship closure cloning contract 增加。
7777

0 commit comments

Comments
 (0)