Skip to content

Commit 4078f8f

Browse files
shps951023shps951023
committed
Add atomic explicit-schema async XLSX exports with .NET parity
1 parent 693dfbf commit 4078f8f

13 files changed

Lines changed: 687 additions & 10 deletions

File tree

‎.github/workflows/rust.yml‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -290,6 +290,37 @@ jobs:
290290
--filter "FullyQualifiedName~MiniExcelOpenXmlExporterTests.CopyAndInsertSheetTest|FullyQualifiedName~MiniExcelOpenXmlExporterAsyncTests.CopyAndInsertSheetTest"
291291
--verbosity minimal
292292
293+
async-export-parity:
294+
strategy:
295+
matrix:
296+
os: [ubuntu-latest, windows-latest]
297+
runs-on: ${{ matrix.os }}
298+
steps:
299+
- uses: actions/checkout@v4
300+
- name: Checkout MiniExcel .NET
301+
uses: actions/checkout@v4
302+
with:
303+
repository: mini-software/MiniExcel
304+
ref: b9a76d7af62142e0e38545b6905b01a06e8d160e
305+
path: dotnet-source
306+
- name: Install Rust 1.85.0
307+
uses: dtolnay/rust-toolchain@1.85.0
308+
- uses: Swatinem/rust-cache@v2
309+
with:
310+
key: async-export-parity-${{ matrix.os }}
311+
- name: Setup .NET
312+
uses: actions/setup-dotnet@v4
313+
with:
314+
dotnet-version: 10.0.x
315+
- name: Run Rust async export tests
316+
run: cargo +1.85.0 test -p miniexcel --test export_async --features async --locked
317+
- name: Run MiniExcel .NET async export baseline tests
318+
run: >-
319+
dotnet test dotnet-source/tests/MiniExcel.OpenXml.Tests/MiniExcel.OpenXml.Tests.csproj
320+
--framework net10.0
321+
--filter "FullyQualifiedName~MiniExcelOpenXmlExporterAsyncTests.SaveAsByAsyncEnumerable|FullyQualifiedName~MiniExcelOpenXmlExporterAsyncTests.SaveEmptyAsyncEnumerableTest"
322+
--verbosity minimal
323+
293324
msrv:
294325
runs-on: ubuntu-latest
295326
steps:

‎README.md‎

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -125,6 +125,24 @@ Explicit cancellation yields an error recognized by `Error::is_cancelled()`; dro
125125
requests cancellation without blocking the executor. Parser initialization or the current row may
126126
finish before background cleanup completes.
127127

128+
Explicit-schema async row streams can also create new workbooks atomically:
129+
130+
```rust
131+
let count = MiniExcel::save_as_with_schema_async_with_cancellation(
132+
"book.xlsx",
133+
&["Name".to_owned(), "Version".to_owned()],
134+
rows,
135+
&WriteOptions::new().with_sheet_name("Async"),
136+
cancellation,
137+
).await?;
138+
```
139+
140+
The schema is required so empty streams and one-pass producers remain deterministic. Rows cross a
141+
bounded channel and are disk-spooled before the blocking constant-memory writer runs. Producer
142+
errors, cancellation, dropped futures, validation failures, and destination races leave an existing
143+
target byte-identical or a missing target absent. `with_overwrite_file(true)` enables atomic
144+
replacement. The returned count excludes the header.
145+
128146
## Borrowed Readers And Writers
129147

130148
Use visitor APIs for caller-owned `Read + Seek` sources without materializing all rows or transferring ownership:
@@ -573,7 +591,7 @@ Version 1 does not implement `@group`, `@if`, parametrized sheet cloning, `$=` f
573591
- `MiniExcel::query()` and `query_as()` strictly stream worksheet XML from paths.
574592
- Grouped analytics retain state proportional to distinct groups and stop at `max_groups`.
575593
- RAG exports never recalculate formulas and reject hidden sheets unless explicitly allowed.
576-
- Synchronous streaming queries use one worker thread per active query. Optional async query and Insert APIs use bounded channels around blocking XLSX workers; ZIP/XML/filesystem work is not async I/O.
594+
- Synchronous streaming queries use one worker thread per active query. Optional async query, explicit-schema export, and Insert APIs use bounded channels around blocking XLSX workers; ZIP/XML/filesystem work is not async I/O.
577595
- Save creates new workbooks and refuses existing target paths by default. `MiniExcel::insert*()` atomically appends or strictly replaces a worksheet in an existing `.xlsx` path, or creates a workbook when the path is missing. `copy_and_add_sheet*()` creates a validated source-derived destination. `rename_sheet()`, `set_sheet_visibility()`, and `reorder_sheet()` atomically change existing workbook sheet metadata.
578596

579597
## Not Supported

‎README.zh-CN.md‎

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -123,6 +123,23 @@ async executor。它们不会把 filesystem access 变成 async I/O，也不依
123123
返回可由 `Error::is_cancelled()` 识别的错误；丢弃 stream 会请求取消，但不会阻塞 executor。
124124
Parser 初始化或当前 row 可能会在后台清理完成前继续执行。
125125

126+
显式 schema async row stream 也可原子创建新 workbook：
127+
128+
```rust
129+
let count = MiniExcel::save_as_with_schema_async_with_cancellation(
130+
"book.xlsx",
131+
&["Name".to_owned(), "Version".to_owned()],
132+
rows,
133+
&WriteOptions::new().with_sheet_name("Async"),
134+
cancellation,
135+
).await?;
136+
```
137+
138+
显式 schema 使空 stream 与 one-pass producer 的行为保持确定。Row 通过 bounded channel，
139+
并在 blocking constant-memory writer 运行前落盘 spool。Producer error、cancellation、
140+
drop future、validation failure 与 destination race 都会使已有 target 字节不变，或使缺失
141+
target 继续不存在。`with_overwrite_file(true)` 启用原子 replacement。返回 count 不含 header。
142+
126143
## 借用 Reader 与 Writer
127144

128145
对调用方持有的 `Read + Seek` source，可使用 visitor API 流式处理，不会物化全部 row 或转移所有权：
@@ -564,7 +581,7 @@ MiniExcel::save_as_template(
564581
- `MiniExcel::query()` 和 `query_as()` 会从路径严格流式解析 worksheet XML。
565582
- 分组分析保留与不同 group 数量成比例的状态，并在 `max_groups` 停止。
566583
- RAG 导出不会重新计算公式，hidden sheet 未显式允许时会拒绝处理。
567-
- 同步流式 query 每个活动 query 使用一个 worker thread。可选 async query/Insert API 通过 bounded channel 包装 blocking XLSX worker；ZIP/XML/filesystem 工作并不是 async I/O。
584+
- 同步流式 query 每个活动 query 使用一个 worker thread。可选 async query、显式 schema export 与 Insert API 通过 bounded channel 包装 blocking XLSX worker；ZIP/XML/filesystem 工作并不是 async I/O。
568585
- Save 会创建新工作簿，并默认拒绝已有目标路径。`MiniExcel::insert*()` 会向现有 `.xlsx` path 原子 append 或严格 replace worksheet；路径不存在时则创建 workbook。`copy_and_add_sheet*()` 创建经过验证、源自 source 的 destination。`rename_sheet()`、`set_sheet_visibility()` 和 `reorder_sheet()` 会原子修改现有 workbook 的 sheet metadata。
569586

570587
## 暂不支持

‎docs/compatibility.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,7 @@ The latest `calamine 0.36` and `rust_xlsxwriter 0.97` require Rust 1.88. The MVP
5555
| `InsertSheet` append/replace | `insert()` / `insert_with_schema()` / `insert_serialized()` / borrowed reader-to-writer variants | Path APIs are atomic; separate borrowed streams require an empty sink and preserve package behavior without atomic commit |
5656
| Async Insert producer | `insert_with_schema_async*()` | Optional `async` feature; bounded producer channel with blocking XLSX work on a dedicated thread |
5757
| Async path query | `query_async*()` / `query_as_async*()` | Optional `async` feature; bounded dynamic/Serde streams, cooperative cancellation, blocking XLSX workers |
58+
| Async explicit-schema export | `save_as_with_schema_async*()` | Optional `async` feature; bounded row producer, atomic destination, cooperative cancellation |
5859
| Per-sheet visibility | `WriteOptions::with_sheet_visibility()` | Visible, hidden, and very hidden; first visible sheet is active |
5960
| `overwriteFile` | `WriteOptions::with_overwrite_file()` | Defaults to `false`; existing paths require explicit opt-in |
6061
| `FreezeRowCount` / `FreezeColumnCount` | `WriteOptions::with_freeze_row_count()` / `with_freeze_column_count()` | Defaults to one frozen row and zero frozen columns |
@@ -184,6 +185,7 @@ The contract covers only the current common surface: dynamic/typed path queries,
184185
| Addressed JSONL/Markdown/manifest RAG export | Rust research extension | No |
185186
| Async Insert producer | Implemented behind optional feature | Rust cancellation tests; not shared internals |
186187
| Async dynamic/typed path query | Implemented behind optional feature | Rust parity, cancellation, error, and cleanup tests |
188+
| Async explicit-schema path export | Implemented behind optional feature | Rust success, rollback, cancellation, and cleanup tests; pinned .NET async-enumerable baseline |
187189
| DataReader and broader stream ownership | Deferred | No |
188190
| Append worksheet to existing `.xlsx` workbook | Implemented and atomically committed | Rust tests; shared parity contract not yet extended |
189191
| Strict worksheet replacement | Implemented for plain targets and supported target-owned closures | Rust tests; stale calcChain removed and full recalculation requested |
@@ -198,4 +200,4 @@ This matrix is the coverage claim: Rust does not yet provide complete API parity
198200

199201
## Deferred Work
200202

201-
SQL text parsing, `HAVING`, `ORDER BY`, joins, windows, pivots, disk-spill aggregation, vector indexing, model calls, old Excel formats, advanced template directives and sheet cloning, image authoring, merged-cell APIs, formula calculation/dependency expansion, formula authoring, general styling, async export/template I/O, async borrowed readers, and borrowed XLSX lazy readers require separate design and acceptance milestones. CSV DataReader/DataTable adapters are intentionally replaced by Rust iterators, and a one-call CSV/XLSX converter is not exposed; callers compose query and save APIs. See the [Insert migration guide](insert-v1-migration.md) for supported workflows and deliberate differences.
203+
SQL text parsing, `HAVING`, `ORDER BY`, joins, windows, pivots, disk-spill aggregation, vector indexing, model calls, old Excel formats, advanced template directives and sheet cloning, image authoring, merged-cell APIs, formula calculation/dependency expansion, formula authoring, general styling, async template I/O, inferred/typed async export sources, async borrowed readers/writers, and borrowed XLSX lazy readers require separate design and acceptance milestones. CSV DataReader/DataTable adapters are intentionally replaced by Rust iterators, and a one-call CSV/XLSX converter is not exposed; callers compose query and save APIs. See the [Insert migration guide](insert-v1-migration.md) for supported workflows and deliberate differences.

‎docs/compatibility.zh-CN.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,7 @@ Rust MVP 在统一的 `MiniExcel` facade 后实现最小但实用的 MiniExcel
5555
| `InsertSheet` append/replace | `insert()` / `insert_with_schema()` / `insert_serialized()` / borrowed reader-to-writer variants | Path API 为原子操作；独立 borrowed stream 要求空 sink，并在无原子 commit 的情况下保持相同 package 行为 |
5656
| Async Insert producer | `insert_with_schema_async*()` | 可选 `async` feature；bounded producer channel，XLSX 工作在专用 blocking thread |
5757
| Async path query | `query_async*()` / `query_as_async*()` | 可选 `async` feature；bounded 动态/Serde stream、协作式 cancellation、blocking XLSX worker |
58+
| Async 显式 schema export | `save_as_with_schema_async*()` | 可选 `async` feature；bounded row producer、原子 destination、协作式 cancellation |
5859
| 每表 visibility | `WriteOptions::with_sheet_visibility()` | visible、hidden、very hidden；第一个 visible sheet 为 active |
5960
| `overwriteFile` | `WriteOptions::with_overwrite_file()` | 默认 `false`；已有路径需要显式允许覆盖 |
6061
| `FreezeRowCount` / `FreezeColumnCount` | `WriteOptions::with_freeze_row_count()` / `with_freeze_column_count()` | 默认冻结一行、零列 |
@@ -184,6 +185,7 @@ Rust workflow 会在 Linux 和 Windows 上运行 Rust 契约。其 .NET parity j
184185
| 带地址 JSONL/Markdown/manifest RAG 导出 | Rust 研究扩展 | 否 |
185186
| Async Insert producer | 已通过可选 feature 实现 | Rust cancellation test；不共享内部行为 |
186187
| Async 动态/类型化 path query | 已通过可选 feature 实现 | Rust parity、cancellation、error 与 cleanup 测试 |
188+
| Async 显式 schema path export | 已通过可选 feature 实现 | Rust success、rollback、cancellation 与 cleanup 测试；固定 .NET async-enumerable 基准 |
187189
| DataReader 与更广泛的 stream ownership | 延后 | 否 |
188190
| 向现有 `.xlsx` workbook 追加 worksheet | 已实现并原子提交 | Rust 测试；共享 parity contract 尚未扩展 |
189191
| 严格 worksheet replacement | 已支持 plain target 与受支持的 target-owned closure | Rust 测试；删除 stale calcChain 并要求 full recalculation |
@@ -198,4 +200,4 @@ Rust workflow 会在 Linux 和 Windows 上运行 Rust 契约。其 .NET parity j
198200

199201
## 延后工作
200202

201-
SQL 文本解析、`HAVING`、`ORDER BY`、join、window、pivot、磁盘 spill 聚合、向量索引、模型调用、旧 Excel 格式、高级 template 指令与 sheet 克隆、image authoring、merged-cell API、公式计算/依赖展开、公式编写、通用 style、async export/template I/O、async borrowed reader，以及 borrowed XLSX lazy reader，都需要独立的设计与验收里程碑。CSV DataReader/DataTable adapter 已有意替换为 Rust iterator；当前不提供一步式 CSV/XLSX converter，调用方可组合 query 与 save API。支持工作流与有意保留的差异见 [Insert 迁移说明](insert-v1-migration.zh-CN.md)。
203+
SQL 文本解析、`HAVING`、`ORDER BY`、join、window、pivot、磁盘 spill 聚合、向量索引、模型调用、旧 Excel 格式、高级 template 指令与 sheet 克隆、image authoring、merged-cell API、公式计算/依赖展开、公式编写、通用 style、async template I/O、推断/类型化 async export source、async borrowed reader/writer，以及 borrowed XLSX lazy reader，都需要独立的设计与验收里程碑。CSV DataReader/DataTable adapter 已有意替换为 Rust iterator；当前不提供一步式 CSV/XLSX converter，调用方可组合 query 与 save API。支持工作流与有意保留的差异见 [Insert 迁移说明](insert-v1-migration.zh-CN.md)。

‎docs/dotnet-feature-gaps.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ This report compares observable public capabilities in the local checkouts below
88

99
| Project | Revision |
1010
| --- | --- |
11-
| MiniExcel-Rust | working tree based on `db95441` (`0.3.0`) |
11+
| MiniExcel-Rust | working tree based on `693dfbf` (`0.3.0`) |
1212
| MiniExcel .NET | `b9a76d7af62142e0e38545b6905b01a06e8d160e` |
1313

1414
The comparison uses the .NET public APIs, their controlling implementations, and focused tests under the sibling `../MiniExcel` checkout. Rust status is based on the public `MiniExcel` facade, options, integration tests, and [compatibility boundary](compatibility.md).
@@ -31,7 +31,7 @@ Rust already implements dynamic and Serde-typed XLSX path queries, inclusive A1
3131
| Named tables | Implemented | Dynamic/typed path queries, byte queries, and borrowed-reader visitors use table metadata headers and bounds with case-insensitive table-name matching. |
3232
| DataReader and DataTable | Different by design | Rust exposes iterators and borrowed visitors instead of .NET tabular interfaces. A Rust-native Arrow/record-batch adapter is deferred until a concrete integration requires it and does not block parity completion. |
3333
| Caller-owned streams | Partial | Borrowed synchronous dynamic/typed/structured visitors, metadata reads, dynamic/schema/typed/multi-sheet writers, and separate reader-to-writer Insert are implemented with leave-open semantics. Borrowed lazy iterators, borrowed async streams, and template streams remain unsupported. |
34-
| Async and cancellation | Partial | Optional runtime-neutral dynamic/Serde path queries and explicit-schema Insert support bounded streams and cooperative cancellation. Async export/template operations, async write sources, borrowed async readers, and progress callbacks remain unsupported. ZIP and filesystem work remains blocking on dedicated workers. |
34+
| Async and cancellation | Partial | Optional runtime-neutral dynamic/Serde path queries, explicit-schema path export, and explicit-schema Insert support bounded streams and cooperative cancellation. Async template operations, inferred/typed async write sources, borrowed async I/O, and progress callbacks remain unsupported. ZIP and filesystem work remains blocking on dedicated workers. |
3535
| General save inputs | Partial | Export from general objects/enumerables, dictionaries, `DataTable`, `IDataReader`, and async enumerables, with progress. Rust accepts dynamic or same-type Serde slices and reports per-sheet row counts. |
3636
| Multi-sheet export | Partial | Rust creates ordered visible, hidden, and very-hidden worksheets, but does not yet accept heterogeneous Serde row types in one call. |
3737
| Existing-workbook operations | Implemented | Rust atomically appends, strictly replaces, renames, changes visibility, reorders, and performs .NET-style source-workbook copy-and-add while preserving unrelated package parts and worksheet identity. Rename preserves formula text; visibility rejects hiding the final visible sheet; reorder remaps active/view/local-name indices; copy-and-add preserves the source and atomically publishes a separate destination. |
@@ -52,6 +52,7 @@ Rust already implements dynamic and Serde-typed XLSX path queries, inclusive A1
5252
| --- | --- | --- |
5353
| Public read/write boundary | `miniexcel/src/facade.rs`, `miniexcel/src/options.rs` | `src/MiniExcel.OpenXml/Api/OpenXmlImporter.cs`, `OpenXmlExporter.cs` |
5454
| Async query | `MiniExcel::query_async*` and `query_as_async*`; Rust focused parity/cancellation/error/cleanup tests | `OpenXmlImporter.QueryAsync`; `MiniExcelOpenXmlImporterAsyncTests` |
55+
| Async export | `MiniExcel::save_as_with_schema_async*`; Rust focused rollback/cancellation/cleanup tests | `OpenXmlExporter.ExportAsync`; async-enumerable exporter tests |
5556
| Tables | `MiniExcel::query_table*`; Rust focused tests use the exact `TestQueryTable.xlsx` fixture (SHA-256 `04F719BF9F9E99D9B437A8FB32F8111FD92580A1D29ACAD10B6ED128C0564501`) | `OpenXmlImporter.QueryTableAsync`; `tests/MiniExcel.OpenXml.Tests/Tables/` |
5657
| Comments | `MiniExcel::get_comments*`; Rust focused tests use `TestCommentsAndNotes.xlsx` (SHA-256 `3A855CE896ED62DC27C91797432DD89EE081F07CD03AB05BF1B0CD745543A3FC`) | `OpenXmlImporter.RetrieveCommentsAsync`; `tests/MiniExcel.OpenXml.Tests/Comments/` |
5758
| DataReader/DataTable | Rust iterators and borrowed visitors are the native abstraction; no literal .NET tabular adapter is planned | `OpenXmlImporter.GetDataReader`, `GetAsyncDataReader`, `QueryAsDataTableAsync`; `tests/MiniExcel.OpenXml.Tests/DataReader/` |
@@ -66,7 +67,7 @@ The .NET APIs marked with `Async` also have generated synchronous counterparts t
6667

6768
## Suggested Implementation Order
6869

69-
1. **Async export/template APIs**: extend runtime-neutral producer/cancellation patterns without presenting blocking ZIP work as async I/O.
70+
1. **Async template and broader export APIs**: extend runtime-neutral cancellation to templates and typed/inferred producers without presenting blocking ZIP work as async I/O.
7071
2. **Advanced templates and Fluent Mapping**: add grouped/conditional templates, parametrized sheets, and mapping through separate compatibility milestones.
7172
3. **Selected-sheet cloning**: not part of .NET `CopyAndAddSheet`; add only for a concrete Rust use case with a relationship-closure cloning contract.
7273

0 commit comments

Comments
 (0)