Skip to content

Commit 2463ce0

Browse files
shps951023shps951023
committed
Add borrowed reader-to-writer XLSX insert APIs
1 parent 1da11a1 commit 2463ce0

9 files changed

Lines changed: 520 additions & 22 deletions

File tree

‎README.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -112,6 +112,11 @@ Typed and structured visitors plus sheet names, information, dimensions, and col
112112

113113
Dynamic, explicit-schema, typed, and multi-sheet workbooks can be written to a borrowed `Write + Send` sink with the `*_to_writer` APIs. The library does not close readers or writers. Reader position is unspecified after a call. Writer output begins at the current position and does not truncate existing content, so callers should provide an empty or already-truncated sink.
114114

115+
Existing workbooks can be appended to or replaced through separate borrowed streams with the
116+
`insert*_from_reader_to_writer` APIs. These require a `Read + Seek` source and an empty
117+
`Write + Seek` destination. Both remain open, but output is not atomic and is not rolled back after
118+
a destination error. Source and destination must not alias the same underlying stream.
119+
115120
> **Memory boundary:** the streaming path keeps workbook metadata, styles, a small row channel, and parser buffers in memory. Shared-string tables at least 5 MiB spill to indexed temporary files by default; dropping the iterator removes them. Configure this with `with_shared_string_disk_cache()`, `with_shared_string_cache_size()`, and `with_shared_string_cache_path()`. The directory must already exist. Byte/WASM queries always keep shared strings in memory. Worksheet XML and prior rows are never retained. Peak memory can still grow with a single exceptionally large row, but not with the full worksheet row count.
116121
117122
## Structured Streaming Query
@@ -291,6 +296,15 @@ assert_eq!(count, 1);
291296

292297
`insert_with_schema()` accepts a fallible, one-pass dynamic iterator. Source rows are disk-spooled and the constant-memory backend retains only the current row while generating the donor workbook; style rebasing currently materializes the generated worksheet XML. `insert_serialized()` accepts Serde structs. Existing unrelated ZIP entries, worksheet identities, formulas, and cached values are preserved, and an existing workbook is replaced only after the rewritten package validates and syncs.
293298

299+
For separate borrowed streams, use `insert_from_reader_to_writer()`,
300+
`insert_with_schema_from_reader_to_writer()`, or
301+
`insert_serialized_from_reader_to_writer()`. The source must implement `Read + Seek`, the
302+
destination must implement `Write + Seek`, and both remain open. The destination must be empty:
303+
MiniExcel neither truncates it nor rolls it back after an error, so a destination failure can
304+
leave a partial XLSX package. The two handles must not alias the same underlying stream. These
305+
stream APIs preserve the same package behavior as path Insert but do not provide its atomic commit
306+
or post-write validation guarantee.
307+
294308
The default `ExistingSheetPolicy::Reject` rejects duplicate worksheet names case-insensitively. Use `ExistingSheetPolicy::Replace` to replace a worksheet in place while preserving its workbook order, ID, relationship/path, visibility, and active state. The default `TargetRelationshipPolicy::Reject` accepts only a plain target with no worksheet relationships. `RemoveSupported` can remove target-owned tables, drawings with exclusively owned images, comments, VML drawings, and external hyperlinks; pivots, external links, unknown relationships, and shared/global parts are rejected or preserved conservatively. Insert writes XLSX packages, rejects macro-enabled `.xlsm` paths, and rejects `WriteOptions::with_overwrite_file(true)` because workbook replacement is controlled by the insert policy.
295309

296310
Appending a formula-free worksheet preserves an existing calculation chain and workbook calculation properties. Replacement removes the complete stale `calcChain` part, relationship, and content-type override, then sets `fullCalcOnLoad` and `forceFullCalc` so Excel recalculates on the next open. MiniExcel does not evaluate or rewrite formulas; formulas and cached values in untouched worksheets remain byte-identical.

‎README.zh-CN.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -112,6 +112,11 @@ MiniExcel::visit_rows_from_reader(&mut input, &options, |excel_row, row| {
112112

113113
动态、显式 schema、类型化和多工作表 workbook 可通过 `*_to_writer` API 写入借用的 `Write + Send` sink。库不会关闭 reader 或 writer。调用结束后的 reader position 不保证；writer 从当前位置开始写入且不截断既有内容，因此调用方应提供空或已截断的 sink。
114114

115+
现有 workbook 可通过独立 borrowed stream 的 `insert*_from_reader_to_writer` API 执行
116+
append 或 replacement。这些 API 要求 `Read + Seek` source 与空的 `Write + Seek`
117+
destination。调用后两者保持 open，但 output 不具备 atomicity，destination error 后也不会
118+
rollback。Source 与 destination 不得指向同一个底层 stream。
119+
115120
> **内存边界：** 流式路径会在内存中保留工作簿元数据、样式、少量行 channel 和 parser buffer。默认情况下，至少 5 MiB 的 shared-string table 会 spill 到带索引的临时文件；丢弃 iterator 后自动删除。可通过 `with_shared_string_disk_cache()`、`with_shared_string_cache_size()` 和 `with_shared_string_cache_path()` 配置，目录必须预先存在。Byte/WASM query 始终将 shared string 保留在内存中。Worksheet XML 和先前 row 永远不会保留；峰值内存仍可能随单个超大 row 增长，但不会随 worksheet 总行数增长。
116121
117122
## 保留结构的流式 Query
@@ -291,6 +296,14 @@ assert_eq!(count, 1);
291296

292297
`insert_with_schema()` 接受可返回错误、只消费一次的动态 iterator。源 row 会先落盘 spool，constant-memory backend 在生成 donor workbook 时只保留当前 row；style rebase 当前仍会物化生成的 worksheet XML。`insert_serialized()` 接受 Serde struct。现有无关 ZIP entry、worksheet identity、formula 和 cached value 均会保留；只有重写 package 完成验证并同步后，才原子替换现有 workbook。
293298

299+
对于两个独立的 borrowed stream，可使用 `insert_from_reader_to_writer()`、
300+
`insert_with_schema_from_reader_to_writer()` 或
301+
`insert_serialized_from_reader_to_writer()`。Source 必须实现 `Read + Seek`，destination
302+
必须实现 `Write + Seek`，调用后两者都保持 open。Destination 必须为空：MiniExcel 不会
303+
truncate，也不会在错误后 rollback，因此 destination 写入失败可能留下部分 XLSX package。
304+
两个 handle 不得指向同一个底层 stream。这些 stream API 保持与 path Insert 相同的 package
305+
行为，但不提供 path API 的 atomic commit 或写后验证保证。
306+
294307
默认的 `ExistingSheetPolicy::Reject` 会不区分大小写地拒绝重复 worksheet name。使用 `ExistingSheetPolicy::Replace` 可原位替换 worksheet，并保留其 workbook 顺序、ID、relationship/path、visibility 与 active state。默认 `TargetRelationshipPolicy::Reject` 只接受没有 worksheet relationship 的 plain target。`RemoveSupported` 可删除 target-owned table、drawing 及其独占 image、comment、VML drawing 和 external hyperlink；pivot、external link、未知 relationship 与 shared/global part 会被拒绝或保守保留。Insert 写入 XLSX package，拒绝 macro-enabled `.xlsm` path，并拒绝 `WriteOptions::with_overwrite_file(true)`，因为 workbook replacement 由 Insert policy 控制。
295308

296309
追加 formula-free worksheet 时会保留已有 calculation chain 与 workbook calculation property。Replacement 会完整删除 stale `calcChain` part、relationship 和 content-type override，并设置 `fullCalcOnLoad` 与 `forceFullCalc`，让 Excel 下次打开时重算。MiniExcel 不执行或改写公式；未修改 worksheet 中的 formula 与 cached value 保持原始字节。

‎docs/compatibility.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ The latest `calamine 0.36` and `rust_xlsxwriter 0.97` require Rust 1.88. The MVP
4747
| Dynamic export | `save_as()` / `save_as_with_schema()` | Map serialization is implemented internally |
4848
| Typed export | `save_as_serialized<T>()` | Uses Serde mapping internally |
4949
| Multi-sheet export | `save_as_sheets()` / `save_as_serialized_sheets()` | Preserves input sheet order and returns data-row counts |
50-
| `InsertSheet` append/replace | `insert()` / `insert_with_schema()` / `insert_serialized()` | Atomic append; strict in-place replacement preserves sheet identity and has an explicit relationship policy |
50+
| `InsertSheet` append/replace | `insert()` / `insert_with_schema()` / `insert_serialized()` / borrowed reader-to-writer variants | Path APIs are atomic; separate borrowed streams require an empty sink and preserve package behavior without atomic commit |
5151
| Per-sheet visibility | `WriteOptions::with_sheet_visibility()` | Visible, hidden, and very hidden; first visible sheet is active |
5252
| `overwriteFile` | `WriteOptions::with_overwrite_file()` | Defaults to `false`; existing paths require explicit opt-in |
5353
| `FreezeRowCount` / `FreezeColumnCount` | `WriteOptions::with_freeze_row_count()` / `with_freeze_column_count()` | Defaults to one frozen row and zero frozen columns |
@@ -114,7 +114,7 @@ Path RAG exports retain parser state, repeated header context, and one output ch
114114

115115
The backend makes two sequential, bounded-memory passes over the selected worksheet entry. The first records the used extent and compact merged-cell rectangles. This is required for MiniExcel-compatible stable dynamic schemas when legal files omit `<dimension>`, to preserve style-only row elements like the .NET reader, and to support opt-in merged-cell filling without expanding ranges into an address map. The second pass emits rows and retains only anchor values for currently active merged ranges. Worksheet XML and prior rows are never retained; memory consists primarily of in-memory or disk-indexed shared strings, styles, merge metadata, parser buffers, the current row, and the bounded channel.
116116

117-
The internal writer assembles a new ZIP package with one or more worksheets. Path saves refuse existing files by default and can explicitly replace them. Insert APIs append a worksheet through a validated package rewrite and atomic sibling-file replacement; unchanged ZIP entries and existing worksheet identities are preserved. Fallible explicit-schema producers are consumed once through a disk spool and a constant-memory worksheet writer, while the generated donor worksheet XML is materialized for style rebasing. Template fills rewrite worksheet XML within a copied package; worksheet styles and unrelated ZIP parts are retained. Array expansion shifts row and cell addresses and updates the worksheet dimension. Formula expressions are preserved but not recalculated, and version 1 does not adjust formula references, merged ranges, tables, drawings, or defined names after inserted rows.
117+
The internal writer assembles a new ZIP package with one or more worksheets. Path saves refuse existing files by default and can explicitly replace them. Path Insert APIs append or replace a worksheet through a validated package rewrite and atomic sibling-file replacement; unchanged ZIP entries and existing worksheet identities are preserved. Separate borrowed Insert APIs accept `Read + Seek` input and an empty `Write + Seek` output, leave both open, and preserve the same package behavior without atomic commit, rollback, or post-write validation. Fallible explicit-schema producers are consumed once through a disk spool and a constant-memory worksheet writer, while the generated donor worksheet XML is materialized for style rebasing. Template fills rewrite worksheet XML within a copied package; worksheet styles and unrelated ZIP parts are retained. Array expansion shifts row and cell addresses and updates the worksheet dimension. Formula expressions are preserved but not recalculated, and version 1 does not adjust formula references, merged ranges, tables, drawings, or defined names after inserted rows.
118118

119119
## Test Sources
120120

‎docs/compatibility.zh-CN.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ Rust MVP 在统一的 `MiniExcel` facade 后实现最小但实用的 MiniExcel
4747
| 动态导出 | `save_as()` / `save_as_with_schema()` | map 序列化在内部实现 |
4848
| 类型化导出 | `save_as_serialized<T>()` | 内部使用 Serde 映射 |
4949
| 多工作表导出 | `save_as_sheets()` / `save_as_serialized_sheets()` | 保留输入工作表顺序并返回数据行数 |
50-
| `InsertSheet` append/replace | `insert()` / `insert_with_schema()` / `insert_serialized()` | 原子 append；严格原位 replacement 保留 sheet identity，并使用显式 relationship policy |
50+
| `InsertSheet` append/replace | `insert()` / `insert_with_schema()` / `insert_serialized()` / borrowed reader-to-writer variants | Path API 为原子操作；独立 borrowed stream 要求空 sink，并在无原子 commit 的情况下保持相同 package 行为 |
5151
| 每表 visibility | `WriteOptions::with_sheet_visibility()` | visible、hidden、very hidden；第一个 visible sheet 为 active |
5252
| `overwriteFile` | `WriteOptions::with_overwrite_file()` | 默认 `false`；已有路径需要显式允许覆盖 |
5353
| `FreezeRowCount` / `FreezeColumnCount` | `WriteOptions::with_freeze_row_count()` / `with_freeze_column_count()` | 默认冻结一行、零列 |
@@ -114,7 +114,7 @@ Rust MVP 在统一的 `MiniExcel` facade 后实现最小但实用的 MiniExcel
114114

115115
backend 对所选 worksheet entry 执行两次顺序、有界内存扫描。第一次记录使用范围和紧凑 merged-cell 矩形。这是为了在合法文件省略 `<dimension>` 时保持 MiniExcel 兼容的稳定动态 schema、像 .NET reader 一样保留仅含 style 的 row element，并在不展开地址 map 的情况下支持按需 merged-cell 填充。第二次扫描输出 row，只保留当前活动 merge range 的锚点值。Worksheet XML 和先前 row 永远不会保留；内存主要由内存或磁盘索引的 shared string、style、merge metadata、parser buffer、当前 row 和有界 channel 构成。
116116

117-
内部 writer 组装包含一个或多个工作表的新 ZIP package。路径保存默认拒绝已有文件，也可显式替换。Insert API 通过验证后的 package rewrite 与同目录临时文件原子替换来追加 worksheet；未修改的 ZIP entry 和现有 worksheet identity 会保留。可返回错误的显式 schema producer 只消费一次，经磁盘 spool 与 constant-memory worksheet writer 处理；生成的 donor worksheet XML 会为 style rebase 而物化。模板填充会在复制的 package 中重写 worksheet XML；worksheet 样式和无关 ZIP part 会保留。数组展开会移动 row/cell 地址并更新 worksheet dimension。公式表达式会保留但不会重算；版本 1 不会在插行后调整公式引用、merge range、table、drawing 或 defined name。
117+
内部 writer 组装包含一个或多个工作表的新 ZIP package。路径保存默认拒绝已有文件，也可显式替换。Path Insert API 通过验证后的 package rewrite 与同目录临时文件原子替换来追加或替换 worksheet；未修改的 ZIP entry 和现有 worksheet identity 会保留。独立 borrowed Insert API 接受 `Read + Seek` input 与空的 `Write + Seek` output，调用后两者保持 open，并在不提供 atomic commit、rollback 或写后验证的情况下保持相同 package 行为。可返回错误的显式 schema producer 只消费一次，经磁盘 spool 与 constant-memory worksheet writer 处理；生成的 donor worksheet XML 会为 style rebase 而物化。模板填充会在复制的 package 中重写 worksheet XML；worksheet 样式和无关 ZIP part 会保留。数组展开会移动 row/cell 地址并更新 worksheet dimension。公式表达式会保留但不会重算；版本 1 不会在插行后调整公式引用、merge range、table、drawing 或 defined name。
118118

119119
## 测试来源
120120

‎docs/insert-existing-workbook-plan.md‎

Lines changed: 33 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -102,7 +102,17 @@ impl MiniExcel {
102102
where
103103
T: Serialize;
104104

105-
pub fn insert_from_reader_to_writer<R, W, I>(
105+
pub fn insert_from_reader_to_writer<R, W>(
106+
source: &mut R,
107+
destination: &mut W,
108+
rows: &[DynamicRow],
109+
options: &InsertOptions,
110+
) -> Result<usize>
111+
where
112+
R: Read + Seek,
113+
W: Write + Seek;
114+
115+
pub fn insert_with_schema_from_reader_to_writer<R, W, I>(
106116
source: &mut R,
107117
destination: &mut W,
108118
schema: &[String],
@@ -113,6 +123,17 @@ impl MiniExcel {
113123
R: Read + Seek,
114124
W: Write + Seek,
115125
I: IntoIterator<Item = Result<DynamicRow>>;
126+
127+
pub fn insert_serialized_from_reader_to_writer<T, R, W>(
128+
source: &mut R,
129+
destination: &mut W,
130+
rows: &[T],
131+
options: &InsertOptions,
132+
) -> Result<usize>
133+
where
134+
T: Serialize,
135+
R: Read + Seek,
136+
W: Write + Seek;
116137
}
117138
```
118139

@@ -338,18 +359,24 @@ the .NET Open XML SDK 3.5.1 before and after the roundtrip.
338359

339360
Depends on Task 9.
340361

341-
- [ ] Implement `insert_from_reader_to_writer` for separate borrowed input/output objects.
342-
- [ ] Require `Read + Seek` input and `Write + Seek` output; leave both open.
343-
- [ ] Do not truncate the destination; document that callers must provide an empty sink.
344-
- [ ] Propagate source, row-iterator, and destination errors without consuming additional rows.
345-
- [ ] Keep same-stream mutation unsupported because it cannot provide the same atomicity contract.
362+
- [x] Implement `insert_from_reader_to_writer` for separate borrowed input/output objects.
363+
- [x] Require `Read + Seek` input and `Write + Seek` output; leave both open.
364+
- [x] Do not truncate the destination; document that callers must provide an empty sink.
365+
- [x] Propagate source, row-iterator, and destination errors without consuming additional rows.
366+
- [x] Keep same-stream mutation unsupported because it cannot provide the same atomicity contract.
346367

347368
Acceptance:
348369

349370
- Input remains unchanged and both objects remain usable after success and failure.
350371
- Output matches the path API package inventory.
351372
- Focused command: `cargo +1.85.0 test -p miniexcel --test insert borrowed_io --locked`.
352373

374+
Completed on 2026-08-25. Dynamic, explicit-schema iterator, and Serde reader-to-writer APIs
375+
share package preflight and append/replace behavior. Tests compare append and replacement output
376+
with path API inventories, prove source/policy failures consume no rows, stop immediately on
377+
producer errors, preserve non-empty sinks, and propagate destination failures while leaving both
378+
borrowed objects usable.
379+
353380
### Task 11: Security, Resource, And Stress Hardening
354381

355382
Depends on Task 10.

0 commit comments

Comments
 (0)