You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The script resolves and archives `v1.x-maintenance` from the local .NET repository without changing its checkout. It compares .NET v1 `MiniExcel.Query(..., useHeaderRow: false)` with Rust `MiniExcel::query` against the same 100,000-row, 10-column XLSX workbook. Release builds are warmed up, then run in alternating order for five independent iterations with three full query passes per process. The harness verifies matching row counts and reports median elapsed time, throughput, and sampled peak working set. Process startup and .NET JIT time are included; save performance is not.
29
+
The script resolves and archives `v1.x-maintenance` from the local .NET repository without changing its checkout. It compares .NET v1 `MiniExcel.Query(..., useHeaderRow: false)` with Rust `MiniExcel::query` against the same 100,000-row, 10-column XLSX workbook. It reports separate cold first-call and warmed steady-state results, runs five fresh processes in alternating order, and verifies matching row and cell counts. Query timing excludes process launch; peak working set covers the complete process. Save performance is not included.
30
30
31
-
The machine-readable report is written to `target/benchmarks/dotnet-v1-vs-rust.json`. Use `-Passes`, `-Iterations`, `-Workbook`, `-DotNetRevision`, or `-OutputJson` to change the workload. Results vary by environment, so compare only values produced on the same machine.
31
+
The machine-readable report is written to `target/benchmarks/dotnet-v1-vs-rust.json`. Use `-Scenario`, `-Passes`, `-WarmupPasses`, `-Iterations`, `-Workbook`, `-DotNetRevision`, or `-OutputJson` to change the workload. Results vary by environment, so compare only values produced on the same machine.
32
32
33
33
See [the benchmark methodology and recorded result](docs/dotnet-v1-query-benchmark.md).
Copy file name to clipboardExpand all lines: docs/dotnet-v1-query-benchmark.md
+62-17Lines changed: 62 additions & 17 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,7 +11,21 @@ This benchmark compares dynamic, headerless XLSX streaming over the same workboo
11
11
12
12
Both runners enumerate every returned row without retaining the complete worksheet. Save performance, typed mapping, formulas, and other APIs are outside this comparison.
13
13
14
-
The harness starts a fresh process for each measured iteration. Each process reads the workbook three times, so process startup and .NET JIT are included but amortized across 300,000 rows. A separate warm-up process runs first to populate operating-system file caches. The five measured iterations alternate runtime order. Peak working set is sampled approximately every 10 ms.
14
+
## Fairness Controls
15
+
16
+
- Both runners use Release builds, the same workbook, and equivalent public dynamic Query APIs.
17
+
- Both runners count rows and cells. A result is rejected unless both counts match in every iteration.
18
+
- An untimed preflight process for each runtime populates operating-system file caches.
19
+
- Query time is measured inside each runner, excluding process launch and result serialization.
20
+
- Five measured iterations use fresh processes and alternate runtime order. The median is reported.
21
+
- No custom PGO, native CPU target, CPU affinity, or runtime tuning is applied.
22
+
23
+
Two scenarios keep startup and sustained throughput separate:
24
+
25
+
-**Cold:** one measured Query with no in-process warm-up. It includes first-call library initialization and .NET JIT, but not process launch. This is not a cold-disk test because the preflight has warmed the operating-system cache.
26
+
-**Steady:** one complete Query warm-up inside each process, followed by three measured Queries. The warm-up is outside Query timing. The .NET runner performs a full garbage collection after warm-up and before timing so warm-up garbage is not charged to the measured queries. This measures sustained in-process throughput after .NET JIT and runtime caches are warm.
27
+
28
+
Peak working set is sampled approximately every 10 ms over the whole process. It therefore includes runtime startup and, for the steady scenario, the warm-up pass. Total process elapsed time is retained as a secondary metric but is not used to calculate Query throughput.
15
29
16
30
## Environment
17
31
@@ -23,33 +37,62 @@ The following result was captured on 2026-08-26:
For the first Query in a fresh process, Rust delivered 1.30x the .NET v1 throughput, completed in 23.0% less Query time, and used 85.6% less peak working set.
60
+
61
+
After an in-process warm-up, Rust delivered 0.61x the .NET v1 throughput and took 62.8% more Query time. Its median peak working set remained 87.4% lower, at 9.91 MB versus 78.75 MB.
62
+
63
+
The result is therefore workload-dependent: Rust has lower first-call latency and substantially lower memory use here, while .NET v1 has higher sustained throughput after JIT warm-up.
| 1 |3,513.19 ms |81.49 MB |3,948.10 ms | 9.79 MB |
45
-
| 2 |3,294.73 ms |79.91 MB |4,139.42 ms | 9.84 MB |
46
-
| 3 |3,249.22 ms |79.73 MB |3,939.59 ms | 9.90 MB |
47
-
| 4 |3,369.14 ms |79.99 MB |3,962.12 ms | 9.91 MB |
48
-
| 5 |3,513.73 ms |79.85 MB |3,955.76 ms | 9.86 MB |
69
+
| 1 |2,063.54 ms |65.97 MB |1,571.51 ms | 9.66 MB |
70
+
| 2 |1,950.17 ms |67.22 MB |1,397.59 ms | 9.75 MB |
71
+
| 3 |1,890.76 ms |68.12 MB |1,461.67 ms | 9.77 MB |
72
+
| 4 |1,816.06 ms |67.63 MB |1,394.83 ms | 9.71 MB |
73
+
| 5 |1,793.67 ms |66.70 MB |1,455.79 ms | 9.67 MB |
49
74
50
-
For this workload, Rust took 17.4% longer and delivered 14.8% lower throughput than .NET v1. Its median peak working set was 87.7% lower: 9.86 MB versus 79.91 MB, or about one eighth of the .NET v1 footprint.
75
+
## Steady Results
51
76
52
-
These numbers describe one workbook and one machine, not a general performance guarantee. In particular, the process-level timing includes runtime startup, and the sampled working set includes runtime overhead. Use the harness on representative workbooks before drawing application-specific conclusions.
77
+
Each timed value below covers three complete Query passes after one untimed warm-up pass.
| 1 | 2,401.10 ms | 79.22 MB | 4,209.62 ms | 9.92 MB |
82
+
| 2 | 2,602.98 ms | 78.44 MB | 4,128.13 ms | 9.91 MB |
83
+
| 3 | 2,554.63 ms | 82.07 MB | 4,357.14 ms | 9.89 MB |
84
+
| 4 | 2,766.50 ms | 78.75 MB | 4,205.49 ms | 9.94 MB |
85
+
| 5 | 2,585.18 ms | 78.27 MB | 4,254.68 ms | 9.80 MB |
86
+
87
+
## Interpretation
88
+
89
+
The benchmark workbook contains 1,000,000 shared-string cells, 100,000 unique strings, and no merged ranges. Its worksheet XML expands to 39,167,847 bytes.
90
+
91
+
Rust currently performs a complete worksheet scan before the emitting pass, even though this workbook has a valid `<dimension>` and merged-cell filling is disabled. It also clones each shared-string value into an intermediate `Data::String` and then again into the public `CellValue::String`, clones dynamic column names for every row, and transfers each parsed row through a bounded synchronous channel. These choices preserve bounded memory and iterator ownership but add steady-state work.
92
+
93
+
.NET v1 can stop its preliminary extent check as soon as it reads `<dimension ref="A1:J100000">`. Its JIT cost makes the first Query slower, but later Query passes benefit from already compiled and dynamically optimized hot paths. This explains why Rust leads in the cold scenario while .NET v1 leads in the steady scenario.
94
+
95
+
These numbers describe one workbook and one machine, not a general performance guarantee. The harness does not pin CPU affinity or suppress all operating-system noise; the median limits the effect of outliers. Use representative workbooks before drawing application-specific conclusions.
53
96
54
97
## Reproduce
55
98
@@ -59,4 +102,6 @@ Keep the Rust and .NET repositories in sibling directories, then run from the Ru
The script reads `v1.x-maintenance` from the local .NET Git repository without changing its checkout. It builds both runners in Release mode, verifies matching row counts, prints the comparison, and writes the full machine-readable report to `target/benchmarks/dotnet-v1-vs-rust.json`.
105
+
The script reads `v1.x-maintenance` from the local .NET Git repository without changing its checkout. It builds both runners in Release mode, verifies matching row and cell counts, prints both scenarios, and writes the full machine-readable report to `target/benchmarks/dotnet-v1-vs-rust.json`.
106
+
107
+
Use `-Scenario Cold` or `-Scenario Steady` to run one scenario. `-Passes` and `-WarmupPasses` configure the steady scenario; `-Iterations` controls the number of fresh measured processes.
0 commit comments