Block-structured AMR with batched fine advance, load-balance and active-box infrastructure - #1628

…r stage over the plan's flat transfer list (amr_device_pack)

…se-grid halo is 39% of growth, batched advance saves 0.60 s/step, our geo-mean equals the bar rather than beating it

…the batched advance worth 0.38; the uniform denominator re-measured at 240 steps

…gather is 0.224 s/step of non-wait work -- 25% wait, not 12%, after review folded pgather into its own phase bracket

… GB of output byte-identical, the cost removed is per-map not per-launch, and three measurement failures including a retraction of ledger 74's invented bracket-inflation caveat

…receives per box-peer run) and flag the exactly-tiled pool assumption at the fused copyout

…ff amr_device_pack flag (-0.14 s/step, 8.1x fewer device maps, 12 GB of field output byte-identical)

…chemistry-AMR tests it does not cover

… np8 spread was one outlier, settled by the 2-node follow-on; k004-004 and k004-005 are sick

…overs 7 percent of what the fused packs recover, so the cost is the map machinery not the malloc, and ledger 75's hedge is corrected

…istry-AMR tests needed the bench fftw on LD_LIBRARY_PATH, not a code fix

… session -- it explains the drift, the unusable step column and probably the np16 hang, and the wait-free gather column was immune

…was too weak to see the degradation it exists to catch

… export list

The single-block seed is the batched loop's one-member case (own
intersection origin, zero slab offset, the installed bounds), so the
duplicated kernel goes. s_amr_loc_index_init and s_amr_st_finalize are
module-private. Goldens unchanged (61 AMR tests, 19-test multi-rank
debug lane).

…terer

s_amr_sort_by_key (bottom-up stable merge sort of indices by int8 key)
replaces three hand-copied merge sorts. Goldens unchanged (61 AMR
tests, 19-test multi-rank debug lane).

…ll waves

s_amr_fill_drain walks a block's received transfers (fused or
per-transfer) for both fill waves, and s_amr_unpack_box_device takes the
patch origin instead of having a zero-origin twin. Goldens unchanged (61
AMR tests, 19-test multi-rank debug lane).

Both callers copied a slot's whole buffered box, so the six bound
arguments were noise. Goldens unchanged (61 AMR tests, 19-test
multi-rank debug lane).

77c0bbe moved the rank subdomain to amr_sidx/amr_ext but left
s_set_amr_fine_geometry's near-boundary test on its old locals, so
amr_xchg_coarse_ghosts was decided from stack garbage. Found by review.

…docs match the rules

A case-optimized build aborts when the runtime riemann_solver differs
from the baked one instead of leaving every flux buffer stale; jac_old
traffic moves to its own kernel, since m_igr allocates it for the Jacobi
solver alone; chem_params%diffusion without chemistry is rejected (its
consumers read below the source-flux band); the L0 tile rebalancer
slices the block-indexed arrays to the tile count; s_amr_build_block_coords
takes the global boundary array's lower bound instead of assuming -1;
and an Open MPI launch that oversubscribes says so.

check_amr's physics doc claimed phase change, 2D axisymmetric and
stretched grids work under amr while the rules reject all three
unconditionally, which also left four rules unreachable: the doc now
states what is enforced and the dead rules are gone.

…es nothing

The packer reads the simulation's ASCII dump under D/, which only the
serial writer emits, so the three cases that keep parallel_io = T packed
nothing and compared empty-against-empty: MPI-IO AMR restart, np=2
dynamic regrid and load_balance were all passing vacuously.

A golden case whose pack is empty now fails with a message saying so.
The load_balance case gets a probe at the material interface its
weighted split is decided from, so its golden holds real numbers; the
two MPI-IO restart cases are registered as run-only (kind="smoke"),
which is what they always were - their value is the round trip
completing with the reader's own header and extent checks armed - and
their empty golden files are gone. A serial multi-level restart np=2
case carries the numbers the MPI-IO twin cannot.

…ative

s_read_amr_restart pushed every owned block's host column to the device
after both readers had run. The serial reader already pushes each block
as it reads it - it must, because allocating the next slot can grow the
store device-side - and by then the reconcile's compaction has moved
columns on the device too, so the shared push copied an undefined host
mirror over the state just restored. The push now sits at the end of the
parallel reader, the only path whose host copy is authoritative when it
finishes. CPU builds cannot see the difference (host == device), so the
new 'store growth' case, which owns 13 blocks against the store's
initial 8 slots at np=1, earns its place on the GPU gate.

The coarse stage applies body forces, synthetic turbulence, the
continuum-damage clamp and the operator-split reaction/burn substeps to
the base grid; the batched fine stage applied none of them, and
s_amr_step_fold then overwrote the covered coarse cells with the
unforced fine average. With bf_x = T and amr = T the refined region
evolved with no gravity at all - and the AMR bodyforces golden, whose
comment claims coarse and fine see one consistent forcing, encoded that.

Body forces and the damage clamp now run per fine block through the same
swap/load/store path the 6-equation relaxation uses (s_apply_bodyforces
moves to m_body_forces so the AMR modules can reach it without a cycle).
The rest have no fine-block form - synthetic turbulence and the
spatial-support body force build their source on the coarse grid, and
the reaction and burn substeps are whole-domain operator splits run
after the fold - so they are rejected with amr instead of being silently
skipped. The bodyforces golden is regenerated: the refined region now
gains rho*g*t momentum (2.75e-2 over six steps at g = 10), which is the
forcing it was missing.

nvfortran rejects a CYCLE that follows an associated loop construct in
the same iteration body: 'CYCLE may only apply to the innermost
associated loop'. Caught by the NVHPC 25.5 gate.

…eady does

The rebuild seeds every new block by prolonging from its coarse side and
then copies back, from the stashed old blocks, the detail that coarse
side cannot carry. Level >= 2 skipped that second step, so an L2 block
was rebuilt from its parent's covered cells - which hold nothing but the
end-of-step restriction of that same block. The round trip
restrict-then-prolong is a low-pass filter, so everything finer than an
L1 cell was discarded every amr_regrid_int steps and the finest level's
accuracy could never accumulate past one regrid interval. The L2 stash
and its cross-rank migration were built and shipped for a reader that
did not exist.

The carry-forward now runs at every level. amr_isect_lo cannot be used
for the shift (for level >= 2 it is a footprint in the PARENT's fine
frame, not an L0 index); both stashes are anchored at their block's own
region_lo and hold amr_ref_ratio**lev fine cells per L0 cell, so the
frames differ by amr_ref_ratio**lev*(region_lo_new - region_lo_old).

Three multi-level dynamic-regrid goldens are regenerated: the solution
moves by up to 1.0e-2 relative (9.2e-4 absolute in 1D, 7.9e-3 in 2D,
1.5e-4 in 3D) - the L2 detail that was being thrown away - while total
mass is unchanged to 8e-16, which is also what says the frame shift is
right: a wrong one would not conserve.

…metry

post_process's per-rank-file reader decided which blocks carried a data
record from the geometric intersection with the local subdomain. Blocks
are owned WHOLE by one rank under the SFC cut, and that rank need not be
the one whose subdomain they sit in, so the writer emits a record only
for the rank's own blocks and marks the rest m = -1. The moment a block
crossed a rank seam the reader walked off the record boundary and the
overlay aborted with 'writer and reader header layouts have drifted'.
np = 1 never saw it (one rank owns and intersects everything), which is
the configuration the suite ran.

The reader now keys the stream on the file's flag, derives the
refinement ratio from the block's own region rather than the
intersection (a block may hang over the subdomain edge), keeps the
intersection sub-box the way the MPI-IO branch already does, and skips
level-0 tiles. What serial mode cannot do is render a block whose data
sits in another rank's file, so it says so once instead of quietly
writing that region coarse.

The new np=2 case puts a 32-cell block across the seam of a 63-cell
grid; it fails without this fix (post_process exits 1) and passes with
it. Reaching it needed the IO-key honoring to cover the post_process
branch too: that branch forces parallel_io = T, so no test could ask for
the serial reader at all - which is why this survived.

… them

With l0_ntile > 0 the tiles are the slot pool's fixed prefix and
amr_num_blocks counts them, so the AMR restart writer emits them beside
the fine blocks - and the reader rejected the very first record with
'block level outside 1..amr_max_level'. Every coexist run therefore
wrote checkpoints it could not read back, with nothing in the validator
to say so.

A level-0 record is now read as what it is: the tile already exists
(s_l0_tiles_init runs before the read), so the record is checked against
it and its data consumed - the level-0 field the ordinary restart file
restores is what reseeds the tiles anyway - and the fine-geometry
rebuild skips the prefix, which s_amr_assign_block_owners already did.

Both writers also record the block box from the replicated metadata
rather than amr_slots(k)%region: that field is written only by
s_set_amr_fine_geometry, which never runs for a tile, so a coexist file
carried the slot struct's uninitialized box. The serial writer now marks
a slot it does not own with -1 extents, the reader's data-presence flag,
instead of reporting extents it never set.

The coexist static np=1 case gains restart_check: it fails without this
(the round trip aborts) and passes with it.

…ree lane-specific gaps

The case-optimization guard I added to s_initialize_riemann_solvers_module
expanded to an if/call/end-if BEFORE the routine's declarations, which is
not Fortran: every --case-optimization simulation build failed to compile.
Verified from the generated source (the executable statements sat at line
453 and the declaration at 463); the guard now follows the declarations.
None of the local gates build case-optimized, which is how it got past
them.

Three more, each invisible to the lane we gate on:
 - post_process assigned to unallocated extents_dp, relying on F2003
   allocate-on-assignment, which ifx/ifort disable by default
   (-assume norealloc_lhs); both sites now allocate and free explicitly.
 - q_T_sf's Newton-guess seed reached the host only: @:ALLOCATE creates
   the device copy without copying and the ACC_SETUP copyin is Cray-only,
   so every other offload build iterated on uninitialized device memory
   over the part of the widened allocation no coarse conversion writes.
 - the post AMR overlay allocated q_cons over post's sys_size but filled
   only the simulation's, leaving the post-only Lagrange beta slot as heap
   garbage for the conversion to read; the block is zeroed at allocation,
   which is also the right value (fine blocks exclude the cloud).

Also: the restart round-trip no longer goes quiet when a non-smoke case
produces nothing to compare, and format.sh matches the token ffmt emits
('unmatched Fortran') instead of the bare word, with the temp file
removed on any exit.

…ms in the internals doc

The AMR physics doc claims chemistry support 'single- and multi-rank'
including species diffusion, but the diffusion case ran at np=1 only: the
one part of that claim with no test was the multi-rank seam, which is
exactly what the flux_src reflux of the species and energy diffusion
fluxes exists for. The np=2 twin keeps the derived cap, so the block
tiles to the rank extent and its coarse/fine boundary is also a rank
seam. Its conserved totals match the np=1 run to 4e-16.

Two statements in the internals doc had outgrown the code: store growth
now stages on the device up to amr_grow_dev_bytes (the carry-forward and
the stash became device kernels, so it no longer needs host == device),
and the communication is no longer blocking with fixed tags - every
exchange is a keyed-tag wave. In place of the second, the section now
records what the coverage audit found: 49 of 60 AMR cases pin
amr_max_grid_size and 30 regrid, while most physics couplings have one
case each at np=1, so the seam and ownership paths that have carried the
defects are the thin ones.

Four labelled paragraphs in a row (Exact conservation. / Free-stream
preservation. / Element-exact multi-rank. / Accuracy posture.) is a
template, not writing; they are two paragraphs of plain prose now. Both
table sections opened by announcing the table the reader is already
looking at. 'Ensuring', 'automatically', 'This correction is what makes
the scheme exactly conservative' and a tricolon in the first sentence
went the same way, as did a limitation that restated its own first
clause and one that only pointed at the section above it.

Also corrects regrid step 5, which said the new blocks are prolonged
from the coarse solution and stopped there - since 1654204 an old block
of the same level hands its own solution forward.

Same content, 543 words lighter (3455 -> 2912). The prose said things
twice (a paragraph that defined prolongation and then restated what it
guarantees), qualified what needed no qualifying ('dynamically
repositioned', 'performed on-device for every supported physics
configuration', 'a deterministic feasibility clamp ensures'), and buried
the point of a paragraph in its last clause. Long sentences carrying
three clauses are now two or three sentences. Nothing was dropped except
repetition.

…is wrong

1654204 read the same-level stash at every level with sh = amr_ref_ratio**lev*(amr_region_lo - old_ilo). Two frames in that expression do not hold. The stash extent is amr_ref_ratio**level times the block's L0 span (s_amr_regrid_stash_migrate), but a live level-3 block is sized at 4x its 19-cell span, not 8x, so the copy is told the source is 152 cells when it holds 76. It also replaced amr_isect_lo with amr_region_lo for level 1, and those differ whenever a block hangs over a rank subdomain edge (amr_isect_lo = max(lo, amr_sidx)).

The result is an out-of-range overlap copy: benign garbage under GNU and AMD, a segfault under CCE, and one level-3 block left identically zero, which post_process turns into a NaN. 'AMR -> 1D -> three levels' passes at 677d8c9 and fails at 1654204; 23 AMR tests segfault on the Frontier CCE CPU lane.

The three goldens regenerated by that commit are restored with it. Carrying level>=2 detail across a regrid is still worth doing, but it needs a frame conversion that the stash does not currently carry.

HONOR_IO_SNIPPET drops every mod the case already sets, so a test can keep its own parallel_io. BASE_CFG sets precision = 2 for every case, so the filter also deleted the --single guard's precision = 1 that the lines above it had just applied. All four honor_io_keys cases then asked post_process for double output on a single-precision build and failed validation: 'precision = 2 (double output) requires MFC built without --single'.

precision is a build guard, not an IO choice, so it is exempt from the filter. Verified on the --single lane with --test-all: C26D81E1, 5EFB3277, 78314D65 and 0253D658 pass, and C26D81E1 still keeps its explicit parallel_io = F.

…e procedure

s_amr_apply_reflux unrolled its batched loop nest three times into one procedure via the
Fypp #:for over the face directions. CCE 19.0.0 cannot compile that, and it fails
differently per target: the offload lanes abort at compile time with ftn-7991 "PDG node N
requires copy" (pdgcs/v_pdg.c), while the CPU lane compiles, silently emits bad code, and
segfaults the moment refluxing runs.

Bisected on a single-file compile with the CI flags: one direction compiles clean, two
(either 1,2 or 2,3) ICE, three ICE. It is the repetition, not any one construct, so the
fix is one loop nest per procedure -- a driver plus Fypp-generated
s_amr_apply_reflux_d1/d2/d3. Everything compiles at full optimization; no -Oipa0
exemption, which would cost occupancy on a hot kernel and which CMakeLists.txt already
warns can break cross-file device-call registration on GPU builds.

This is one root cause behind two apparently unrelated CI failures: the eight red CCE
offload checks (test lanes, Case Opt, Bench) and the eighteen AMR tests segfaulting on the
CCE CPU lane. The CPU half looked like a source bug and was not: -h bounds reports no
violation, and a debug build passes, because there is no out-of-range access to find. A
single-variable run on one node, everything else identical, pins it -- original routine:
5/5 segfault; split: 5/5 pass.

Verified on Frontier, all 64 AMR/L0/active-box goldens: CCE gpu-mp 64/64, CCE gpu-acc
64/64, CCE cpu 64/64 with zero segfaults.

…C lane

Three routines in m_amr_store read host-side arrays from inside a GPU region carrying
default(present): s_amr_st_move_slot, s_amr_st_reserve and s_amr_br_load/_store used
mbuf_lo/mbuf_hi (module integers in m_amr_state, never mapped) as loop bounds, and
s_amr_br_load/_store_faces read the dummy arrays w_lo(d)/w_hi(d) inside the loop body.
CCE's OpenACC runtime aborts on each with "find_in_present_table failed", while OpenMP
offload maps them implicitly and never noticed.

Scalars are firstprivate in both offload models, so hoisting needs no mapping and no
update when the store resizes -- unlike adding the arrays to a GPU_DECLARE, which would
add state to keep in sync. The bounds go through a new AMR_MBUF_BOUNDS macro so the
reason is stated once. s_amr_reflux_apply_faces already did exactly this for its own
copies of the same arguments, and says why; s_amr_br_*_faces now matches its sibling.

This was invisible until now: the CCE OpenACC lanes never reached execution because the
reflux ICE killed the build first. With that fixed, all 26 AMR tests the lane attempted
failed on one signature, then 45/19 after the mbuf half.

Verified on Frontier: CCE gpu-acc 64/64 with zero present-table errors, CCE gpu-mp 64/64.

is_buildable() returns False under --no-build, so every build() call in the test
pre-build loop was a no-op: the loop computed each unique slug and built nothing. A
variant the build phase had not produced then surfaced only once srun reached it, as
"execve(): .../bin/syscheck: No such file or directory" naming a bare hash, hundreds of
lines into the run and charged to the test rather than to the build that never happened.

Under --no-build the loop now checks that each selected case's binaries exist and fails up
front, naming the missing paths and the first case that needs each. Behaviour without
--no-build is unchanged.

This is the general fix for the AMD lane's missing eos_state_dependent variant: the next
compile-time axis added to Case.get_fpp's _prepend() will fail loudly instead of silently.
It caught three separate real mistakes while this branch was being fixed -- the missing
variant itself, a --debug flag inherited from MFC's persisted build config, and a partial
variant build -- each time turning a confusing late failure into an obvious immediate one.

The AMD GPU lane is the only one that does not enumerate build variants. Every other lane
runs ./mfc.sh test --dry-run -a over the whole case set, which builds each unique slug it
finds; 0857ace replaced that with a hardcoded "for v in base chem" so the two ~1 h
amdflang device LTO links could run as concurrent jobs inside the 2 h walltime. That was
complete when chemistry was the only always-on axis in Case.get_fpp's _prepend().

This branch moved eos_state_dependent from _case_optimization() into _prepend(), which
applies to every target, so it now feeds the build slug of every target including
syscheck. base (the empty case) bakes it False and chem selects on Chemistry, so nothing
built the True side and the lane ran srun against binaries that were never produced.

The correlation is exact: all 12 cases with a Mie-Gruneisen, JWL or Vinet fluid out of
831, and only those 12, failed. They share one slug per target, so any one of them builds
the variant for all; -o eos=mie_gruneisen selects nine and is enough.

Verified locally: with only base and chem built, those 12 report three missing binaries,
including the gpu-mp-a0d86a439e/bin/syscheck path from the failing CI log; after the eos
variant build, all present.

prebuild-case-optimization.sh builds syscheck, pre_process and post_process once from
benchmarks[0] behind a done-marker and lets concurrent shards skip them. That serializes
only the benchmarks[0] slug, so it is correct just while every benchmark hashes those
three targets to that same slug -- if two cases in different shards shared some other
slug, both shards would build that staging directory at once, unserialized.

The slug moves with any compile-time axis in Case.get_fpp: chemistry and (as of this
branch) eos_state_dependent from _prepend(), analytic ICs from the pre_process half. None
of the five benchmarks trips any of them today, which is luck rather than design -- the
suite's own hypoelasticity case uses Mie-Gruneisen while the benchmark of the same physics
does not set an eos at all.

Check it up front instead, in every shard so they all fail together rather than leaving
the others to wait out the 5400 s marker timeout. Verified both ways: passes on the
current five, and rejects a seeded Mie-Gruneisen benchmark naming all three targets.

./mfc.sh test writes this file when cases fail and deletes it again on a clean run, so a
committed copy appears and disappears in every working tree depending on the outcome of
the last run. 77c0bbe added it to the index on this branch. The existing /tests/*/**
pattern covers tests/<uuid>/ but not tests/ itself, so it needs its own entry.

Nine upstream commits, 11 conflict hunks across 7 files. Most resolved as unions or by
taking master's widened guards over this branch's allocation bounds; three needed more.

m_rhs / s_compute_additional_physics_rhs. 0ac3a3f threads flux_src_n through as an
explicit argument, but this branch rewrote that routine's body to read flux_src_rsx_vf
(m_riemann_state) and consequently allocates flux_src_n only for hypo_nc_mode_dual_pass.
Taking master's signature made every viscous and species-diffusion case dereference an
unallocated array at the call site; the parameter is unused by this branch's body, so it
is dropped. Note git also silently dropped master's flux_src_n_in declaration outside any
conflict region -- no marker, caught only by the compiler.

m_conduction. Fourier heat conduction accumulates into flux_src_n(E), which this branch
neither allocates nor reads, so master's conduction tests would have crashed -- and merely
allocating the array would have been worse, computing a flux nothing reads and failing on
values instead. Ported onto the path this branch already uses for chemistry diffusion: the
flux accumulates into flux_src_rsx_vf, which s_compute_additional_physics_rhs reads back
and which m_riemann_state already zeroes per sweep under the guard master extended.
m_conduction lives in src/simulation, so it uses the module array directly rather than
m_chemistry's flat dummy -- m_chemistry is in src/common, cannot use m_riemann_state, and
so pins src_lo to 1 to keep its lower bounds in agreement. src_lo must also start at
mom%beg when conduction is on: it writes E alone, and E sits below adv%beg, so the adv-only
band would put the accumulation out of range.

lint_test_suite. Upstream's new linter forbids a test case needing a build no CI lane
pre-builds -- the same failure this branch hit on the Frontier AMD gpu-omp lane, where
eos_state_dependent (a _prepend compile-time constant, like chemistry) moves the slug of
all twelve Mie-Gruneisen/JWL/Vinet cases. That lane's build is already extended with a
third `eos` variant, so this teaches _allowed_slugs about it, exactly as the linter's own
docstring prescribes for a lane whose build commands change.

Verified on Frontier, merged tree: CCE gpu-acc 68/68 -- the 64 AMR/L0/active-box goldens
plus master's four conduction cases. gpu-mp, gpu-acc and CPU all build clean.

Three upstream commits; the substantive one is dd7877d, which extracts m_eos and gives
the EOS families a single registry. Six conflict hunks in four files, plus two changes in
files that did not conflict at all.

m_eos. Master moved s_compute_mixture_coefficients and its _dt companion out of
m_variables_conversion into the new module, so the conflict reads as "delete this branch's
copy" -- but this branch's copy carries the stiffened-gas fast path, and the relocated one
does not. Taking the deletion alone would have dropped it silently: the result still
compiles and still computes the right answer, just with the state-dependent EOS chain
(s_phase_coefficients -> s_eos_coefficients -> s_reference_curve, a Newton loop) dragged
into every conversion and Riemann kernel of a stiffened-gas build. Ported into the
relocated routine instead; the _dt companion is identical and needed nothing. The
any_state_dependent_eos handling in s_initialize_eos_module also arrives as master's
runtime form, so it is restored to this branch's parameter semantics: unconditional
PROHIBIT, never assigned, never pushed to the device.

case.py. Master re-added the case-optimization block for any_state_dependent_eos; this
branch bakes eos_state_dependent in every build from _prepend, so the block goes -- but
master's EOS_STATE_DEPENDENT_VALUES registry replaces the hardcoded {3, 4, 5} here, so the
family set has one owner (params/eos_families.py) rather than two copies.

test_eos_selector. Master's new check requires every eos_* identifier under src/ to be a
registry-derived selector or a declared non-family name. eos_state_dependent is the Fypp
name _prepend bakes -- it carries whether ANY fluid's family is state-dependent, not which
one -- so it joins eos_coeffs and eos_rk4_steps in that list rather than EOS_FAMILIES.

The rest are unions: the public lists in m_variables_conversion, the use statements in
post_process/m_data_output, and m_eos alongside this branch's modules in
docs/module_categories.json.

Verified on Frontier, merged tree: CCE gpu-acc 68/68 -- the 64 AMR/L0/active-box goldens
plus master's four conduction cases -- and CPU builds clean. Note that a staging directory
configured before m_eos.fpp existed fails its CMake reconfigure with a GLOB mismatch and an
MPI detection error; clearing build/staging for the affected config is enough.