← Explainer · Markdown · Source on GitHub
# Align the existing data model with AMReX concepts **Design proposal, 2026-09-26.** Effort: AMR-02; prerequisite design: AMR-01. Status and owner: [work board](/docs/WORKBOARD). No runtime implementation is delivered by this document. S2/S3 behavior belongs to the [separate plan](/docs/amrex-alignment/S2_S3_PLAN). ## Outcome and strict boundary Represent every currently supported simulation using explicit domain geometry, index bounds, field placement and patch ownership. A running simulation still has **one physical resolution level and one patch covering the domain**, on one GPU. Existing moving geometry, boundary conditions, advection choices and measurements remain supported within their current limits. This effort adds no new scientific case, mesh arrangement, user option or numerical method. The benefit is a precise vocabulary and one current representation that S2 can extend. This is more than renaming `GridBlock`: consumers must use the new source of truth for extents, placement and coordinates. It is less than implementing a general AMR library. Keep C#/.NET and ILGPU. Do not add a C++ AMReX dependency, MPI, device distribution, new precision modes, arbitrary centering, generic field algebra or alternate storage backends. Do not add same-level exchange, coarse/fine interpolation, composite projection, refinement, adaptation or time subcycling. Do not turn an internal descriptor's ability to represent bounds into an advertised simulation capability. ## Current implementation and fit Inspection baseline: source tree reviewed 2026-09-26; recheck it when work starts. | Current structure | Reusable foundation | Alignment work / later boundary | | --- | --- | --- | | [GridBlock](https://github.com/hankbeasley/polycfd/blob/main/src/PolyCfd.Core/Core/GridBlock.cs), [GridDescriptor](https://github.com/hankbeasley/polycfd/blob/main/src/PolyCfd.Gpu/Core/GridDescriptor.cs) | Cartesian sizes, spacing and efficient kernel values | Separate physical domain from a patch's global index box and local array strides. No new nonzero-origin simulation support in AMR-02. | | [DeviceMacState](https://github.com/hankbeasley/polycfd/blob/main/src/PolyCfd.Gpu/Core/DeviceMacState.cs), [halo layouts](https://github.com/hankbeasley/polycfd/blob/main/docs/halo-layout.md) | Dense cell/face arrays; explicit device owners | Describe valid samples and actual stored halo topology accurately. GPU velocity has no persistent halos; pressure has packed face halos without corners. Preserve that layout initially. | | [SimulationDefinition](https://github.com/hankbeasley/polycfd/blob/main/src/PolyCfd.Gpu/Integration/SimulationDefinition.cs), [SimulationSession](https://github.com/hankbeasley/polycfd/blob/main/src/PolyCfd.Gpu/Integration/SimulationSession.cs) | Independent runtime, one numerical owner, selective capture | Bind the session to a one-level/one-patch layout and route existing fields through it. No independent per-patch timesteppers. | | [AdvectionGpu](https://github.com/hankbeasley/polycfd/blob/main/src/PolyCfd.Gpu/Advection/AdvectionGpu.cs), [PatchBcSet](https://github.com/hankbeasley/polycfd/blob/main/src/PolyCfd.Core/BC/Patches/PatchBcSet.cs) | Current validated boundary and sampling behavior | Make the current patch-equals-domain precondition explicit. Preserve clamping/wrapping; replacing it with interpatch sampling belongs to S2. A BC patch is a boundary region, not a mesh patch. | | [IGeometryDescriptor](https://github.com/hankbeasley/polycfd/blob/main/src/PolyCfd.Core/Geometry/IGeometryDescriptor.cs), [GpuMgHierarchy](https://github.com/hankbeasley/polycfd/blob/main/src/PolyCfd.Gpu/Multigrid/GpuMgHierarchy.cs) | Rebuildable solids and distinct solver-coarse geometry | Distinguish coordinate geometry from embedded-boundary metrics, and physical levels from solver levels. Preserve current geometry sampling and coarse construction. | | [ResultGrid](https://github.com/hankbeasley/polycfd/blob/main/src/PolyCfd.Core/Results/ResultPlanes.cs), [result bundles](https://github.com/hankbeasley/polycfd/blob/main/src/PolyCfd.Core/Results/ResultBundle.cs), [HDF5](https://github.com/hankbeasley/polycfd/blob/main/docs/HDF5_FIELD_FORMAT.md) | Explicit centering, physical coordinates, units and selected products | Derive current single-grid descriptors from the canonical runtime layout. Keep existing wire meanings and versions; native patch output is later work. | MAC staggering, matrix-free operators, device residency, Half geometry, float fields and double weighted reductions fit this direction. The major adaptation risk is the assumption that every block edge is a physical or periodic domain boundary. AMR-02 names and guards that restriction; it does not claim to remove it. ## Proposed contracts Names below are illustrative C# concepts, not a shipped API or schema. | Proposed concept | Meaning | AMReX reference concept | | --- | --- | --- | | `IndexBox` | Integer lower/upper bounds and sample placement | `Box`, `IntVect`, `IndexType` | | `DomainGeometry` | Physical bounds, domain index bounds, spacing, periodic axes | `Geometry` | | `LevelLayout` | Ordered patch boxes at a single physical resolution | `BoxArray` and per-level mesh metadata | | `PatchFieldView` | Borrowed typed data, placement, valid extents, storage strides and available ghosts | Per-box array view | | `LevelFields` | Field owners/views bound to one layout | Relevant subset of `MultiFab` semantics | | Session layout | Physical levels and ownership; initially exactly one of each | Hierarchy organization | AMReX gives each axis cell or node placement; a face field is node-based along its normal and cell-based along the other axes. Collections of boxes describe a level; field collections bind arrays to that layout. These are the concepts being borrowed, not its exact memory organization. [AMReX basics](https://amrex-codes.github.io/amrex/docs_html/Basics.html) ### Bounds and placement Use explicitly documented **inclusive** integer endpoints for the proposed box contract, matching AMReX's box convention. Convert once at existing half-open loops; do not scatter implicit `+1` adjustments through callers. For a cell box `[lo, hi]`, the cell count on an axis is `hi - lo + 1`; an X-face box has upper X endpoint `hi.x + 1`. Check overflow before computing allocations. Define an empty intersection explicitly rather than treating reversed endpoints as a valid patch. Use the current four placements: cell, X-face, Y-face and Z-face. For a global level index `i`, its physical sample coordinate on an axis is `domainMin + (i - domainIndexMin + offset) * spacing`, where `offset` is zero on a face's normal axis and one-half on cell-centered axes. A local buffer index first adds the patch lower index. Preserve the existing float evaluation in numerical kernels; existing double result-coordinate arithmetic is a separate contract. Descriptor tests should cover nonzero and negative integer boxes to catch hidden zero-based assumptions. Executable simulations retain current supported coordinate and extent restrictions. The session must reject an unsupported layout before allocating fields; descriptor expressiveness is not runtime acceptance. ### Valid regions, ghosts and memory Valid field extents describe physical samples. Allocated storage describes where the samples and any ghosts live. Neither one implies a padded rectangular allocation. In particular, packed pressure face halos must not be represented as a fully grown box with available edge/corner values. Mark actual halo topology and availability. Preserve the current dense field buffers and specialized kernel views. Do not allocate velocity halos or gather/scatter full fields merely to resemble a `MultiFab` layout. S2 will choose the storage and sampling policy required by its operators and prove the cost. AMR-02 only needs metadata/views actually consumed by today's runtime. One session owns the layout and device state; views borrow from those owners. Preserve disposal order, completed-step capture identity and capture masks. A topology revision mechanism is unnecessary while topology cannot change; design its later invalidation responsibilities without adding unused caches or schedulers now. Keep layout independent of execution placement. AMReX's `DistributionMapping` maps boxes to MPI ranks; it is not a GPU-thread layout. PolyCFD initially uses the session's single accelerator owner. Do not add fake ranks or a distribution service to imitate that API, and do not encode CUDA-specific device identity into an index box. ### Domain and embedded geometry Keep physical coordinates/periodicity distinct from solid descriptions and cut-cell fractions. Embedded-boundary data belongs to a patch and geometry revision. Current, previous-motion and solver-coarse metrics must remain separate owners or explicitly borrowed views. Do not change SDF quadrature, solid thickening, classification, small-volume treatment or precision as part of a structural migration. AMReX likewise has a separate embedded-boundary geometric database and field data support; adopting that separation does not require changing PolyCFD's cut-cell method. [AMReX embedded boundaries](https://amrex-codes.github.io/amrex/docs_html/EB.html) ### Physical levels and solver levels A physical level holds simulation state at a spatial resolution. A multigrid level holds the auxiliary state used to solve an equation. They need distinct identities and types even when both use box/placement descriptors. Keep `GpuMgHierarchy` as a solver concern. Do not wrap its coarse buffers as additional simulated levels. AMReX's linear-operator API distinguishes AMR and MG level indices as well. [AMReX linear solvers](https://amrex-codes.github.io/amrex/docs_html/LinearSolvers.html) ## Migration sequence These are ordered steps within AMR-02, not independently completed capabilities. 1. **Capture the existing contract.** Inventory descriptor consumers, BC dispatch, geometry builders, source terms, probes, resource estimates and output adapters. Select representative uniform, cut-cell, periodic and moving cases with the currently accepted settings. Record the accepted revision and baseline identities. 2. **Introduce the canonical descriptors.** Implement bounds, placement and domain mapping, then use them to derive the existing kernel descriptors and result metadata. Keep fast value-type kernel parameters; avoid virtual dispatch in kernels. Test box conversion, centering counts, coordinates, empty intersections and overflow. 3. **Migrate ownership and consumers.** Make the session's one-patch level real in its runtime path. Move geometry/BC responsibilities to their appropriate owners. Preserve numerical loops and field storage. Migrate maintained callers and remove superseded representations and compatibility aliases in the same change. 4. **Prove equivalence and document the boundary.** Exercise all maintained execution, measurement, capture, replay and viewer paths. Update component guides and runtime contracts. Retain unsupported-layout rejection and measure the cost of the refactor. **Context:** `src/PolyCfd.Core/Core`, `src/PolyCfd.Gpu/Core`, `src/PolyCfd.Gpu/Integration`, `src/PolyCfd.Gpu/Advection`, `src/PolyCfd.Gpu/Multigrid`, `src/PolyCfd.Core/Results`, `docs/RUNTIME_CONTRACTS.md`, `docs/MAC_GRID_CONVENTIONS.md`, `docs/halo-layout.md`. ## Acceptance - Current supported scenarios and typed API callers preserve their resolved physical inputs, field placement, solver options, source/stage ordering, measurements and capture semantics. Existing limitations remain explicit. Multi-patch/multilevel requests are rejected before device allocation, even if internal descriptors can express them. - Existing scientific gates pass without changed references, windows or tolerances. Compare preserved and candidate builds on representative fields and measurements. Expect unchanged indexing and values where arithmetic is unchanged; diagnose every difference rather than requiring bitwise equality of unrelated parallel reductions. - No extra steady-state geometry upload, duplicate capture, full-volume readback for scalar/plane demand, or new field-sized allocation solely for the abstraction. Follow [architecture performance acceptance](https://github.com/hankbeasley/polycfd/blob/main/docs/PERFORMANCE_VALIDATION.md#architecture-change-acceptance), including per-workload measurements on an idle machine. Unresolved regressions block acceptance. - Run `python3 tools/verify static fast gpu` on the Windows checkout. Run affected Workbench tests in WSL with Linux-local artifacts as required by [AGENTS.md](https://github.com/hankbeasley/polycfd/blob/main/AGENTS.md). Run affected viewer tests/build/lint. If a wire contract must change, use the official generators and update all consumers; do not invent an AMR wire format in this effort. - For runtime/indexing/operator integration, run the appropriate CUDA `verify-all` coverage from [validation selection](https://github.com/hankbeasley/polycfd/blob/main/docs/COMPLETE_VALIDATION.md#choose-coverage-for-the-change), and the full suite before broad numerical acceptance. Record receipt paths, counts, `notRun` entries, reference limitations and timing conditions. CPU backend not run (user directive). Isolate build artifacts if another campaign owns binaries. The plan completes only after these gates pass; writing types or documentation alone does not complete it. Move lasting contracts into current runtime/geometry/result guides and retire this plan with links updated when implementation is accepted. ## Decisions before implementation Adopt the semantic separation and single-patch constraint described above. Exact C# names and whether a one-level wrapper earns its cost should be resolved against real callers in step 1; avoid unused generic layers. The proposed inclusive box convention needs explicit implementation review because existing loops use half-open bounds. Native AMReX integration would reuse more code, but changes the retained language, backend and ownership architecture. Copying the complete AMReX API would create unsupported surface area. Both are outside this effort. Data-model alignment saves structural design work; it does not implement or validate AMR numerical algorithms.