Box
Where a patch lives in the level's integer index space.
[lower, upper]AMReX alignment / design proposal / 26 September 2026
From one grid
to a hierarchy.
Give today's solver a clearer structure. Validate better transport on the same grid. Then connect patches and add resolution where the physics needs it.
Four separate efforts. Alignment preserves today's behavior. Conservative transport must prove its single-grid benefit before multiblock and refinement acceptance. These are plans, not delivered capabilities.
One hard execution requirement across every effort: all simulation processing stays on the GPU, with no device-to-host copies for CPU computation, including scalar results. GPU-01 closes current gaps; requested final output is a separate boundary.
01 / Vocabulary
A domain is a place.
A patch is a piece of storage.
The domain is the physical region being simulated. A level gives it a particular grid spacing. A patch stores a rectangular part of that level.
One coarse patch can cover the entire domain. Fine levels can use several smaller patches. Splitting a level into patches does not change its resolution.
AMReX uses boxes and field collections to express this separation. Official data-model reference.
Physical domainbounds · coordinates · periodicity
Level 0 · spacing h
Patch Aindex box + fields
Patch Bindex box + fields
Placement
Where the quantity is sampled relative to a cell.
cell · x/y/z faceField view
How a kernel reaches values without owning their memory.
bounds + strides + dataLevel layout
Which boxes share a resolution and cover which region.
spacing + patch boxes02 / Follow the same physical domain
The representation changes first.
The mesh changes later.
Choose a stage to inspect the difference. These are schematic 2D grids, not simulation results.
AMR-02 / same capability
One patch, expressed clearly.
The solver still advances the same grid. Domain coordinates, patch bounds, field placement and memory ownership become explicit and consistently used.
- What changes
- Data contracts and their consumers.
- What must be proved
- Existing scientific results, selective capture and performance are preserved.
160 active cells · 1 physical level · 1 patch
Why does keeping one coarse grid not solve all the interface work?
It guarantees base coverage. Fine patches still need values from their neighbors, and the coarse and fine regions must agree on normal volume flux and pressure coupling. The coarse solution must respond to the fine region as part of one physical solution.
Why are allocated and active cells different in the refined illustration?
The fine patch covers 48 of the 160 coarse cells and contains 192 fine cells. We retain 160 + 192 = 352 cell slots, but the physical composite uses 160 − 48 + 192 = 304 active cells. A uniformly fine 2D grid would contain 640 cells. These counts exclude face arrays, ghosts, geometry and solver buffers; they are not a GPU memory or speed prediction. In 3D, a 2:1 refinement divides a coarse cell into eight fine cells.
03 / Improve the foundation first
Balance what moves.
Then measure the accuracy.
Current transport
Trace back and interpolate
Each velocity sample follows the fluid backward and reads an interpolated old value. Our MacCormack correction reduces interpolation error, but does not explicitly balance momentum transferred between neighboring control volumes.
Proposed conservative transport
Use a shared transfer
The same amount leaves one control volume and enters its neighbor. Internal transfers cancel in the total. Boundary flows, pressure, viscosity and applied forces still need their own accounting.
TRN-01 is a numerical effort on the existing grid. Start with current momentum budgets, test a conservative scalar building block, then derive MAC momentum transport and qualify boundaries and cut cells. A scalar test is not proof of momentum conservation or a delivered temperature feature.
| Question | Evidence |
|---|---|
| Is transport conservative? | Momentum budgets close after accounting for physical boundary exchange and sources. Pressure projection's small divergence is a different check. |
| Is the solution more accurate? | Preselected traveling-flow, vortex, separation and cut-cell cases measure field, phase, force and energy errors against appropriate references. |
| Is the benefit worth its cost? | Compare matched grid/timestep results and time to reach a target accuracy. A tighter stability limit can require more steps. |
Starting candidate: limited second-order finite-volume transport compatible with our MAC grid; exact method selection remains open. Scope, tests and acceptance. AMReX supports different equation formulations; conservation is our design choice, not a requirement of its data types.
Keep method selection small and explicit.
We already support SemiLagrangian and MacCormack. TRN-01 will retain useful choices and evaluate one conservative candidate. Select one method per run, use it across all patches and levels, and record it with the scientific settings. Each method owns its temporary buffers and declares its timestep, stencil and geometry requirements.
Support is qualified separately for each method and configuration. Initially S3 can require the accepted conservative method. Unsupported combinations must be rejected clearly; the solver must not silently switch methods.
Drill down: method selection and integration · current transport equations · shared-flux derivation.
04 / What already fits
Our MAC fields have a natural home.
Pressure and velocity should not be forced into one identically shaped array. A patch with Nx × Ny × Nz cells has different valid extents for each field.
- Pressure
- Nx × Ny × Nz
- U velocity
- (Nx + 1) × Ny × Nz
- V velocity
- Nx × (Ny + 1) × Nz
- W velocity
- Nx × Ny × (Nz + 1)
We keep device-resident arrays, float fields, Half geometry and double weighted reductions. Alignment makes their meaning and ownership explicit.
Drill down: physical equations and MAC control volumes · why scalar transport is not enough for momentum.
05 / Where the hard work lives
Three boundaries.
Three different responsibilities.
The physical boundary
A wall, inlet, outlet or periodic domain edge. Apply the problem's physical conditions here.
A same-resolution patch edge
An internal storage split. Obtain neighboring samples and agree on the shared face value; this edge is not a wall.
A coarse/fine interface
Different resolutions describe the same fluid. Couple pressure, transfer fields and make weighted fluxes consistent.
S2 must use the accepted transport method's reconstruction, ghost reach and time stages, and apply a consistent flux across shared interfaces. Any retained semi-Lagrangian method also needs patch-aware departure-point sampling in place of current local clamping or wrapping.
Drill down: interface cancellation, average-down and refluxing · cut-cell and moving-boundary balances. References: AMReX boundary filling and AMReX-Hydro projection methods.
06 / Our current code
Adaptable foundations.
Specific assumptions to untangle.
| Area | AMR-02: representation | TRN-01 / S2 / S3: behavior |
|---|---|---|
| Transport | Preserve current equations and numerical choices. | TRN-01 validates conservative momentum transport on one grid before extending it to interfaces. |
| Grid & coordinates | Separate domain geometry, global boxes and local strides. | Support several patches and new refinement layouts. |
| Session & fields | One level with one patch; explicit owners and borrowed views. | Coordinate operators and reductions across patches. |
| Boundary sampling | Name and guard the current patch-equals-domain restriction. | Exchange same-level data, then fill coarse/fine data. |
| Solid geometry | Separate coordinates from cut-cell metrics; preserve sampling. | Establish consistent geometry and fluxes across interfaces. |
| Results & viewers | Derive current metadata from the canonical layout. Preserve wire meanings. | Carry native patch/level coverage or label resampled previews. |
Physical refinement levels
Where the solution lives
Coarse background plus finer local regions. These levels carry physical simulation state and affect resolution.
Coarse domain → fine region
Multigrid solver levels
How an equation is solved
Auxiliary coarse problems accelerate pressure convergence. They do not add locally refined simulation state.
Pressure problem → coarser corrections
Both can use box descriptors. They need separate identities and ownership. Existing GpuMgHierarchy is the second kind.
Names that make the responsibilities clear
| Current name | Planned name | Responsibility |
|---|---|---|
BoundaryPatch / PatchBcSet | BoundaryRegion / DomainBoundaryConditions | Physical boundary regions and conditions, separate from mesh patches. |
IGeometryDescriptor | IEmbeddedBoundarySource | Solid description used to build numerical embedded-boundary data on the GPU. |
CellGeometry | EmbeddedBoundaryData | Cell and face cut metrics/classifications; the device type follows the same name. |
GridBlock | MeshPatch | A patch within a physical level, after domain/level responsibilities are separated. |
GridDescriptor | PatchKernelDescriptor | A compact derived kernel input; canonical bounds, placement and storage define it. |
Separate GridMetrics' coordinate scalars from borrowed embedded-boundary metrics. Share a canonical field-placement description between runtime and output. A level coordinator schedules patch kernels; physical boundary application, same-level exchange and later coarse/fine filling have distinct jobs and explicit stage freshness.
Keep meaningful names such as SimulationSession, MacState, PressureWorkspace and GpuMgHierarchy. Naming, ownership and migration design.
07 / Hard requirement
Compute on the GPU.
Keep the next input there.
All simulation processing must stay on the device: initialization, numerical geometry, transport, solvers, complete reductions, convergence and timestep decisions, measurements and later refinement. Copying a small scalar to the CPU to finish a calculation is still a computational readback and is prohibited.
Resident computation
GPU kernels consume fields, coefficients, time state, component information and fluxes directly. Host code can parse inputs, own resources and schedule launches; it cannot finish numerical work from downloaded data.
Explicit final output
Compute a requested measurement, slice or export product on the GPU first. Only the final requested product crosses to file or presentation code. Its host copy never becomes input to the numerical update.
Gate: zero computational device-to-host calls and bytes across setup, steps, geometry changes and failures. Audit direct and indirect library copies as well as wrappers. Requested output is counted separately. Scientific checks, precision and performance limits remain binding.
GPU-01 scope, source inventory and acceptance · Algorithm execution details.
08 / Numerical review
Follow an idea all the way
to its equations and evidence.
The algorithm companion starts with the physical model and opens into discrete equations, source code and validation protocols. Current behavior, proposed algorithms and unresolved derivations are labeled separately.
- Start with physics and field locationsIncompressible flow, units and staggered control volumes.
- Trace today's complete time stepForces, diffusion, pressure signs and continuity.
- Inspect today's transport equationsBacktracing, interpolation, correction and limiting.
- Derive a conservative updateShared transfers and an illustrative scalar algorithm.
- Identify the work still to deriveStaggered momentum, cut volumes and moving walls.
- Review the proof and test obligationsConservation, stability, accuracy and physical validation.
09 / Delivery gates
Prove each step before adding the next.
GPU-01's hard execution gate applies to every runtime acceptance below. Its migration can overlap AMR-02; writing these documents does not complete it.
- AMR-02
Align today's structures
Inventory consumers, introduce canonical descriptors, migrate ownership and callers, then remove superseded representations.
Gate: existing inputs, fields, measurements, outputs and scientific acceptance remain valid. No new mesh capability and no abstraction-driven performance regression.
- TRN-01
Validate selectable single-grid transport
Measure current budgets, test shared scalar fluxes, formulate MAC momentum transport, then qualify boundaries, cut cells and separately supported motion. Integrate one recorded method choice per run with explicit support, timestep and memory requirements.
Gate: conservation and stability checks pass, retained methods preserve their scientific gates, and preselected cases demonstrate candidate accuracy benefit with acceptable cost. Selection is reproducible and unsupported configurations are rejected.
- S2
Connect patches at one resolution
Begin with the same problem split in two. Extend the accepted transport method with shared fluxes and same-level sampling, plus coupled pressure/diffusion, geometry, measurements and output.
Gate: partitioned and monolithic solutions and momentum budgets agree within declared tolerances, including periodic and cut interfaces. A simple two-patch test alone does not finish S2.
- S3 / fixed
Add resolution in one region
Start with one coarse base, one aligned 2:1 fine patch and synchronized timesteps. Add coverage masks, transfers and composite equations.
Gate: accuracy and conservation pass against uniform-fine references, with less measured memory or time at comparable measurement accuracy.
- S3 / adaptive
Let refined regions change
Extend fixed refinement to multiple patches/levels. Then add tagging, regridding, safe state transfer, layout revisions and geometry/solver rebuilding.
Gate: refine/derefine cycles preserve the required budgets and constraints. Thermal and moving-geometry combinations need their own evidence. Time subcycling is a later decision.
S2 layout work can overlap transport development. Its transport interfaces follow TRN-01's established method. Coordinate scalar work with T1, while keeping thermal acceptance separate.
Continue reading
The design behind the diagrams.
Algorithm companionDetailed numerical referenceData-model alignmentSingle-level transportS2 and S3 implementationRoadmapCurrent task status
The Markdown plans define scope and acceptance. The work board owns current status. Proposed type names and diagrams are explanatory; they are not executable API or file-format specifications.