← Explainer · Markdown · Source on GitHub
# S2 and S3: from aligned data to multiresolution simulation **Design proposal, 2026-09-26.** Roadmap outcomes: [S2 and S3](/docs/ROADMAP#s2--uniform-multiblock-on-one-gpu). Current status: [work board](/docs/WORKBOARD). No executable multiblock or AMR support is established by this document. Begin with [AMR-02 data alignment](/docs/amrex-alignment/DATA_MODEL_ALIGNMENT). ## Delivery sequence AMR-02 preserves one block. S2 adds one resolution across multiple patches. S3 adds coupled refinement, then adaptation. Keep the same level/patch representation through all three; do not create an alternative refined solver alongside a legacy uniform path. Existing roadmap workflow priorities and one-GPU scope remain in force. | Gate | Smallest useful implementation | What passing establishes | | --- | --- | --- | | AMR-02 | One level, one full-domain patch | Structural migration with existing behavior preserved | | S2 first experiment | One uniform problem split into two patches | Same-resolution interfaces work for the declared simple case | | S2 acceptance | Partitioned operators, supported geometry/BCs, measurements and output | A complete supported multiblock workflow on one GPU | | S3 fixed refinement | One full-domain coarse patch and one aligned 2:1 fine region; shared timestep | Validated two-way coupling for an explicitly bounded case | | S3 extension | Several fine patches and levels, then regridding | Each added layout/physics combination passes its own acceptance | A two-patch experiment is not full S2 acceptance. Fixed refinement is not adaptive refinement. Thermal or moving-geometry support does not transfer automatically to a new layout. Unsupported combinations must fail preflight, not silently fall back. ## S2: same-resolution patches on one GPU **Depends on:** accepted AMR-02, W1 workflow and representative S1 measurements. ### 1. Layout and field ownership Accept a bounded collection of non-overlapping cell boxes whose union covers the domain. All patches at this level share spacing and physical coordinate mapping. Validate gaps, overlaps, bounds, minimum operator extents and allocation overflow. Keep cell valid regions distinct from ghost storage and shared face samples. Build adjacency/intersection information, including periodic images, once per layout. Cell boxes are disjoint, but neighboring face-centered arrays include the same shared face. Define one authoritative value or a deterministic synchronization rule, and avoid counting shared faces twice in relevant reductions. Keep ownership on one GPU; do not add MPI, load balancing or a general scheduler as prerequisites. ### 2. Boundary filling and advection Separate physical boundary application, periodic-domain mapping and interior patch exchange. Fill same-level data through device operations at the stages that need it. Never apply a wall condition merely because a stencil reaches a patch edge. Replace local-domain clamping/wrapping in advection with a sampling policy that can reach neighboring patches. RK2 departure points and MacCormack's forward/backward stencils can require more data than a nearest-neighbor pressure stencil. Decide between sufficient filled ghost regions, patch-aware lookup or a bounded combination, and derive the required reach from the supported timestep and scheme. Handle edge and corner samples and periodic seams explicitly. Do not assume the current packed pressure face halos are sufficient for velocity interpolation. AMReX distinguishes neighbor exchange from physical boundary filling and its `FillPatch` utilities assemble same-level or coarse-level values at the requested time. Use that separation as the model, with numerical policies chosen for our solver. [AMReX AmrCore](https://amrex-codes.github.io/amrex/docs_html/AmrCore.html) ### 3. Coupled operators, geometry and diagnostics Advance one coupled solution, not independent per-patch simulations. Pressure and implicit diffusion must account for cross-patch stencils. Extend pressure connected components and gauges across interfaces, including periodic joins; reductions remain double and appropriately weighted. A solid may span a patch boundary, and disconnected fluid pockets may span several patches. Validate both. Build cut-cell geometry at global physical coordinates with consistent interface metrics. Preserve distinct current/previous/solver-coarse states. Extend sources, motion updates, force extraction, section integration and domain mass balance to the layout. Bound each supported motion combination with evidence; preserve existing single-patch functionality throughout the work. ### 4. Authoring, results and workflow Add explicit typed/serialized partition support with capability checks and resource estimates. Carry patch extents, placement and ownership through selective capture, saved fields and live products. Introduce only the versioned contract changes actually needed; update maintained fixtures, generators, decoders and viewers together. A stitched or resampled preview may be useful, but it must identify its sampling rule and must not masquerade as native samples. Measurements keep physical coordinates and their scientific meaning independent of the partition. Verify replay of the resolved problem and layout; it remains replay, not numerical restart. ### S2 acceptance Compare monolithic and partitioned runs of the same physical problem, using declared unchanged scientific tolerances. Start with two patches and add unequal extents, noncubic domains, all interface orientations, corners, periodic seams, cut geometry crossing interfaces and cross-patch pressure components. Compare field trajectories, divergence, operator residuals, forces/sections and mass balance, not just final images. Verify shared-face consistency, cancellation/disposal, selective capture and memory limits. Measure setup, solver, exchange and output costs independently. The one-patch path must preserve its accepted performance; partitioning is not assumed to accelerate the same grid. Apply [performance acceptance](https://github.com/hankbeasley/polycfd/blob/main/docs/PERFORMANCE_VALIDATION.md#architecture-change-acceptance). ## S3: fixed refinement, then adaptation **Depends on:** accepted S2; explicitly integrate each delivered thermal/motion capability. ### 1. Restrict the first problem Start with a single rectangular coarse patch covering the entire domain and one stationary rectangular fine patch. Proposed initial ratio: 2:1 on each axis, with coarse-cell-aligned extents and a shared timestep selected for the finest constraints. Use a simple stationary flow case first. Specify the allowed boundary contacts and proper-nesting/ghost-support buffer before implementation; reject other layouts. This is a proposed first subset, not a final decision on every supported refinement ratio. The representation must allow a level to contain multiple patches. Multiple fine regions, patch adjacency, more levels and physical-boundary contacts each require later checks. A full-domain coarse patch can remain the initial runtime restriction. ### 2. Coverage, transfer and geometric consistency Build coverage masks so diagnostics and composite equations use uncovered coarse cells plus fine cells. Covered coarse values may still serve transfers and solver work, but must not be counted again as physical volume or heat content. Define restriction, prolongation and synchronization separately for cell quantities, face velocity/flux and embedded geometry. Preserve constants and the relevant weighted integrals; test linear fields where the chosen method claims that accuracy. Constrain the coarse-face normal volume flux to the sum of the corresponding fine open-face fluxes, with consistent area/aperture weights. Specify treatment of partially blocked interfaces, small cells and unresolved geometry before supporting cut interfaces. Rebuilding an SDF at each resolution does not by itself make coarse and fine open areas or volumes consistent. Establish a metric aggregation/correction policy and show its effect on conservation and physical accuracy; do not change the physical solid merely to make coupling convenient. ### 3. Composite pressure, diffusion and transport Construct a pressure correction that couples active cells across coarse/fine interfaces, with compatible discrete divergence/gradient operators and a gauge for each physical fluid component. Verify the symmetry/positive-definiteness assumptions required by PCG in the chosen weighted inner product. Adapt multigrid to the composite operator; its coarsening levels remain distinct from simulation refinement levels. Independent fine-patch pressure solves with interpolated boundaries are not acceptance. AMReX-Hydro documents multilevel projection operators; they are a reference for the coupling problem, not a drop-in implementation of PolyCFD's cut-cell discretization. [Projection methods](https://amrex-fluids.github.io/amrex-hydro/docs_html/Projections.html) Implicit diffusion needs cross-level coupling or a demonstrated synchronization scheme as well. Define velocity sampling and limiter behavior across resolution changes. Existing semi-Lagrangian/MacCormack transport is not a conservative finite-volume flux update: adding a generic flux register does not automatically make it conservative. Decide whether to retain it with explicit validated budgets or introduce a separate, deliberately scoped numerical change. Do not silently weaken mass, momentum or heat checks to retain the current method. For quantities advanced through conservative face fluxes, specify mismatch correction using the actual area/time-integrated fluxes. Synchronized timesteps avoid temporal interpolation initially, but do not eliminate spatial flux mismatch. AMReX's flux-register example explains this distinction. [Flux registers](https://amrex-codes.github.io/amrex/docs_html/AmrCore.html#using-fluxregisters) ### 4. Measurements, output and delivered physics Integrate named sections and global budgets using the finest active coverage without double counting. Define a physical plane that intersects several levels; one global integer slice index is insufficient. Native output must retain level spacing, patch bounds, coverage and completed-frame/layout identity. Preview resampling must remain explicit and bounded in memory, readback and retained leases. If temperature is delivered, validate scalar transfer, diffusion, sources and heat budgets across interfaces before advertising refined thermal cases. Validate moving geometry crossing levels separately, including connectivity, previous/current volumes and pressure gauges. Declare unsupported combinations in preflight and UI. ### 5. Adaptation after fixed-refinement acceptance Add deterministic tagging/clustering, proper nesting and grid-size constraints. Regrid only at defined synchronized states. Transfer solution state, restrict removed fine regions appropriately, initialize new fine cells, and restore the required divergence/flux constraints. Rebuild geometry, connectivity, solver state, source bindings, coverage and selection mappings safely. Introduce layout revision and lifetime rules so outstanding frames keep valid storage or are explicitly rejected; a geometry revision alone cannot identify changed topology. Record actual layouts/tagging settings for reproducibility, and validate memory peaks while old and new layouts coexist. Time subcycling remains a separate later decision. ### S3 acceptance Use manufactured/interface tests and uniform-fine reference solutions. Establish spatial and timestep convergence, coarse/fine flux mismatch, divergence, pressure coupling and applicable mass/momentum/heat budgets. Freeze case-specific limits before evaluating results. Solver residual reduction alone establishes neither conservation nor physical accuracy. Fixed refinement must reach declared measurement accuracy with less measured memory or time than a comparable uniform-fine solve; include solver, geometry and required outputs. Then test multiple/touching patches, additional levels and supported boundary contacts. Adaptation needs separate refine/derefine cycles, moving features, transfer error, budgets and cost evidence. A fixed-refinement pass does not complete all of S3. ## Verification and decisions For each implementation increment run repository static/fast/GPU verification and appropriate CUDA numerical coverage, then full coverage for broad acceptance. Use the [verification guide](https://github.com/hankbeasley/polycfd/blob/main/test/README.md), [suite selection](https://github.com/hankbeasley/polycfd/blob/main/docs/COMPLETE_VALIDATION.md#choose-coverage-for-the-change) and [runtime contracts](https://github.com/hankbeasley/polycfd/blob/main/docs/RUNTIME_CONTRACTS.md). CPU backend not run (user directive). Keep existing references, scientific limits and performance allowances unchanged; report missing hardware, occupied devices and incomplete checks. New supported cases need intentionally reviewed references, not refreshed baselines to hide regressions. Before S2 coding, resolve velocity ghost/sampling reach and shared-face ownership. Before S3 coding, resolve nesting, transfer operators, composite pressure/diffusion, transport conservation and cut-interface metrics. These are scientific decisions with validation obligations; the AMReX-style data model does not decide them for us. AMR-02 does not alter W4's binding decisions. If later result changes affect those decisions, follow their approval and generation rules at that time. Move accepted runtime contracts into the current guides and retire completed plan sections once the corresponding milestone is delivered.