Repository navigation
Dof access - #400
Open
jacobmerson wants to merge 2 commits into
Open
Dof access#400jacobmerson wants to merge 2 commits into
jacobmerson wants to merge 2 commits into
Conversation
The FieldData class is intended to be a opaque type that only the field that creates it can unpack/work with. In some cases, like nimrod/fusion-io fields, we do not actually have access to any data/field coefficients explicitly, although we can evaluate those fields. This indirection provides us a way to model FieldData that we have the coefficients for, vs. those we don't. The side benefit of this approach is it gives us a convenient place to deal with packing data on device side, and for fixing the set/get full fields rather than subsets.
jacobmerson
force-pushed
the
dof-access
branch
from
October 25, 2026 22:25
ac9a79c to
8fc002b
Compare
Collaborator
|
That's right. I don't use C-API in XGC anymore. |
Collaborator
Author
|
Most of the diff here is whitespace changes in the generated Fortran/C code. |
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Host reads add full-field device allocations and copies to every exchange, increasing memory traffic and risking GPU memory exhaustion.
1 open issue
What changed in this PR
This PR separates direct degree-of-freedom access from field representation and centralizes overlap handling, preparing PCMS for evaluation-only sources alongside owned and wrapped field data.
Changes:
- Introduces optional
DOFAccessand routes field reads, writes, and serialization through it. - Moves XGC replica synchronization into field data and registers overlap masks separately.
- Updates C/Fortran bindings and tests for the revised interfaces.
| File | Description |
|---|---|
test/xgc_n0_coupling_server.cpp |
Uses default field serialization. |
test/test_xgc_field_data.cpp |
Tests classification and overlap separately. |
test/test_uniform_grid_field.cpp |
Uses field-level data setters. |
test/test_proxy_coupling.cpp |
Uses registered fields for data access. |
test/test_proxy_coupling_xgc_server.cpp |
Separates overlap and replica synchronization. |
test/test_proxy_coupling_xgc_server_overlap.cpp |
Updates overlap and field registration. |
test/test_proxy_coupling_xgc_client.c |
Registers overlap through the new C API. |
test/test_proxy_coupling_xgc_client_fortran.f90 |
Registers overlap through the Fortran API. |
test/test_polynomial_reconstruction_mls_evaluation.cpp |
Migrates field initialization calls. |
test/test_polynomial_reconstruction_function_space.cpp |
Updates data round-trip access. |
test/test_point_evaluator.cpp |
Migrates evaluation fixtures to field accessors. |
test/test_omega_h_lagrange_field.cpp |
Updates field initialization and reads. |
test/test_omega_h_field2_outofbounds.cpp |
Simplifies field initialization. |
test/test_mfem_adapter.cpp |
Exercises MFEM access through a field factory. |
test/test_field_serializer.cpp |
Tests generic XGC serialization with overlap. |
test/test_field_evaluation.cpp |
Updates field initialization. |
test/field_test_utils.h |
Centralizes initialization through field setters. |
src/pcms/fortranapi/pcms.f90 |
Updates generated handles and overlap bindings. |
src/pcms/fortranapi/client.i |
Revises SWIG interface declarations. |
src/pcms/fortranapi/client_wrap.c |
Updates generated C wrappers. |
src/pcms/field/layout/xgc.h |
Removes overlap from layout construction. |
src/pcms/field/layout/xgc.cpp |
Classifies and owns all plane entries. |
src/pcms/field/function_space/xgc.h |
Updates construction and data validation. |
src/pcms/field/function_space/spline.cpp |
Validates size through DOFAccess. |
src/pcms/field/function_space/polynomial_reconstruction.cpp |
Validates size through DOFAccess. |
src/pcms/field/function_space/mfem.hpp |
Validates wrapped data through DOFAccess. |
src/pcms/field/function_space/lagrange.cpp |
Updates wrapped-data validation. |
src/pcms/field/field.h |
Routes access through optional capability and host staging. |
src/pcms/field/field_data.h |
Defines optional DOFAccess capability. |
src/pcms/field/data/xgc.h |
Implements direct access and replica broadcasts. |
src/pcms/field/data/simple.h |
Implements device-backed direct access. |
src/pcms/field/data/point_cloud.h |
Declares direct-access interface. |
src/pcms/field/data/point_cloud.cpp |
Removes backend host staging. |
src/pcms/field/data/mfem.hpp |
Declares direct access and internal gather/scatter helpers. |
src/pcms/field/data/mfem.cpp |
Routes direct access through gather/scatter. |
src/pcms/field/data/mesh_fields.h |
Implements direct access with backend synchronization. |
src/pcms/coupler/serializer/xgc.h |
Removes specialized XGC serializer. |
src/pcms/coupler/overlap_mask.h |
Evaluates callback masks against layouts. |
src/pcms/coupler/field_serializer.h |
Makes serialization non-polymorphic. |
src/pcms/coupler/field_layout_communicator.cpp |
Initializes unmapped entries on nonparticipating ranks. |
src/pcms/coupler/field_communicator.hpp |
Stores the generic serializer directly. |
src/pcms/coupler/coupler.hpp |
Requires direct access for exchanged fields. |
src/pcms/coupler/CMakeLists.txt |
Removes deleted serializer header registration. |
src/pcms/capi/client.h |
Declares separate overlap registration. |
src/pcms/capi/client.cpp |
Implements overlap registration and revised XGC setup. |
.gitattributes |
Marks SWIG outputs as generated. |
Review effort: Balanced
Comment on lines
+145
to
+154
| // Stage in a device view of the default layout so the copy to the | ||
| // host handles whatever layout the backend's view carries. | ||
| Kokkos::View<T**, DeviceMemorySpace> staged( | ||
| Kokkos::view_alloc(Kokkos::WithoutInitializing, "owned_values_staged"), | ||
| values.extent(0), values.extent(1)); | ||
| CopyDeviceRank2ViewToRank2View(MakeRank2View(staged), values); | ||
| host_values_ = Kokkos::View<T**, HostMemorySpace>( | ||
| Kokkos::view_alloc(Kokkos::WithoutInitializing, "owned_values_host"), | ||
| values.extent(0), values.extent(1)); | ||
| DeepCopyMismatchLayouts(host_values_, staged); |
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The goal of this PR is to centralize the handling of overlaps and allow us to distinguish between fields that PCMS can access the underlying data to serialize such as fields it owns, or fields that it wraps underlying data like MFEM and XGC to those where it can evaluate like upcoming work on fusion-io/nimrod.
@Fuad-HH with the changes to XGC, it's my understanding that you are now no longer using the CAPI, but this does replace the CAPI w.r.t. how you pass the overlap arrays in.