Skip to content

ADIOS2 variable shape LocalValue#1601

Open
franzpoeschel wants to merge 16 commits intoopenPMD:devfrom
franzpoeschel:adios2-local-values
Open

ADIOS2 variable shape LocalValue#1601
franzpoeschel wants to merge 16 commits intoopenPMD:devfrom
franzpoeschel:adios2-local-values

Conversation

@franzpoeschel
Copy link
Copy Markdown
Contributor

@franzpoeschel franzpoeschel commented Mar 11, 2024

Ref https://adios2.readthedocs.io/en/v2.9.2/components/components.html#shapes

This PR considers LocalValues in ADIOS2 merely an implementation detail, without actually exposing an API analogous to to how LocalValues work in ADIOS2.

The reasoning is:

  1. LocalValues are useful not only for their API, but also since they give a hint to ADIOS2 that the variable should be handled as metadata and take part e.g. in metadata aggregation. With this PR, datasets such as the particlePatches in PIConGPU (one single value per MPI rank) can be given to the metadata system without changing a single line of code in PIConGPU. This can be crucial in streaming setups: When each reading rank loads the entire dataset naively (without this setting), the data plane of SST will open a connection between every writer and reader for just a single value each. Better to aggregate.
  2. If needed at some point, a LocalValue-like API can still be exposed on top of this.

TODO

  • Documentation
  • Check usage of readOperators, m_operators

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants