Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[on hold] Support for sub arrays of dynamic size (jagged arrays) #305

Draft
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

bernhardmgruber
Copy link
Member

@bernhardmgruber bernhardmgruber commented Jun 23, 2021

This PR adds support for fields of dynamic array type. Specifically:

  • allow dynamic field types in the record dimension
  • add specializations to most of the core functions
  • add llama::dynamic to signal a dynamic array member in a RecordCoord
  • extend VirtualRecord to allow holding dynamic indices
  • extend blobNrAndOffset to allow for additional dynamic indices
  • add OffsetTable mapping
  • add customization allowing to dump OffsetTable mappings
  • add a few unit tests

@bernhardmgruber bernhardmgruber force-pushed the jagged_array branch 12 times, most recently from cc4b647 to dd45ecc Compare June 29, 2021 13:35
@bernhardmgruber bernhardmgruber force-pushed the jagged_array branch 6 times, most recently from bf0f7fc to 641b014 Compare July 7, 2021 15:39
@bernhardmgruber
Copy link
Member Author

Given a small HEP style data structure:

    struct run {};
    struct luminosityBlock {};
    struct Electrons {};
    struct Muons {};
    struct Eta{};
    struct Mass{};
    struct Phi{};

    using Electron = llama::Record<
        llama::Field<Eta, float>,
        llama::Field<Mass, float>,
        llama::Field<Phi, float>
    >;
    using Muon = llama::Record<
        llama::Field<Eta, float>,
        llama::Field<Mass, float>,
        llama::Field<Phi, float>
    >;
    using Event = llama::Record<
        llama::Field<run, std::int32_t>,
        llama::Field<luminosityBlock, std::int32_t>,
        llama::Field<Electrons, Electron[]>,
        llama::Field<Muons, Muon[]>
    >;
....
    auto mapping = llama::mapping::OffsetTable<llama::ArrayDims<1>, Event>{
        llama::ArrayDims{30},
        llama::ArrayDims{50},
        llama::ArrayDims{40}};

Here is a dump of the mapping using AoS for the sub mappings:
image
Using SoA for the sub mappings:
image
Using Aos for the events and SoA for the electrons and muons:
image

* allow dynamic field types in the record dimension
* add specializations to most of the core functions
* add llama::dynamic to signal a dynamic array member in a RecordCoord
* extend VirtualRecord to allow holding dynamic indices
* extend blobNrAndOffset to allow for additional dynamic indices
* add OffsetTable mapping
* add customization allowing to dump OffsetTable mappings
* add a few unit tests
@bernhardmgruber bernhardmgruber changed the title Support for sub arrays of dynamic size Support for sub arrays of dynamic size (jagged arrays) Mar 16, 2023
@bernhardmgruber bernhardmgruber changed the title Support for sub arrays of dynamic size (jagged arrays) [on hold] Support for sub arrays of dynamic size (jagged arrays) Sep 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant