Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid array allocation for moving to equal domain #25896

Merged
merged 4 commits into from
Sep 10, 2024

Conversation

mppf
Copy link
Member

@mppf mppf commented Sep 6, 2024

Resolves #25741

chpl__coerceMove(_array) already has the ability to omit an array copy when the source and destination arrays refer to the same domain.

However, when the domains have equal value but are separate instances, chpl__coerceMove would allocate a separate array and then move elements into it from the original array.

This leads to unnecessary memory usage in some cases; for example, a recursive function that returns an array and so declares the return type.

This commit adds a mechanism for array implementations to provide a way to steal the data buffers from another array which can be used in this scenario. In some ways this is similar to doiOptimizedSwap. Note that doiOptimizedSwap cannot be used here because we want to avoid allocation for one of the arrays.

Future Work:

  • add an implementation of doiBuildArrayMoving for more array types (most notably, Block, Cyclic, and Stencil arrays)
  • figure out how to get this optimization working for arrays-of-arrays. Currently, they are disabled, because something is going wrong with correctly setting up the domain for the nested arrays.

Reviewed by @benharsh - thanks!

  • primers pass with valgrind
  • full comm=none testing
  • full comm=gasnet oversubscribed testing

chpl__coerceMove(_array) already has the ability to omit an
array copy when the source and destination arrays refer to the
same domain.

However, when the domains have equal value but are separate instances,
chpl__coerceMove would allocate a separate array and then move elements
into it from the original array.

This leads to unnecessary memory usage in some cases; for example, a
recursive function that returns an array and so declares the return
type.

This commit adds a mechanism for array implementations to provide a way
to steal the data buffers from another array which can be used in this
scenario. In some ways this is similar to 'doiOptimizedSwap'. Note that
'doiOptimizedSwap' cannot be used here because we want to avoid
allocation for one of the arrays.

---
Signed-off-by: Michael Ferguson <[email protected]>
---
Signed-off-by: Michael Ferguson <[email protected]>
---
Signed-off-by: Michael Ferguson <[email protected]>
@mppf mppf merged commit eb45d74 into chapel-lang:main Sep 10, 2024
7 checks passed
@mppf mppf deleted the avoid-allocate-same-domain-value branch September 10, 2024 18:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: surprising memory usage for returning an array with declared return type
2 participants