Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow different hash table size for dim/var/attr #132

Merged
merged 4 commits into from
Mar 18, 2024
Merged

Conversation

wkliao
Copy link
Member

@wkliao wkliao commented Mar 16, 2024

As the numbers of variables, dimensions, and attributes can be different significantly,
using the same hash table size will not run efficiently and can waste memory allocated.
This PR allows different hash table sizes, which can also be changed by users through
PnetCDF hints.

4 new PnetCDF user hints have been added.

  • nc_hash_size_dim: Set hash table size for dimension names
  • nc_hash_size_var: Set hash table size for variable names
  • nc_hash_size_gattr: Set hash table size for global attribute names
  • nc_hash_size_vattr: Set hash table size for variable attribute names

3 new test programs have beed added to test hashing performance.

  • test/largefile/tst_hash_large_ndims.c
  • test/largefile/tst_hash_large_nvars.c
  • test/largefile/tst_hash_large_ngattr.c

…_hash_size_vattr

Add 4 new PnetCDF hints.
nc_hash_size_dim:   Set hash table size for dimension names
nc_hash_size_var:   Set hash table size for variable names
nc_hash_size_gattr: Set hash table size for global attribute names
nc_hash_size_vattr: Set hash table size for variable attribute names
* Add constant PNC_HSIZE_DIM, the default hash table size for
  dimensions.
* Add constant PNC_HSIZE_VAR, the default hash table size for
  variables.
* Add constant PNC_HSIZE_VATTR, the default hash table size for
  attributes per variable.
* Add constant PNC_HSIZE_GATTR, the default hash table size for
  global attributes.
* Add PNC_VATTR_ARRAY_GROWBY, which is used to grow the array of
  attributes per variable. The number of attributes per variable is
  expected to be much smaller than the global attributes.
* Rename some NC_ constant in ncmpio_NC.h to PNC_
@wkliao wkliao merged commit cd2fa63 into master Mar 18, 2024
3 of 4 checks passed
@wkliao wkliao deleted the inc_hash_size branch March 21, 2024 16:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant