Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementation of the mpio driver with selection I/O. #3222

Merged
merged 30 commits into from
Aug 7, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
25c434e
Implementation of the mpio driver with selection I/O.
vchoi-hdfgroup Jul 3, 2023
45e0709
Committing clang-format changes
github-actions[bot] Jul 3, 2023
a8c0351
Add comments for the changes. Minor modifications to the mpio driver …
vchoi-hdfgroup Jul 6, 2023
c3fee29
Committing clang-format changes
github-actions[bot] Jul 6, 2023
f75cbd3
Correct spelling error.
vchoi-hdfgroup Jul 6, 2023
db37097
Merge branch 'HDFGroup:develop' into new_mpio_develop
vchoi-hdfgroup Jul 11, 2023
d1d5abf
Merge branch 'HDFGroup:develop' into new_mpio_develop
vchoi-hdfgroup Jul 14, 2023
b6cc695
Merge branch 'HDFGroup:develop' into new_mpio_develop
vchoi-hdfgroup Jul 17, 2023
a39f805
Merge branch 'HDFGroup:develop' into new_mpio_develop
vchoi-hdfgroup Jul 19, 2023
56e0ca3
Add tests for the three bugs found when setting H5D_SELECTION_IO_MODE…
vchoi-hdfgroup Jul 19, 2023
41f68d4
Committing clang-format changes
github-actions[bot] Jul 19, 2023
eddb315
Correct spelling error.
vchoi-hdfgroup Jul 19, 2023
6604ad4
Merge branch 'HDFGroup:develop' into new_mpio_develop
vchoi-hdfgroup Jul 24, 2023
accb22f
Merge branch 'HDFGroup:develop' into new_mpio_develop
vchoi-hdfgroup Jul 26, 2023
8b080ec
Merge branch 'HDFGroup:develop' into new_mpio_develop
vchoi-hdfgroup Jul 27, 2023
5f20722
Add tests for selection I/O which are derived from test/vfd.c and mod…
vchoi-hdfgroup Jul 27, 2023
efbe96c
Committing clang-format changes
github-actions[bot] Jul 27, 2023
40c3d64
Correct spelling errors.
vchoi-hdfgroup Jul 27, 2023
83ab87d
Committing clang-format changes
github-actions[bot] Jul 27, 2023
ab7746c
Merge branch 'HDFGroup:develop' into new_mpio_develop
vchoi-hdfgroup Aug 1, 2023
eeb29b2
Merge branch 'develop' into new_mpio_develop
vchoi-hdfgroup Aug 3, 2023
1fadf80
Committing clang-format changes
github-actions[bot] Aug 3, 2023
340cf6b
Merge branch 'HDFGroup:develop' into new_mpio_develop
vchoi-hdfgroup Aug 3, 2023
a1fd1f4
Fix CI failures due ot ";" after macros.
vchoi-hdfgroup Aug 3, 2023
de6a7fd
Committing clang-format changes
github-actions[bot] Aug 3, 2023
8e42490
This changes the default selection I/O to on for MPIO.
vchoi-hdfgroup Aug 4, 2023
dc886ee
Merge branch 'HDFGroup:develop' into new_mpio_develop
vchoi-hdfgroup Aug 5, 2023
d61facc
Fixes for the two issues in PR review:
vchoi-hdfgroup Aug 5, 2023
2621bbb
Committing clang-format changes
github-actions[bot] Aug 5, 2023
3bf430f
Address spelling/grammar errors from PR review.
vchoi-hdfgroup Aug 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 16 additions & 14 deletions src/H5Dio.c
Original file line number Diff line number Diff line change
Expand Up @@ -415,8 +415,14 @@ H5D__read(size_t count, H5D_dset_io_info_t *dset_info)
HGOTO_ERROR(H5E_DATASET, H5E_CANTGET, FAIL, "can't get MPI-I/O transfer mode");

/* Only report the collective I/O mode if we're actually performing collective I/O */
if (xfer_mode == H5FD_MPIO_COLLECTIVE)
if (xfer_mode == H5FD_MPIO_COLLECTIVE) {
H5CX_set_mpio_actual_io_mode(io_info.actual_io_mode);

/* If we did selection I/O, report that we used "link chunk" mode, since that's the most
* analogous to what selection I/O does */
if (io_info.use_select_io == H5D_SELECTION_IO_MODE_ON)
H5CX_set_mpio_actual_chunk_opt(H5D_MPIO_LINK_CHUNK);
}
}
#endif /* H5_HAVE_PARALLEL */
}
Expand Down Expand Up @@ -1131,20 +1137,16 @@ H5D__typeinfo_init_phase2(H5D_io_info_t *io_info)
assert(io_info);

/* If selection I/O mode is default (auto), enable it here if the VFD supports it (it will be turned off
* later if something else conflicts), otherwise disable it. If we're using the MPIO VFD, the automatic
* selection will happen in H5D__mpio_opt_possible() inside H5D__ioinfo_adjust(). */
#ifdef H5_HAVE_PARALLEL
if (!io_info->using_mpi_vfd)
#endif /* H5_HAVE_PARALLEL */
if (io_info->use_select_io == H5D_SELECTION_IO_MODE_DEFAULT) {
if (H5F_has_vector_select_io(io_info->dsets_info[0].dset->oloc.file,
io_info->op_type == H5D_IO_OP_WRITE))
io_info->use_select_io = H5D_SELECTION_IO_MODE_ON;
else {
io_info->use_select_io = H5D_SELECTION_IO_MODE_OFF;
io_info->no_selection_io_cause |= H5D_SEL_IO_DEFAULT_OFF;
}
* later if something else conflicts), otherwise disable it */
if (io_info->use_select_io == H5D_SELECTION_IO_MODE_DEFAULT) {
if (H5F_has_vector_select_io(io_info->dsets_info[0].dset->oloc.file,
io_info->op_type == H5D_IO_OP_WRITE))
io_info->use_select_io = H5D_SELECTION_IO_MODE_ON;
else {
io_info->use_select_io = H5D_SELECTION_IO_MODE_OFF;
io_info->no_selection_io_cause |= H5D_SEL_IO_DEFAULT_OFF;
}
}

/* If we're doing type conversion and we might be doing selection I/O, check if the buffers are large
* enough to handle the whole I/O */
Expand Down
22 changes: 3 additions & 19 deletions src/H5Dmpio.c
Original file line number Diff line number Diff line change
Expand Up @@ -622,6 +622,9 @@ H5D__mpio_opt_possible(H5D_io_info_t *io_info)
if (!H5FD_mpi_opt_types_g)
local_cause[0] |= H5D_MPIO_MPI_OPT_TYPES_ENV_VAR_DISABLED;

/* Decision on whether to use selection I/O should have been made by now */
assert(io_info->use_select_io != H5D_SELECTION_IO_MODE_DEFAULT);

/* Datatype conversions and transformations are allowed with selection I/O. If the selection I/O mode
* is auto (default), disable collective for now and re-enable later if we can */
if (io_info->use_select_io != H5D_SELECTION_IO_MODE_ON) {
Expand Down Expand Up @@ -731,25 +734,6 @@ H5D__mpio_opt_possible(H5D_io_info_t *io_info)
HMPI_GOTO_ERROR(FAIL, "MPI_Allreduce failed", mpi_code)
} /* end else */

/* If the selection I/O mode is default (auto), decide here whether it should be on or off */
if (io_info->use_select_io == H5D_SELECTION_IO_MODE_DEFAULT) {
/* If the only reason(s) we've disabled collective are type conversions and/or transforms, enable
* selection I/O and re-enable collective I/O since it's supported by selection I/O */
if (global_cause[0] && !(global_cause[0] & ~((unsigned)H5D_MPIO_DATATYPE_CONVERSION |
(unsigned)H5D_MPIO_DATA_TRANSFORMS))) {
assert(!(local_cause[0] &
~((unsigned)H5D_MPIO_DATATYPE_CONVERSION | (unsigned)H5D_MPIO_DATA_TRANSFORMS)));
local_cause[0] = 0;
global_cause[0] = 0;
io_info->use_select_io = H5D_SELECTION_IO_MODE_ON;
}
else {
/* Otherwise, there's currently no benefit to selection I/O, so leave it off */
io_info->use_select_io = H5D_SELECTION_IO_MODE_OFF;
io_info->no_selection_io_cause |= H5D_SEL_IO_DEFAULT_OFF;
}
}

/* Set the local & global values of no-collective-cause in the API context */
H5CX_set_mpio_local_no_coll_cause(local_cause[0]);
H5CX_set_mpio_global_no_coll_cause(global_cause[0]);
Expand Down
Loading