Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add retries as option when working with data transfers #919

Draft
wants to merge 7 commits into
base: devel
Choose a base branch
from

Conversation

JoshuaSBrown
Copy link
Collaborator

@JoshuaSBrown JoshuaSBrown commented Nov 3, 2023

Description

In the face of network faults, DataFed is canceling tasks. This needs to be controlled with an option. Instead, the default Globus behavior should be used, if a user specifically wants to cancel a transfer after a determined number of faults that is an option they can specify. This is particularly important when first testing a repository and attempting to detect network errors, but it is also useful to cancel tasks early in cases where there is high network traffic and an alternative transfer request to a different Globus endpoint might be preferable.

Tasks

  • - Upgrade version
  • - Changelog comment
  • - Add tests
  • - Feature description
  • - Working tests in ci
  • - Formatting passes
  • - Documentation added
  • - Add labels
  • - Assign work
  • - Assign expected time

Work time

Expected time: 24 hours

@JoshuaSBrown JoshuaSBrown self-assigned this Nov 3, 2023
@JoshuaSBrown JoshuaSBrown marked this pull request as draft November 3, 2023 02:31
@JoshuaSBrown
Copy link
Collaborator Author

Still need to address the faults logic in the GlobusAPI.cpp file and update the documentation. Also need to update the web server to support a retries parameter.

@JoshuaSBrown JoshuaSBrown added Priority: High Highest priority Component: Database Relates to database microservice / data model Component: Core Relates to core service Component: Python API Relates to Python API Type: Refactor Imlplementation change, same functionality Type: Test Related to unit or integration testing labels Nov 7, 2023
@JoshuaSBrown
Copy link
Collaborator Author

@par-hermes format

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: Core Relates to core service Component: Database Relates to database microservice / data model Component: Python API Relates to Python API Priority: High Highest priority Type: Refactor Imlplementation change, same functionality Type: Test Related to unit or integration testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant