Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remote filesystem scanning improvements #430

Merged
merged 5 commits into from
Aug 15, 2023
Merged

Remote filesystem scanning improvements #430

merged 5 commits into from
Aug 15, 2023

Conversation

ml-evs
Copy link
Member

@ml-evs ml-evs commented Aug 10, 2023

This PR adds a series of fixes for remote filesystems, in particular making them more robust to expected failures, e.g., host key changes or folders being removed.

Previously, an JSON unhandled error regarding a single remote would break the entire /list-remote-directories response, but now the error is nested within that remote (closes #428).

It adds an admin task invoke admin.check-remotes that can check the sync status of remotes individually using a new endpoint for listing a single remote (could be used in the future by the app for selective syncing).

@ml-evs ml-evs changed the title Remote filesystem tweaks Remote filesystem scanning fixes Aug 10, 2023
@cypress
Copy link

cypress bot commented Aug 10, 2023

Passing run #408 ↗︎

0 44 0 0 Flakiness 0

Details:

Merge 78c8386 into 5179311...
Project: datalab Commit: 6a695d02d9 ℹ️
Status: Passed Duration: 04:03 💡
Started: Aug 15, 2023 2:16 PM Ended: Aug 15, 2023 2:20 PM

This comment has been generated by cypress-bot as a result of this project's GitHub integration settings.

@ml-evs ml-evs force-pushed the ml-evs/fix-remotes branch 7 times, most recently from 1156919 to c17d734 Compare August 14, 2023 12:30
…emote endpoint

- Return remote filesystem error at top level

- Clearer remote error messages

- Improve remote sync error handling

- Upgrade failed sync alert in UI to an error
@ml-evs ml-evs marked this pull request as ready for review August 15, 2023 14:41
@ml-evs ml-evs changed the title Remote filesystem scanning fixes Remote filesystem scanning improvements Aug 15, 2023
@ml-evs ml-evs added server refactoring For issues/PRs that refactor existing code/features usability labels Aug 15, 2023
@ml-evs ml-evs merged commit 8aad9e1 into main Aug 15, 2023
4 checks passed
@ml-evs ml-evs deleted the ml-evs/fix-remotes branch August 15, 2023 14:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
refactoring For issues/PRs that refactor existing code/features server usability
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Return more instructive error message when cache invalidation fails for list-remote-directories
1 participant