Improve Monitoring Tool UX related to annotations #601
Conversation
aristizabal95
had a problem deploying
to
testing-external-code
GitHub Actions
Failure
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
aristizabal95
had a problem deploying
to
testing-external-code
GitHub Actions
Failure
aristizabal95
had a problem deploying
to
testing-external-code
GitHub Actions
Failure
| def brain_has_been_reviewed(brainpath, backup_brainpath): | ||
| if not os.path.exists(backup_brainpath): | ||
| return False | ||
|
|
||
| brain_hash = get_hash(brainpath) | ||
| backup_hash = get_hash(backup_brainpath) | ||
| return brain_hash != backup_hash |
There was a problem hiding this comment.
To tell truth I don't understand the whole pipeline logics quite well. Can you explain plz why difference between brain_hash and backup_hash means anything was reviewed? For me it means only that the content was modified comparing to the backup...
There was a problem hiding this comment.
The brain mask will only change if someone opened the file and changed the contents, so this is why this check works to identify if the brain mask has been reviewed. We don't care about brain masks that have been opened but not modified.
As for tumor segmentations, there are situations in which someone might review the file but leave it unchanged, and we do need to identify those cases to be able to move forward. That's why for tumor segmentations we instead check the existence of that segmentation in a "finalized" folder. That file will only be there if a user puts it there (either manually or by using the monitoring tool "set as finalized" button).
|
Closing for a bigger PR with more UX improvements |
This PR adds a few QOL improvements to the annotation procedure. It does so by:
These have been points of confusion for several users, thinking their changes have not been taken into account because the monitoring tool doesn't update until the pipeline is running. This PR should improve that experience.