-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve Monitoring Tool UX related to annotations #601
Improve Monitoring Tool UX related to annotations #601
Conversation
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
def brain_has_been_reviewed(brainpath, backup_brainpath): | ||
if not os.path.exists(backup_brainpath): | ||
return False | ||
|
||
brain_hash = get_hash(brainpath) | ||
backup_hash = get_hash(backup_brainpath) | ||
return brain_hash != backup_hash |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To tell truth I don't understand the whole pipeline logics quite well. Can you explain plz why difference between brain_hash and backup_hash means anything was reviewed? For me it means only that the content was modified comparing to the backup...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The brain mask will only change if someone opened the file and changed the contents, so this is why this check works to identify if the brain mask has been reviewed. We don't care about brain masks that have been opened but not modified.
As for tumor segmentations, there are situations in which someone might review the file but leave it unchanged, and we do need to identify those cases to be able to move forward. That's why for tumor segmentations we instead check the existence of that segmentation in a "finalized" folder. That file will only be there if a user puts it there (either manually or by using the monitoring tool "set as finalized" button).
Closing for a bigger PR with more UX improvements |
This PR adds a few QOL improvements to the annotation procedure. It does so by:
These have been points of confusion for several users, thinking their changes have not been taken into account because the monitoring tool doesn't update until the pipeline is running. This PR should improve that experience.