Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add additional fields to published projects CSV download #2280

Merged
merged 4 commits into from
Aug 23, 2024

Conversation

briangow
Copy link
Contributor

This PR adds fields to indicate the resource/project type to the published projects CSV download available in the console. It also addresses these suggested updates for projects from #2277 :

  • Rename to "Published Projects" (renamed the button, everything else seemed to have both words already)
  • Add approximate storage size, if possible (storage_volume) (added main_storage_size which is what the entire project takes up across versions. Could add incremental_storage_volume also if desired).
  • Add number of signed DUAs (signed_dua_count) Add this by counting the DUASignature objects for a given project_id.

@tompollard
Copy link
Member

Looks good, thanks. Please could you convert bytes to something easier to work with (perhaps mb or gb?).

I'm not quite sure what the size is telling us right now, so we need to look into this (part of reason I hadn't implemented).

Is it the storage size allocated to the project? If so, I'm not sure whether or not this is an accurate reflection of storage?

@bemoody
Copy link
Collaborator

bemoody commented Aug 23, 2024

main_storage_size = total size of this version's file content in bytes. The number of bytes that you will download - not counting directories or metadata - if you download the project using wget.

This is calculated after the project is published. It's not equal to physical disk usage, nor is the sum of main_storage_size for all versions equal to the total quota usage. And of course, the value might be wrong if the project was published in a non-standard way. But this is the most relevant number for users. If the number is wrong, it should be fixed.

@briangow
Copy link
Contributor Author

@tompollard , I'm converting the storage size to MB now. It sounds like we want to continue reporting main_storage_size.

@tompollard
Copy link
Member

main_storage_size = total size of this version's file content in bytes. The number of bytes that you will download - not counting directories or metadata - if you download the project using wget.

Nice, thanks Benjamin.

And of course, the value might be wrong if the project was published in a non-standard way. But this is the most relevant number for users. If the number is wrong, it should be fixed.

Does "non-standard way" refer just to projects that are symlinked to a non-standard location? I think this is one or two MIMIC projects (ECG, CXR?).

@tompollard
Copy link
Member

@briangow sorry, this is just a personal thing really, but please could you change “storage_size(MB)” to “storage_size_mb” for consistency with other headers?

@tompollard
Copy link
Member

@briangow sorry, also, probably want to round to the nearest whole number?

@briangow
Copy link
Contributor Author

@tompollard , we don't anticipate needing storage size information on projects < 0.5 MB for any reason? If we might then perhaps I should include a couple of decimal places.

@tompollard
Copy link
Member

@tompollard , we don't anticipate needing storage size information on projects < 0.5 MB for any reason? If we might then perhaps I should include a couple of decimal places.

Perhaps let's go for one decimal place? I can't see us ever wanting more fine-grained than that.

@briangow
Copy link
Contributor Author

@tompollard , I've made these updates.

@tompollard
Copy link
Member

Thanks Brian, looks good to me!

@tompollard tompollard merged commit d183b9a into dev Aug 23, 2024
8 checks passed
@tompollard tompollard deleted the bg_add_project_type_to_download_csv branch August 23, 2024 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants