Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENG-6283] User table allows CSV and TSV downloads #10782

Open
wants to merge 6 commits into
base: feature/insti-dash-improv
Choose a base branch
from

Conversation

Johnetordoff
Copy link
Contributor

@Johnetordoff Johnetordoff commented Oct 18, 2024

Purpose

Allows CSV and TSVs be downloaded from the user tab on the institutional dashboard page by adding a format query param. Such as in v2/institutions/cos/metrics/users/?format=csvor v2/institutions/cos/metrics/users/?format=tsv or v2/institutions/cos/metrics/users/?format=json_file

Changes

  • adds CSV/TSV renderer to view
  • add .extra(size= to ensure spread sheets cover whole report, not one page.
  • adds Json renderer to metrics reporter
  • adds tests

QA Notes

Please make verification statements inspired by your code and what your code touches.

  • Verify
  • Verify

What are the areas of risk?

Any concerns/considerations/questions that development raised?

Documentation

Side Effects

Ticket

https://openscience.atlassian.net/browse/ENG-6283

@Johnetordoff Johnetordoff changed the base branch from develop to feature/insti-dash-improv October 18, 2024 19:52
@Johnetordoff Johnetordoff force-pushed the user-table-downloads branch 3 times, most recently from 7ceda95 to 5deac58 Compare October 21, 2024 14:46
@Johnetordoff Johnetordoff changed the title [ENG-6283] [WIP] User table allows CSV and TSV downloads [ENG-6283] User table allows CSV and TSV downloads Oct 21, 2024
@Johnetordoff Johnetordoff marked this pull request as ready for review October 21, 2024 16:20
Copy link
Contributor

@aaxelb aaxelb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, yay for code reuse!

my only request would be to add a Content-Disposition header that gives the downloaded file a sensical filename


class MetricsReportsJsonRenderer(MetricsReportsBaseRenderer):
media_type = 'application/json'
format = 'json_file'
Copy link
Contributor Author

@Johnetordoff Johnetordoff Oct 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe change format param to fix collision with normal json format?

@Johnetordoff Johnetordoff force-pushed the user-table-downloads branch 4 times, most recently from fe52930 to 7e82b04 Compare October 24, 2024 12:53
@Johnetordoff
Copy link
Contributor Author

@aaxelb There was an issue where the pagination was limiting the files to 10 items, the default page size, so I had to address that between reviews, just heads up if you are wondering about the diff.

api/base/settings/defaults.py Outdated Show resolved Hide resolved
"""Render the full dataset as CSV or TSV format."""
view = renderer_context['view']
view.pagination_class = None # Disable pagination
data = view.get_default_search().extra(size=MAX_SIZE_OF_ES_QUERY).execute()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be good to keep a logical separation of concerns -- a renderer should work with the data it's given; this is logic that belongs in the view

(tho might be even better to skip the special casing and let the client set page[size]=10000)

extension = 'tsv'


class MetricsReportsJsonRenderer(MetricsReportsBaseRenderer):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i don't think we need yet another json renderer; we only need a way to download the existing json as a file (instead of opening in the browser)

how about accepting a forDownload query param in ElasticsearchListView (or even JSONAPIBaseView?) that toggles a Content-Disposition header? could work the same way for any format, reducing special-casing

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In considering this I think the user probably doesn't want meta and general pagination information that would come from this endpoint. It should be a pure list, like the tabular formats are. But still considering options here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is that based on user feedback? (not sure where "it should be a pure list" came from, when talking about a non-tabular format like json)

total counts and pagination information seem like helpful info to include (especially if they can choose a page size, or if there are more than 10000), and easy to ignore if they don't need it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(could understand wanting to omit relationships, but i'd argue the effort would be better spent making ElasticsearchListView support sparse fieldsets (so the download could reflect selected columns from the frontend, implicitly omitting relationships))

@@ -320,43 +320,56 @@ class RecentReportList(JSONAPIBaseView):
MetricsReportsTsvRenderer,
)

def get(self, request, *args, report_name):
def get_default_search(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is getting beyond the scope of the ticket, but it'd be nice for RecentReportList to inherit ElasticsearchListView, now that it exists (instead of only duck-typing get_default_search) -- could be fuller-featured and more consistent, but might also rabbit-hole, so don't bother unless you want to

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants