Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add trending petition journals #561

Closed
wants to merge 4 commits into from
Closed

Conversation

cih
Copy link
Contributor

@cih cih commented Mar 18, 2017

This change adds trending petition journals for recording signature
counts at hourly intervals and returns them from the petition show
API endpoint as suggested in #458.

@pixeltrix
Copy link
Contributor

@cih you can rebase against master and force push to rebuild this and it should work

@cih cih force-pushed the trending-journals branch 2 times, most recently from a5a144f to 11edfa0 Compare April 1, 2017 10:53
@dom96
Copy link

dom96 commented Apr 7, 2019

Why is this stalled?

@pixeltrix
Copy link
Contributor

@dom96 because the journal tables have always been a pain when we get a massive spike in demand like we saw with the Revoke Article 50 petition and the thought of adding more journals fills me with dread at the thought of having to resync them all. Not closed to the idea of providing this data just looking to see if there's a way we can make PostgreSQL do the heavy lifting.

@pixeltrix pixeltrix closed this Apr 19, 2019
@CodePint
Copy link

CodePint commented Sep 28, 2020

Heya, just thought I would share the project i'm working on to add full tracking to petitions by geography/locale.

https://github.com/CodePint/PetitionTrackerUK

Bulk of the work is done, but without optimisations you are correct that regular polling adds a large amount of DB load, I am working on a solution to this currently.

Likely solution will be:

  • Base scenario of only creating a new record every hour for all petitions above 500 signatures
  • Maintain a table of trending petitions which will be polled more regularly
  • Some form of dataset reduction and archival after a month.

Ive got the bulk of the work done and a react frontend with chart.js hooked up. Tried only updating a locale if new signatures we're present in an effort to reduce storage, but unfortunately this just shifts the problem to long query times.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants