Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

taxon/tab embeds: support fast & accurate updates #141

Open
synrg opened this issue Jan 22, 2021 · 1 comment
Open

taxon/tab embeds: support fast & accurate updates #141

synrg opened this issue Jan 22, 2021 · 1 comment
Assignees
Labels
obs iNaturalist observations place iNaturalist places taxon iNaturalist taxa user iNaturalist / Discord users

Comments

@synrg
Copy link
Collaborator

synrg commented Jan 22, 2021

The idea is to no longer tear apart these embeds with regexes on a freshly fetched copy of the embed from the channel each time they are updated, but rather to:

  • Parse the Discord embed only at the very beginning of interacting with the embed into an INatEmbed object.
    • This is a recent work that started in 218a899 following commits.
    • The result will be cached for a while so that several people adding their counts in close succession will see reliable and fast updates.
  • Provide abstractions for each iNat object component of the embed (taxon, places & users listed on the embed, etc.) and the embed's current state (e.g. taxonomy shown/hidden).
    • As of writing this issue, only the taxonomy shown/hidden abstraction remains to be coded
  • Provide a bare-bones version of the embed that can be efficiently cached (i.e. only a dict representing the embed needs to be cached; the actual taxon/place/user records themselves are cached separately).
  • Provide a means to reconstruct the whole embed efficiently from the cached object.
    • The "efficiently" part is somewhat complicated if the embed has aged long enough so that users have updated relevant records in iNat since the embed was first created.
      • The current method preserves the original counts associated with the listed places / users and only updates the totals, as to do otherwise would require re-fetching records that were already recently fetched.
      • To do the same thing now, we will also need to parse & include those counts in the dict, which we don't do yet.
    • This is expected to solve image: taxonomy inserted in wrong spot when counts present #139 since the whole embed will be recreated from scratch instead of relying on finding cut and insertion points in the description to remove / re-add updated material.
  • Rewrite the current update code to:
    • Lock & update only the cached, already parsed copy of the embed, avoiding the round-trip out to Discord which is ultimately the cause of the race condition bug in taxon reactions: Simultaneous reactions add both user & place #101 .
    • As before, with the old regex/insertion method, we will only want to do API calls for new material added (i.e. the new user & updated totals) or removed (recalculate totals without the removed user).
@synrg
Copy link
Collaborator Author

synrg commented Aug 28, 2021

This is going to take a while, and some iNat API endpoints are known to return wrong results when given user_id=login1,login2,... as a parameter, throwing the total count of observations off, so we need a mid-term workaround for that instead of continuing to aggravate users with obviously wrong totals:

1108972 still uses the old approach but takes the list of user ids for the total from the user_id=# parameters in each listed URL instead of from each listed login in the table.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
obs iNaturalist observations place iNaturalist places taxon iNaturalist taxa user iNaturalist / Discord users
Projects
None yet
Development

No branches or pull requests

1 participant