Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 Source zendesk-support: make ticket_metrics stream stop requesting data after start_date #48427

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

willbowditch
Copy link

@willbowditch willbowditch commented Nov 8, 2024

What

The ticket_metrics stream fetches data from the API from most recent to least recent. The state of the most recent record is kept as a cursor for the next stream.

If a start_date is set in the config, the stream should stop fetching data once we have reached that page, however it currently keeps importing data until no more pages are remaining. (The data is queried from the API but not yielded as a record).

The PR stores the oldest seen record and stops fetching data if that is older than the start date.

Closes #48424

How

  • Keep track of the oldest seen record
  • If the oldest seen record is older than the start_date, don't request the next page

Review guide

  • Small change to zendesk-support streams logic

User Impact

  • In cases where the start date has been set the initial stream will complete faster.

Can this PR be safely reverted and rolled back?

  • YES 💚
  • NO ❌

Copy link

vercel bot commented Nov 8, 2024

@willbowditch is attempting to deploy a commit to the Airbyte Growth Team on Vercel.

A member of the Team first needs to authorize it.

@CLAassistant
Copy link

CLAassistant commented Nov 8, 2024

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[zendesk-support] ticket_metrics stream doesn't respect start_date set at source config
3 participants