You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Since we started all fetching calendar values while scraping court calls, the scrape has slow down to the point where we're unable to scrape all available court calls in under 6 hours.
We could try a some things to make the scrapes more efficient:
Avoiding duplicate calendar requests-- on the results page, there are usually at least two court calls listed for a single case. Caching calendar values should reduce the number of case detail requests by at least half.
If (1) isn't enough, we could also limit the dates we're scraping every day. We could try only scraping the court calls for the current or next day.
The text was updated successfully, but these errors were encountered:
Since we started all fetching calendar values while scraping court calls, the scrape has slow down to the point where we're unable to scrape all available court calls in under 6 hours.
We could try a some things to make the scrapes more efficient:
The text was updated successfully, but these errors were encountered: