Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update berlin #119

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Update berlin #119

wants to merge 4 commits into from

Conversation

knutator2
Copy link
Member

adds the following features to the Berlin Spider

  • updates the URL
  • scrapes data of the students for all schools
  • scrapes data of the staff for all schools

Unfortunately scraping this data takes a lot of time. The scraper runs now for approximately 6 hours on scrapinghub but at least the scraping url should be changed.

@knutator2 knutator2 requested a review from k-nut April 19, 2022 11:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@k-nut k-nut Awaiting requested review from k-nut

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@knutator2