Skip to content

Latest commit

 

History

History
4 lines (4 loc) · 466 Bytes

Readme.md

File metadata and controls

4 lines (4 loc) · 466 Bytes
Raw

Web Crawler for a "hidden API" website

  • This crawler scrapes the "All products" page of the website on-running.com
  • The crawling is a lot of different as the website works on sending requests to hidden APIs and hence normal web crawling techniques won't work here.
  • Instead we'll have to directly access those APIs that are being called when we do something like reload a page or press the "Show more" button on a page.