Skip to content

A very simple shell script for scraping foxnews in json

License

Notifications You must be signed in to change notification settings

TUVIMEN/foxnews-scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 

Repository files navigation

foxnews-scraper

A very simple shell script for scraping foxnews in json.

Requirements

  • reliq
  • jq
  • parallel (not mandatory)

Installation

install -m 755 foxnews-scraper /usr/bin

Json format

Here's json.

Usage

Script downloads all articles using sitemap.xml and writes it into stdout.

If you want to download it in parallel just uncomment the 3 comments and comment the for loop.

About

A very simple shell script for scraping foxnews in json

Topics

Resources

License

Stars

Watchers

Forks

Languages