Update copy in README

* Partial work porting over and updating warn-scraper conventions
* Initial pass at porting usage docs biglocalnews#6
* Add cli list command
* Update README
* Fix Makefile help text
* Commit lockfile
* Add stories page
* Add throttle and misc cleanups to cli and runner
* Add dependency to setup.py
* Clobber obsolete SD scraper test module
* Log agency slug in runner
* Add customizable throttling
* Port and update cache
* Remove doc tests and disable Python and other downstream actions in CI (for now)
* Partial work on ca_san_diego_pd
* Fix/update usage docs

* Use pre-commit for make formt

* Rework to use Site class per agency. biglocalnews#3 biglocalnews#4

* Update contributor docs biglocalnews#5
* Update usage docs biglocalnews#6
* update deps
* Update main README biglocalnews#11
* add stub page for maintainer docs biglocalnews#9

tweak

fix README h1

readme fix

Add discussions and issues link in README

…ews#19)

* Implement/update San Diego scrape_meta and scrape methods
* Add cache.write_json and read_json methods
* Add filter option to CLI scrape command
* update reqs

* Add basic test coverage
* Default to empty string for scrape-meta filter arg
* ignore linter code
* linter fixes
* add tox/testing and linting info to maintainers docs
* remove py37 support and bump to Beta dev status
* Add tox config and reqs.txts
* Update Pipfile.lock

* Add CA orange_county_sheriff.py
* add name to contributors
---------

Co-authored-by: jrynning <[email protected]>

* Update contributor docs. Closes biglocalnews#25
* linter fixes

* Sonoma County Scraper (biglocalnews#32) 
* Add ca_sonoma_county_sheriff

---------

Co-authored-by: ochezems <[email protected]>

docs: contrib tweaks; metadata type

* ops: fix runner test

* ops: avoid redundant gha runs on prs

---------

Co-authored-by: Gerald Rich <[email protected]>

* added santa rosa

…alnews#48)

* Added The scraper for Humboldt with successful pre-commit run
* Required Changes done
* removed download page where identical

* docs: metadata spec

* docs: remove refs to scrape

---------

Co-authored-by: Gerald Rich <[email protected]>

* feat: sacramento pd scraper

* fix: isort

* scrape most child pages; todo: get sub-sub pages

* more recursively grab child pages

* inline comments

* fix: fn names, py type

* feat: collect zip & pdfs; todo: handle dupe assets

* chore: ci

* feat: download youtube videos & playlists; remove print stmts

* style: naming

* ops: clean-prefect import clean

* ops: fix runner test (biglocalnews#44)

* ops: fix runner test

* ops: avoid redundant gha runs on prs

---------

Co-authored-by: Gerald Rich <[email protected]>

* ops: current reqs

* naming

* refactor: move around methods

* refactor: add case_num

* Tiny typo fixs

* Ca 43 santa rosa scraper (biglocalnews#45)

* added santa rosa

* Added The scraper for Humboldt with successful pre-commit run (biglocalnews#48)

* Added The scraper for Humboldt with successful pre-commit run
* Required Changes done
* removed download page where identical

* docs: metadata spec (biglocalnews#49)

* docs: metadata spec

* docs: remove refs to scrape

---------

Co-authored-by: Gerald Rich <[email protected]>

* Update contributing.md

* fix: metadata dict types

* fix: import typing_extensions

---------

Co-authored-by: Gerald Rich <[email protected]>
Co-authored-by: Mike Stucka <[email protected]>
Co-authored-by: naumansharifwork <[email protected]>

Co-authored-by: Gerald Rich <[email protected]>

* Force encoding to allow Windows machines to use this

* Update setup.py

---------

Co-authored-by: Gerald Rich <[email protected]>

)

* Show error when state prefix is omitted biglocalnews#55

* Better messaging, upgrade error level

* Patch scrape and scrape-meta both

---------

Co-authored-by: Gerald Rich <[email protected]>

…alnews#66 (biglocalnews#68)

* Add in missing UTF-8 compatibility

* Rework typing

* Exclude general list from JSON write; patch language for function

* refactor: SDPD case num

* copy: case_num >> case_id

* remove type ignore

---------

Co-authored-by: Gerald Rich <[email protected]>

* Implement cache.write_binary biglocalnews#69
---------

Co-authored-by: Gerald Rich <[email protected]>

…t urls

* created scrape and scrape meta for Monterey County District Attorney

* config folder location changed

* removed scrape and changed case_num to case_id and in case we dont find case_id it will have title in case_id instead of none

---------

Co-authored-by: Gerald Rich <[email protected]>

* Patch list_agencies bug biglocalnews#85

* Readability cleanup

* added scrape-meta and scrape for riverside pd

* changes for meta-data file formating

* changed file name

* tiny fix

* removed scrape and changed case_num to case_id

* refactor: deprecate scrape method

* fix: case_number >> case_id

---------

Co-authored-by: Gerald Rich <[email protected]>

* fix: tests bootstrap

* ops: coverage

* add coveragerc

* fix: tox; test py39, 310, 311, 312

---------

Co-authored-by: Gerald Rich <[email protected]>

…iglocalnews#89 (biglocalnews#90)

* Proof of concept, missing Class

* Export out usable case index

* Incremental work consolidating things

* First attempt at class

* closer

* Closer

* Implement logging

* Polish

* Proof of concept, missing Class

* Export out usable case index

* Incremental work consolidating things

* First attempt at class

* Kill notebook version brought back by rebase

* Move ugly details to config file

* Rename ugly detail file

* Linting

* Fix linting

* Proof of concept, missing Class

* Export out usable case index

* Incremental work consolidating things

* First attempt at class

* Kill notebook version brought back by rebase

* Proof of concept, missing Class

* Export out usable case index

* Incremental work consolidating things

* First attempt at class

* Proof of concept, missing Class

* Export out usable case index

* Incremental work consolidating things

* First attempt at class

* Proof of concept, missing Class

* Proof of concept, missing Class

* Export out usable case index

* Incremental work consolidating things

* First attempt at class

* Move ugly details to config file

* Rename ugly detail file

* Linting

* Fix linting

* Build against biglocalnews#69 flag biglocalnews#70

* ...

* Apply suggestions

* Clean up notes

* Proof of concept, missing Class

* Export out usable case index

* Incremental work consolidating things

* First attempt at class

* closer

* Closer

* Implement logging

* Polish

* Proof of concept, missing Class

* Export out usable case index

* Incremental work consolidating things

* First attempt at class

* Kill notebook version brought back by rebase

* Move ugly details to config file

* Rename ugly detail file

* Linting

* Fix linting

* Proof of concept, missing Class

* Export out usable case index

* Incremental work consolidating things

* First attempt at class

* Kill notebook version brought back by rebase

* Proof of concept, missing Class

* Export out usable case index

* Incremental work consolidating things

* First attempt at class

* Proof of concept, missing Class

* Export out usable case index

* Incremental work consolidating things

* First attempt at class

* Proof of concept, missing Class

* Proof of concept, missing Class

* Export out usable case index

* Incremental work consolidating things

* First attempt at class

* Move ugly details to config file

* Rename ugly detail file

* Linting

* Fix linting

* Build against biglocalnews#69 flag biglocalnews#70

* ...

* Apply suggestions

* Clean up notes

* Copypaste around rebase problems

---------

Co-authored-by: Gerald Rich <[email protected]>

* added scrape meta for chula_vista_pd biglocalnews#94

* removed user-agent

* changes done

* Rework URL handling; clean up a little more text

* Linted. Oops.

---------

Co-authored-by: Mike Stucka <[email protected]>

…calnews#113)

* fix: sdpd case_id pagination

* simplify case_id

---------

Co-authored-by: Gerald Rich <[email protected]>

* added scrape meta for fresno pd biglocalnews#114

* file name changed fresno_county_sheriff

* file name changed fresno_county_sheriff

Co-authored-by: Gerald Rich <[email protected]>