Ca 50 created scrape and scrape-meta for laserfichie #52
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Update copy in README
* Partial work porting over and updating warn-scraper conventions * Initial pass at porting usage docs biglocalnews#6 * Add cli list command * Update README * Fix Makefile help text * Commit lockfile * Add stories page * Add throttle and misc cleanups to cli and runner * Add dependency to setup.py * Clobber obsolete SD scraper test module * Log agency slug in runner * Add customizable throttling * Port and update cache * Remove doc tests and disable Python and other downstream actions in CI (for now) * Partial work on ca_san_diego_pd * Fix/update usage docs
* Use pre-commit for make formt * Rework to use Site class per agency. biglocalnews#3 biglocalnews#4
* Update contributor docs biglocalnews#5 * Update usage docs biglocalnews#6 * update deps * Update main README biglocalnews#11 * add stub page for maintainer docs biglocalnews#9
tweak
fix README h1
readme fix
Add discussions and issues link in README
…ews#19) * Implement/update San Diego scrape_meta and scrape methods * Add cache.write_json and read_json methods * Add filter option to CLI scrape command * update reqs
* Add basic test coverage * Default to empty string for scrape-meta filter arg * ignore linter code * linter fixes * add tox/testing and linting info to maintainers docs * remove py37 support and bump to Beta dev status * Add tox config and reqs.txts * Update Pipfile.lock
* Add CA orange_county_sheriff.py * add name to contributors --------- Co-authored-by: jrynning <[email protected]>
* Update contributor docs. Closes biglocalnews#25 * linter fixes
* Sonoma County Scraper (biglocalnews#32) * Add ca_sonoma_county_sheriff --------- Co-authored-by: ochezems <[email protected]>
* created scrape and scrape meta for Monterey County District Attorney * config folder location changed * removed scrape and changed case_num to case_id and in case we dont find case_id it will have title in case_id instead of none --------- Co-authored-by: Gerald Rich <[email protected]>
* Patch list_agencies bug biglocalnews#85 * Readability cleanup
* added scrape-meta and scrape for riverside pd * changes for meta-data file formating * changed file name * tiny fix * removed scrape and changed case_num to case_id
* refactor: deprecate scrape method * fix: case_number >> case_id --------- Co-authored-by: Gerald Rich <[email protected]>
* fix: tests bootstrap * ops: coverage * add coveragerc * fix: tox; test py39, 310, 311, 312 --------- Co-authored-by: Gerald Rich <[email protected]>
* Proof of concept, missing Class * Export out usable case index * Incremental work consolidating things * First attempt at class * closer * Closer * Implement logging * Polish * Proof of concept, missing Class * Export out usable case index * Incremental work consolidating things * First attempt at class * Kill notebook version brought back by rebase * Move ugly details to config file * Rename ugly detail file * Linting * Fix linting * Proof of concept, missing Class * Export out usable case index * Incremental work consolidating things * First attempt at class * Kill notebook version brought back by rebase * Proof of concept, missing Class * Export out usable case index * Incremental work consolidating things * First attempt at class * Proof of concept, missing Class * Export out usable case index * Incremental work consolidating things * First attempt at class * Proof of concept, missing Class * Proof of concept, missing Class * Export out usable case index * Incremental work consolidating things * First attempt at class * Move ugly details to config file * Rename ugly detail file * Linting * Fix linting * Build against biglocalnews#69 flag biglocalnews#70 * ... * Apply suggestions * Clean up notes * Proof of concept, missing Class * Export out usable case index * Incremental work consolidating things * First attempt at class * closer * Closer * Implement logging * Polish * Proof of concept, missing Class * Export out usable case index * Incremental work consolidating things * First attempt at class * Kill notebook version brought back by rebase * Move ugly details to config file * Rename ugly detail file * Linting * Fix linting * Proof of concept, missing Class * Export out usable case index * Incremental work consolidating things * First attempt at class * Kill notebook version brought back by rebase * Proof of concept, missing Class * Export out usable case index * Incremental work consolidating things * First attempt at class * Proof of concept, missing Class * Export out usable case index * Incremental work consolidating things * First attempt at class * Proof of concept, missing Class * Proof of concept, missing Class * Export out usable case index * Incremental work consolidating things * First attempt at class * Move ugly details to config file * Rename ugly detail file * Linting * Fix linting * Build against biglocalnews#69 flag biglocalnews#70 * ... * Apply suggestions * Clean up notes * Copypaste around rebase problems --------- Co-authored-by: Gerald Rich <[email protected]>
* added scrape meta for chula_vista_pd biglocalnews#94 * removed user-agent * changes done * Rework URL handling; clean up a little more text * Linted. Oops. --------- Co-authored-by: Mike Stucka <[email protected]>
* fix: sdpd case_id pagination * simplify case_id --------- Co-authored-by: Gerald Rich <[email protected]>
* added scrape meta for fresno pd biglocalnews#114 * file name changed fresno_county_sheriff * file name changed fresno_county_sheriff
Co-authored-by: Gerald Rich <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.