Merge branch 'rj_updates' of github.com:openstates/openstates-core in…

…to rj_updates

.github/workflows/release.yml

@@ -20,6 +20,8 @@ jobs:
         uses: actions/setup-python@v4
         with:
           python-version: 3.9
+      - name: install GDAL and python bindings
+        run: sudo apt update && sudo apt-get install -y gdal-bin python3-gdal
       - name: install Poetry
         uses: snok/[email protected]
       - name: configure pypi credentials

CHANGELOG.md

@@ -1,9 +1,22 @@
 # Changelog
 
-## 6.18.0 - March 25, 2024
+## 6.18.3 - March 25, 2024
 
 * Add other_names to committee matching for Event Imports
 
+## 6.18.2
+
+* Maine (ME): add a tribal representative district to the jurisdiction metadata
+
+## 6.18.1 - January 24, 2024
+
+* Stop validating the repeated use of the same phone number in People offices data
+
+## 6.18.0 - January 24, 2024
+
+* People: add distinct mailing classification options (district-mail, capitol-mail) and home classification to offices
+* Docs: Add debug instructions for the update command by @jessemortenson in #119
+
 ## 6.17.9 - October 26, 2023
 
 * allow postimport hook to be skipped in os-update command

README.md

@@ -5,3 +5,60 @@ This repository contains the Open States data model and scraper backend.
 ## Links
 
 * [Code of Conduct](https://docs.openstates.org/en/latest/contributing/code-of-conduct.html)
+* [Contributing to Open States](https://docs.openstates.org/contributing/)
+
+## Release steps
+
+See [RELASE.md](./RELEASE.md)
+
+## Debugging openstates-core code
+
+### Update command / scrapers
+
+There are
+[instructions on running a scraper here](https://docs.openstates.org/contributing/scrapers/#running-your-first-scraper)
+but what if you want to debug the command that wraps around the scraper code? This
+repository's [update CLI module](openstates/cli/update.py)
+is the code that accepts parameters and actually executes the scraper (from
+[openstates-scrapers](https://github.com/openstates/openstates-scrapers)). Another wrinkle is that the update command
+needs to be run inside the context of the `openstates-scrapers` repository, as it will attempt to load in the relevant
+scraper for the jurisdiction requested, and that import will fail if you try to run the code here
+within `openstates-core`.
+
+Here's a recipe using PyCharm to successfully debug the update command:
+
+#### Requirements to run code natively (not in docker)
+
+* You need the `gdal` library installed on the host system. For me: `sudo apt install gdal-bin python3-gdal`
+* `openstates-core` checked out at /home/username/repo/openstates/openstates-core/
+* (let's assume you have made some changes in `openstates-core` that you want to test)
+* `openstates-scrapers` checked out /home/username/repo/openstates/openstates-scrapers/
+* Change directory to /home/username/repo/openstates/openstates-scrapers/
+* Install required python version using the `pyenv` utility
+* `pip install poetry` (if that python version doesn't already have it)
+
+#### Debugging natively
+
+* If you have previously installed the `openstates` dependency (eg `openstates-core`), then you need
+  to run `poetry remove openstates` to clear that remotely-installed (from pypi) dependency. Each time you make a round
+  of changes to `openstates-core`, you will need to remove and then re-add the dependency.
+* `poetry add ../openstates-core/` will add the `openstates` dependency from your local filesystem/local checkout
+* `poetry install`
+* In PyCharm, open the `openstates-scrapers` folder
+* In PyCharm, set up a new run config:
+    * type: python
+    * module: openstates.cli.update
+    * parameters: vi bills (or whatever you want to run)
+    * working directory: /home/username/repo/openstates/openstates-scrapers/scrapers
+* Run the run config in debug mode within PyCharm (eg the one working on `openstates-scrapers`). You can set
+  breakpoints within both the scraper code AND in the `openstates-core` code. However you need to open (in PyCharm)
+  the copy of `openstates-core` that poetry installed, which probably is in a location like:
+  `/home/username/.cache/pypoetry/virtualenvs/openstates-scrapers-93BMrPXy-py3.9/lib/python3.9/site-packages/openstates/cli/update.py`
+
+**Reminder**: when further changes are made to the `openstates-core` package locally, and you want to debug them again,
+you need to remove/re-add to update files in `openstates-scrapers`
+
+* Change directory to `/home/username/repo/openstates/openstates-scrapers/`
+* `poetry remove openstates`
+* `poetry add ../openstates-core/`
+* (you will need to re-establish breakpoints in any openstates-core files)

RELEASE.md

@@ -1,9 +1,13 @@
 Release Process:
 
-* bump version in pyproject.toml
-* update changelog
-* tag & push tag
-* poetry publish --build
+* bump version in `pyproject.toml`
+* update `CHANGELOG.md`
+* once code is merged into `main`
+  * Create a new Release in Github
+    * Target `main`
+    * Create a new tag that matches new version from `pyproject.toml`
+  * By creating a tag, this triggers the [release workflow](./.github/workflows/release.yml) which builds and
+    pushes the new version to PyPi
 
 Post-Release updates:
 

openstates/data/models/people_orgs.py

@@ -286,8 +286,11 @@ def __str__(self):
 
 OFFICE_CHOICES = (
     ("district", "District Office"),
+    ("district-mail", "District Mailing Address"),
     ("capitol", "Capitol Office"),
+    ("capitol-mail", "Capitol Mailing Address"),
     ("primary", "Primary Office"),
+    ("home", "Home"),
 )
 
 

openstates/metadata/data/me.py

@@ -1,4 +1,4 @@
-from ..models import State, Chamber, simple_numbered_districts
+from ..models import State, Chamber, District, simple_numbered_districts
 
 ME = State(
     name="Maine",
@@ -18,11 +18,12 @@
         chamber_type="lower",
         name="House",
         organization_id="ocd-organization/a26314f0-dc6f-4cfc-99ca-dee4bafc9d4c",
-        num_seats=151,
+        num_seats=152,
         title="Representative",
         districts=simple_numbered_districts(
             "ocd-division/country:us/state:me", "lower", 151
-        ),
+        ) + [District("Passamaquoddy Tribe", "lower",
+                      "ocd-division/country:us/state:me/sldl:passamaquoddy-tribe")],
     ),
     upper=Chamber(
         chamber_type="upper",

openstates/models/people.py

@@ -85,8 +85,11 @@ class RoleType(str, Enum):
 
 class OfficeType(str, Enum):
     DISTRICT = "district"
+    DISTRICT_MAIL = "district-mail"
     CAPITOL = "capitol"
+    CAPITOL_MAIL = "capitol-mail"
     PRIMARY = "primary"
+    HOME = "home"
 
 
 class PersonIdBlock(BaseModel):

openstates/utils/people/lint_people.py

@@ -94,7 +94,7 @@ def validate_offices(person: Person) -> list[str]:
     for office in person.offices:
         type_counter[office.classification] += 1
         for key, value in office.dict().items():
-            if key == "classification" or not value:
+            if key == "classification" or key == "voice" or key == "fax" or not value:
                 continue
             # reverse lookup to see if we've used this phone number/etc. before
             location_str = f"{office.classification} {key}"

openstates/utils/tests/test_lint.py

@@ -172,15 +172,6 @@ def test_validate_roles_retired(roles, expected):
             ],
             [],
         ),
-        (
-            [
-                {"classification": "district", "voice": "123-444-4444"},
-                {"classification": "capitol", "voice": "123-444-4444"},
-            ],
-            [
-                "Value '123-444-4444' used multiple times: district voice and capitol voice"
-            ],
-        ),
     ],
 )
 def test_validate_offices(offices, expected):

pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "openstates"
-version = "6.18.0"
+version = "6.18.3"
 description = "core infrastructure for the openstates project"
 authors = ["James Turk <[email protected]>"]
 license = "MIT"