Skip to content

Commit

Permalink
feat: Add the ability to skip updating a pre-commit repo as a part of…
Browse files Browse the repository at this point in the history 
… the reusable workflows to enable working around hooks that no longer work if they are updated past a certain version
  • Loading branch information
@nfelt14
nfelt14 committed Oct 22, 2024
1 parent 91faf8c commit 1d25bc5
Show file tree
Hide file tree
Showing 8 changed files with 248 additions and 37 deletions.
13 changes: 10 additions & 3 deletions .github/workflows/_reusable-update-python-and-pre-commit-dependencies.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,13 @@ on:
required: false
type: boolean
default: false
pre-commit-hook-skip-list:
pre-commit-repo-update-skip-list:
description: A comma-separated list of pre-commit repo urls to skip updates
for (only applicable when `update-pre-commit=true`).
required: false
type: string
default: ''
pre-commit-hook-run-skip-list:
description: A comma-separated list of pre-commit hooks to skip (only applicable
when `run-pre-commit=true`).
required: false
Expand Down Expand Up @@ -89,15 +95,16 @@ jobs:
dependency-dict: ${{ inputs.dependency-dict }}
update-pre-commit: ${{ inputs.update-pre-commit }}
run-pre-commit: ${{ inputs.run-pre-commit }}
pre-commit-hook-skip-list: ${{ inputs.pre-commit-hook-skip-list }}
pre-commit-repo-update-skip-list: ${{ inputs.pre-commit-repo-update-skip-list }}
pre-commit-hook-run-skip-list: ${{ inputs.pre-commit-hook-run-skip-list }}
export-dependency-groups: ${{ inputs.export-dependency-groups }}
- if: ${{ !endsWith(github.repository, '/python-package-ci-cd') }} # Run the public action when this is run outside the python-package-ci-cd repository
uses: tektronix/python-package-ci-cd/actions/[email protected]
with:
dependency-dict: ${{ inputs.dependency-dict }}
update-pre-commit: ${{ inputs.update-pre-commit }}
run-pre-commit: ${{ inputs.run-pre-commit }}
pre-commit-hook-skip-list: ${{ inputs.pre-commit-hook-skip-list }}
pre-commit-hook-run-skip-list: ${{ inputs.pre-commit-hook-run-skip-list }}
export-dependency-groups: ${{ inputs.export-dependency-groups }}
- uses: stefanzweifel/git-auto-commit-action@8621497c8c39c72f3e2a999a26b4ca1b5058a842 # v5.0.1
with:
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/test-actions.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,8 @@ jobs:
update-pre-commit: true
run-pre-commit: true
dependency-dict: '{"dev": ["pyright"]}'
pre-commit-hook-skip-list: remove-tabs,forbid-tabs,check-readthedocs,check-dependabot,check-github-actions,check-github-workflows,commitizen,blacken-docs,yamlfix,hadolint,mdformat,markdown-link-check,check-poetry,toml-sort-fix,pyright,poetry-audit,ruff,ruff-format,docformatter,renovate-config-validator,actionlint
pre-commit-repo-update-skip-list: https://github.com/pre-commit/pre-commit-hooks,https://github.com/executablebooks/mdformat
pre-commit-hook-run-skip-list: remove-tabs,forbid-tabs,check-readthedocs,check-dependabot,check-github-actions,check-github-workflows,commitizen,blacken-docs,yamlfix,hadolint,mdformat,markdown-link-check,check-poetry,toml-sort-fix,pyright,poetry-audit,ruff,ruff-format,docformatter,renovate-config-validator,actionlint
export-dependency-groups: |
actions-update_development_dependencies:actions/update_development_dependencies,
tests
Expand Down
5 changes: 5 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,14 @@ Valid subsections within a version are:

Things to be included in the next release go here.

### Added

- Added a new `pre-commit-repo-update-skip-list` input parameter to the `update_development_dependencies` action and the `_reusable-update-python-and-pre-commit-dependencies.yml` workflow to allow users to skip updating specific `pre-commit` hooks.

### Changed

- Bumped dependency versions.
- _**<span style="color:red">BREAKING CHANGE</span>**_: Renamed the `pre-commit-hook-skip-list` input parameter to `pre-commit-hook-run-skip-list` in the `_reusable-update-python-and-pre-commit-dependencies.yml` workflow and `update_development_dependencies` action.

---

Expand Down
7 changes: 6 additions & 1 deletion actions/update_development_dependencies/action.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,12 @@ inputs:
the update-pre-commit input to `true`.
required: false
default: 'false'
pre-commit-hook-skip-list:
pre-commit-repo-update-skip-list:
description: A comma-separated list of pre-commit repo urls to skip updates for
(only applicable when `update-pre-commit=true`).
required: false
default: ''
pre-commit-hook-run-skip-list:
description: A comma-separated list of pre-commit hooks to skip (only applicable
when `run-pre-commit=true`).
required: false
Expand Down
49 changes: 43 additions & 6 deletions actions/update_development_dependencies/main.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@

from pathlib import Path

import yaml

from pypi_simple import PyPISimple
from yamlfix import fix_files # pyright: ignore[reportUnknownVariableType]

Expand Down Expand Up @@ -126,21 +128,51 @@ def update_poetry_dependencies(
)


def update_pre_commit_dependencies(python_executable: str, repository_root_directory: Path) -> None:
def get_pre_commit_repos(repository_root_directory: Path) -> list[str]:
"""Get the list of repo urls from the .pre-commit-config.yaml file.
Args:
repository_root_directory: The root directory of the repository.
Returns:
The list of repo urls.
"""
pre_commit_file_data = yaml.safe_load(
(repository_root_directory / ".pre-commit-config.yaml").read_text()
)
repo_list: list[str] = []
for repo in pre_commit_file_data.get("repos", []):
if (repo_url := str(repo["repo"])) == "local":
continue # skip local repos, they don't need to be updated
repo_list.append(repo_url)
return repo_list


def update_pre_commit_dependencies(
python_executable: str,
repository_root_directory: Path,
pre_commit_repo_update_skip_list: list[str],
) -> None:
"""Update the pre-commit dependencies in the .pre-commit-config.yaml file.
This function will also fix the formatting of the yaml file using the `yamlfix` package.
Args:
python_executable: The path to the python executable to use.
repository_root_directory: The root directory of the repository.
pre_commit_repo_update_skip_list: A list of pre-commit repo urls to skip updating.
"""
run_cmd_in_subprocess(
f"git config --global --add safe.directory "
f'"{repository_root_directory.resolve().as_posix()}"'
)
# Update pre-commit config file
run_cmd_in_subprocess(f'"{python_executable}" -m pre_commit autoupdate --freeze')
# Update every hook in the pre-commit config file, skipping any hooks in the skip list
for repo in get_pre_commit_repos(repository_root_directory):
if repo in pre_commit_repo_update_skip_list:
continue
run_cmd_in_subprocess(
f'"{python_executable}" -m pre_commit autoupdate --freeze --repo {repo}'
)

# Fix the formatting of the pre-commit config file
with warnings.catch_warnings():
Expand Down Expand Up @@ -194,7 +226,10 @@ def main() -> None:
export_dependency_groups = [
x.strip() for x in os.environ["INPUT_EXPORT-DEPENDENCY-GROUPS"].split(",") if x
]
pre_commit_hook_skip_list = os.environ["INPUT_PRE-COMMIT-HOOK-SKIP-LIST"]
pre_commit_hook_run_skip_list = os.environ["INPUT_PRE-COMMIT-HOOK-RUN-SKIP-LIST"]
pre_commit_repo_update_skip_list = [
x.strip() for x in os.environ["INPUT_PRE-COMMIT-REPO-UPDATE-SKIP-LIST"].split(",") if x
]
install_dependencies = os.environ["INPUT_INSTALL-DEPENDENCIES"].lower() in _ENV_VAR_TRUE_VALUES
run_pre_commit = os.environ["INPUT_RUN-PRE-COMMIT"].lower() in _ENV_VAR_TRUE_VALUES
update_pre_commit = os.environ["INPUT_UPDATE-PRE-COMMIT"].lower() in _ENV_VAR_TRUE_VALUES
Expand All @@ -208,14 +243,16 @@ def main() -> None:
python_executable, repo_root_path, dependency_dict, lock_only=not install_dependencies
)
if update_pre_commit or run_pre_commit:
update_pre_commit_dependencies(python_executable, repo_root_path)
update_pre_commit_dependencies(
python_executable, repo_root_path, pre_commit_repo_update_skip_list
)
if export_dependency_groups:
export_requirements_files(python_executable, export_dependency_groups)
if run_pre_commit:
# Run the pre-commit hooks, ignore any errors since they are
# just being run to auto-fix files.
with contextlib.suppress(subprocess.CalledProcessError):
os.environ["SKIP"] = pre_commit_hook_skip_list
os.environ["SKIP"] = pre_commit_hook_run_skip_list
run_cmd_in_subprocess(f'"{python_executable}" -m pre_commit run --all-files')


Expand Down
22 changes: 12 additions & 10 deletions actions/update_development_dependencies/readme.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,16 @@ This action enables updating Python development dependencies using the
## Inputs
| Input variable | Necessity | Description | Default |
| --------------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------- |
| `repo-root` | optional | The root directory of the repository. | `.` |
| `install-dependencies` | optional | A boolean indicating if packages should be installed via poetry (this is not usually needed). | `false` |
| `dependency-dict` | optional | Specify a valid JSON dictionary of dependency groups to update, where each key is a dependency group name, and each value is a list of dependencies to update within that group, e.g. `'{"dev": ["pylint", "ruff"], "tests": ["ruff"]}'`. Use an empty string, e.g. `""`, for dependencies located in the default group' | `{}` |
| `update-pre-commit` | optional | A boolean indicating if the pre-commit hooks should be updated. | `false` |
| `run-pre-commit` | optional | A boolean indicating to run the pre-commit hooks to perform auto-fixing after updating the dependencies. Setting this input to `true` will also set the update-pre-commit input to `true`. | `false` |
| `pre-commit-hook-skip-list` | optional | A comma-separated list of pre-commit hooks to skip (only applicable when `run-pre-commit=true`). | `""` |
| `export-dependency-groups` | optional | A comma-separated list of dependency groups to export to a `requirements.txt` file. The format is `group1,group2:custom-path/to/test/folder`. | `""` |
| Input variable | Necessity | Description | Default |
| ---------------------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------- |
| `repo-root` | optional | The root directory of the repository. | `.` |
| `install-dependencies` | optional | A boolean indicating if packages should be installed via poetry (this is not usually needed). | `false` |
| `dependency-dict` | optional | Specify a valid JSON dictionary of dependency groups to update, where each key is a dependency group name, and each value is a list of dependencies to update within that group, e.g. `'{"dev": ["pylint", "ruff"], "tests": ["ruff"]}'`. Use an empty string, e.g. `""`, for dependencies located in the default group' | `{}` |
| `update-pre-commit` | optional | A boolean indicating if the pre-commit hooks should be updated. | `false` |
| `run-pre-commit` | optional | A boolean indicating to run the pre-commit hooks to perform auto-fixing after updating the dependencies. Setting this input to `true` will also set the update-pre-commit input to `true`. | `false` |
| `pre-commit-repo-update-skip-list` | optional | A comma-separated list of pre-commit repo urls to skip updates for (only applicable when `update-pre-commit=true`). | `""` |
| `pre-commit-hook-run-skip-list` | optional | A comma-separated list of pre-commit hooks to skip (only applicable when `run-pre-commit=true`). | `""` |
| `export-dependency-groups` | optional | A comma-separated list of dependency groups to export to a `requirements.txt` file. The format is `group1,group2:custom-path/to/test/folder`. | `""` |
## Example
Expand All @@ -53,7 +54,8 @@ jobs:
dependency-dict: '{"dev": ["pylint", "ruff"], "tests": ["ruff"]}' # optional, but without it nothing will get updated by Poetry
update-pre-commit: true # optional
run-pre-commit: true # optional
pre-commit-hook-skip-list: 'pylint' # optional, hooks that don't auto-fix things can (and probably should be) skipped
pre-commit-repo-update-skip-list: 'https://github.com/pre-commit/pre-commit-hooks' # optional
pre-commit-hook-run-skip-list: 'pylint' # optional, hooks that don't auto-fix things can (and probably should be) skipped
export-dependency-groups: 'docs,tests:custom-path/to/test/folder' # optional
- uses: stefanzweifel/git-auto-commit-action@v5
Expand Down
Loading

0 comments on commit 1d25bc5

Please sign in to comment.