Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(copier): enhance copier.yml #681

Open
wants to merge 75 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 65 commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
05b1efd
feat: update copier.yml
34j Mar 4, 2024
6acb969
fix(copier.yml): revoke unnecessary changes
34j Mar 4, 2024
7b9ccb5
chore(poerty): add deps
34j Mar 4, 2024
a0b45e3
chore(poetry): add arrow for jinja2-time
34j Mar 4, 2024
ca01b71
fix(copier): suppress error if vscode not found
34j Mar 4, 2024
70ab8c3
test: use same venv version
34j Mar 4, 2024
11c3b24
ci: specify venv version
34j Mar 4, 2024
7b5b111
fix(copier): fix replace command
34j Mar 5, 2024
b6ccbb3
chore(deps): remove unused jinja2-shell
34j Mar 5, 2024
f748aaa
fix(setup-github.bash): remove PYPI_TOKEN setting
34j Mar 5, 2024
28e1082
fix(copier.yml): change the order of project name questions
34j Mar 5, 2024
fbe947f
revert(copier.yml): revert "change the order of project name questions"
34j Mar 5, 2024
f3b80f0
fix(copier): do not run autoupdate
34j Mar 5, 2024
c26c356
fix(copier): add after update
34j Mar 5, 2024
e058cda
fix(copier): always use git bash instead of wsl in windows to avoid c…
34j Mar 5, 2024
89d9205
fix(copier): replace sed command with python
34j Mar 5, 2024
0b96fd4
fix(copier): run poetry as an executable instead of a module
34j Mar 5, 2024
021b162
fix(setup-github.bash): no newlines
34j Mar 5, 2024
4a4a874
fix(setup-github.bash): enclose short desc with double quotes
34j Mar 5, 2024
547eae2
fix(copier): fix poetry creating another venv
34j Mar 5, 2024
8fe09ea
fix(copier): do not activate venv in mac or linux
34j Mar 5, 2024
e997b5c
ci(ci): setup git user
34j Mar 5, 2024
bdf12d9
Merge branch 'main' of https://github.com/34j/pypackage-template into…
34j Oct 22, 2024
eb77837
chore: update lockfile
34j Oct 22, 2024
f15262d
style: run pre-commit
34j Oct 22, 2024
f87ec48
fix: fix jinja2 extensions
34j Oct 22, 2024
24f418c
fix(copier.yml): rearrange questions
34j Oct 22, 2024
40ff221
fix(copier): do not open with vscode by default
34j Oct 23, 2024
158a9d8
docs: update docs
34j Oct 23, 2024
9d76ea2
chore: update script
34j Oct 23, 2024
038b922
fix: use `PYPACKAGE_TEMPLATE_GITHUB_TOKEN` instead of `GITHUB_TOKEN`
34j Oct 23, 2024
2d56b11
ci: install jinja2 packages
34j Oct 23, 2024
7edbd47
fix(copier.yml): do not use sed
34j Oct 23, 2024
5cf2390
fix(copier.yml): fix syntax
34j Oct 23, 2024
b4fdfed
ci: git config globally
34j Oct 23, 2024
16c0812
fix(copier.yml): always run `git add --all`
34j Oct 23, 2024
0dfa65e
docs: update docs
34j Oct 23, 2024
640223b
ci: do not setup github
34j Oct 23, 2024
90b4fb7
fix(copier.yml): fix naming convention
34j Oct 23, 2024
1a69df9
fix: fix syntax
34j Oct 23, 2024
05f6b02
fix: fix syntax
34j Oct 23, 2024
f276626
fix: fix naming convention
34j Oct 23, 2024
fe7e7c9
fix: warn if `PYPACKAGE_TEMPLATE_GITHUB_TOKEN` is not set
34j Oct 23, 2024
f7ce024
fix(copier.yml): do not stop if pre-commit raises
34j Oct 23, 2024
1951f5f
fix: update script
34j Oct 23, 2024
d6980d7
fix: make `PYPACKAGE_TEMPLATE_INSTALLATION_IDS` optional
34j Oct 23, 2024
68b9dc6
fix: fix setup script
34j Oct 23, 2024
9619080
fix: multiple fixes
34j Oct 23, 2024
a177939
fix: multiple fixes
34j Oct 23, 2024
f1f6d23
docs: update readme
34j Oct 23, 2024
731d566
fix: fix python not found error
34j Oct 23, 2024
700e3c3
fix: fix re.sub
34j Oct 23, 2024
36c6ca6
fix(copier.yml): fix python not found error
34j Oct 23, 2024
93f1b05
fix: fix command
34j Oct 23, 2024
eb7503a
docs: update docs
34j Oct 23, 2024
91a227f
fix: support GITHUB_TOKEN as well
34j Oct 23, 2024
7618e1b
docs: update readme
34j Oct 23, 2024
ec16f0f
docs: update readme
34j Oct 23, 2024
e0f94f9
docs: update readme
34j Oct 23, 2024
9232ad5
docs: update readme
34j Oct 23, 2024
462f462
fix: fix command
34j Oct 23, 2024
22955ca
docs: update readme
34j Oct 23, 2024
83fd54b
docs: lower
34j Oct 23, 2024
28f58e5
fix: remove space
34j Oct 23, 2024
08ceaab
fix: replace copier.yml only if initial commit
34j Oct 23, 2024
a424a75
fix(copier.yml): use $VISUAL env variable
34j Oct 23, 2024
875a69f
fix(copier): do not raise if gh is not found
34j Oct 23, 2024
ce0b5f0
fix(copier.yml): fix commands
34j Oct 23, 2024
9f333f6
fix(copier.yml): fix commands
34j Oct 23, 2024
ece0696
fix(copier.yml): set text to True
34j Oct 23, 2024
e8e3b1d
fix(copier.yml): do not use shell and use suppress
34j Oct 23, 2024
2bc6e15
fix(copier.yml): do not use shell and use suppress
34j Oct 23, 2024
034e135
fix(copier.yml): do not use shell and use suppress
34j Oct 23, 2024
d7665ee
fix(copier.yml): do not use shell and use suppress
34j Oct 23, 2024
252f1fc
fix(copier.yml): do not use shell and use suppress
34j Oct 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
- "3.12"
os:
- ubuntu-latest
# - windows-latest
- windows-latest
- macOS-latest
runs-on: ${{ matrix.os }}
name: "Template: ${{ matrix.python-version }} - ${{ matrix.os }}"
Expand Down Expand Up @@ -72,22 +72,28 @@ jobs:
name: "Generated: ${{ matrix.script.name }} ${{ matrix.extra_options.project_name }}"
steps:
- uses: actions/checkout@v4
- run: git config --global user.name github-actions[bot]
- run: git config --global user.email 41898282+github-actions[bot]@users.noreply.github.com
- uses: actions/setup-python@v5
id: setup-python
with:
python-version: "3.11"
- run: pipx install copier
- run: pipx inject copier jinja2-eval jinja2-env jinja2-time arrow
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does it work if folks don't have these installed? That seems like a significant change in workflow for users

Copy link
Contributor Author

@34j 34j Oct 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copier raises an error if not installed. There is no workaround. I suppose that if you are interested in jinja2-time, injecting is inevitable anyway. Of course, these are only necessary when using copier.

- run: |
pipx run copier copy \
copier copy \
. \
my-project \
--UNSAFE \
--data 'full_name=Bruno Alla' \
--data 'email=test@example.com' \
--data 'github_username=dummy' \
--data 'email=41898282+github-actions[bot]@users.noreply.github.com' \
--data 'github_username=github-actions[bot]' \
--data 'project_name=${{ matrix.extra_options.project_name }}' \
--data 'project_slug=${{ matrix.extra_options.project_slug }}' \
--data 'project_short_description=Just a great project' \
--data 'open_source_license=MIT' \
--data "venv_version=3.11" \
--data "setup_github=no" \
${{ matrix.extra_options.value }} \
--defaults
env:
Expand Down
21 changes: 17 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,14 @@ Project template for a Python Package using Copier.

## Usage

Generate a new project with:
First, install Copier and inject some dependencies:

```shell
pipx install copier
pipx inject copier jinja2-eval jinja2-env jinja2-time arrow
```

Next set up [trusted publisher](#trusted-publisher-setup) and then generate a new project with:

```shell
copier copy --trust "gh:browniebroke/pypackage-template" path-to-project
Expand All @@ -35,7 +42,14 @@ copier copy --trust "gh:browniebroke/pypackage-template" path-to-project
This will prompt you for a few questions and create new directory with the name you used as project slug.

> _Note:_
>
> the `--trust` option is required because this template may execute some tasks after generating the project, like initialising the git repo, installing dependencies and so forth. These are all listed in the `copier.yml` of this repo, under the `_tasks` key. They are all optional and safe to run. You can take my word for it, or better, check the code yourself!
>
> set `GITHUB_TOKEN` or `PYPACKAGE_TEMPLATE_GITHUB_TOKEN` environment variable with a [personal access token (PAT)][create-pat] with the `repo` scope.
>
> go to [Applications Settings](https://github.com/settings/installations) and copy the id in the link (`https://github.com/organizations/<Organization-name>/settings/installations/<ID>`) for the `Configure` button for the GitHub Apps you want to have installed automatically, and set `PYPACKAGE_TEMPLATE_INSTALLATION_IDS` environment variable with the comma separated list of IDs. (you may want to install [Renovate](https://github.com/marketplace/renovate), [pre-commit ci](https://github.com/marketplace/pre-commit-ci), as AllContributors and Codecov can be installed globally.)
>
> you need to set `GITHUB_TOKEN` environment variable with a [PAT][create-pat-local] with the `repo` (for app installation) `workflow` (for pushing refs) and `user` (for getting username and email) scopes. (If you login with `gh auth login --scopes repo,workflow,user`, app installation will fail.)

### Start developing

Expand Down Expand Up @@ -71,8 +85,6 @@ The workflows need [a few secrets][gh-secrets] to be setup in your GitHub reposi
- `GH_PAT` a [personal access token (PAT) with the `repo` scope][create-pat] for opening pull requests and updating the repository topics. This is used by the `poetry-upgrade` and `labels` workflows.
- `CODECOV_TOKEN` to upload coverage data to [codecov.io][codecov] in the Test workflow.

If you have the GitHub CLI installed and chose to set up GitHub, they will be created with a dummy value (`changeme`).

### Automated release

By following the conventional commits specification, we're able to completely automate versioning and releasing to PyPI. It runs on every push to your main branch, as part of the `release` job of the `ci.yml` workflow. You shouldn't need to create a token, but you'll need to setup [trusted publisher](https://docs.pypi.org/trusted-publishers/using-a-publisher/) for the project.
Expand Down Expand Up @@ -183,6 +195,7 @@ This project follows the [all-contributors](https://github.com/all-contributors/
[gh-secrets]: https://help.github.com/en/actions/configuring-and-managing-workflows/creating-and-storing-encrypted-secrets
[codecov]: https://codecov.io/
[pypi]: https://pypi.org/
[create-pat]: https://github.com/settings/tokens/new?scopes=repo
[create-pat]: https://github.com/settings/tokens/new?description=pypackage-template&scopes=repo
[create-pat-local]: https://github.com/settings/tokens/new?description=pypackage-template-local&scopes=repo,user,workflow
[rtd-dashboard]: https://readthedocs.org/dashboard/
[all-contribs-install]: https://allcontributors.org/docs/en/bot/installation
105 changes: 70 additions & 35 deletions copier.yml
Original file line number Diff line number Diff line change
@@ -1,26 +1,38 @@
# questions
project_short_description:
type: str
help: "A short description of the project"
default: "Python package for "

has_cli:
type: bool
help: "Does the project have a CLI?"
default: no

is_django_package:
type: bool
help: "Is the project a Django package?"
default: no

full_name:
type: str
help: "What's your name?"
default: '{{ ("exec(''from subprocess import run; import json; j = json.loads(run([\"gh\",\"api\",\"user\"], capture_output=True).stdout)'') or j[''name''] or j[''login'']" | eval) | env("PYPACKAGE_TEMPLATE_NAME") }}'

email:
type: str
help: "Email address"
placeholder: "[email protected]"
default: '{{ ("exec(''from subprocess import run; import json; j = json.loads(run([\"gh\",\"api\",\"user/emails\"], capture_output=True).stdout)'') or [k for k in j if k[''primary'']][0][''email'']" | eval) | env("PYPACKAGE_TEMPLATE_EMAIL") }}'
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if the person doesn't have gh installed? I think it's fully optional right now, but that seem to make it required.

Copy link
Contributor Author

@34j 34j Oct 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed to work even if gh is not installed using contextlib.suppress, confirmed that it works


github_username:
type: str
help: "GitHub Username"

is_django_package:
type: bool
help: "Is the project a Django package?"
default: no
default: '{{ ("exec(''from subprocess import run; import json; j = json.loads(run([\"gh\",\"api\",\"user\"], capture_output=True).stdout)'') or j[''login'']" | eval) | env("PYPACKAGE_TEMPLATE_GITHUB_USER") }}'
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, adding a hard dependency to save a few keystrokes


project_name:
type: str
help: "Project Name (human readable version){% if is_django_package %}, should start by 'Django'{% endif %}."
placeholder: "Python Package"
default: '{{ "exec(''import sys; s = sys.argv[-1]; u = \" \".join([t.capitalize() for t in s.replace(\".\", \" \").replace(\"_\", \" \").replace(\"-\", \" \").split()])'') or u" | eval }}'

project_slug:
type: str
Expand All @@ -32,17 +44,18 @@ package_name:
help: "The name of the main Python package (should be a valid Python identifier{% if is_django_package %} and start by 'django_'{% endif %})"
default: "{{ project_slug.replace('-', '_') }}"

cli_name:
type: str
help: "The name of the CLI"
default: "{{ project_slug }}"
when: "{{ has_cli }}"

django_app_shorthand:
type: str
help: "The Django app shorthand, typically the package name without the 'django_' prefix."
default: "{{ package_name.removeprefix('django_') }}"
when: "{{ is_django_package }}"

project_short_description:
type: str
help: "A short description of the project"
placeholder: "A super helpful small Python package."

open_source_license:
type: str
help: "The open source license to use"
Expand All @@ -55,69 +68,91 @@ open_source_license:
copyright_year:
type: str
help: "Copyright year(s)"
default: "2022"
default: "{% now 'utc', '%Y' %}"

documentation:
type: bool
help: "Generate documentation?"
default: yes

has_cli:
setup_venv:
type: bool
help: "Does the project have a CLI?"
default: no
help: "Setup a virtual environment?"
default: yes
Comment on lines +78 to +81
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why bother with venv creation? Thsi template uses Poetry, which manages venv automatically. Would be similar if/when it's moved to uv...

Copy link
Contributor Author

@34j 34j Oct 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm starting to think that for advanced users like you, this should rather be considered a temporal virtual environment just to support all operations up to pushing to GitHub, in case pre-commit and poetry are not installed globally, rather than used to develop projects. No matter which tool is used, I believe the virtual environment created would be equivalent if the setting corresponds to "poetry config virtualenvs.in-project true". Of course you can switch Python versions using poetry env etc. It is quite difficult to write cross-platform commands to access the right python using something like poetry shell. The change is extremely painful (as jinja2 forgets variables frequently), but it is certainly possible to use poetry env info -p to call python in the virtual environment without using the poetry shell. In that case, the user would need to install poetry beforehand.


cli_name:
venv_version:
type: str
help: "The name of the CLI"
default: "{{ project_slug }}"
when: "{{ has_cli }}"
help: "Python version for the virtual environment"
default: "3.11"
when: "{{ setup_venv }}"
Comment on lines +83 to +87
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a hard block, but FWIW I personally see little value in this question...


run_poetry_install:
type: bool
help: "Run poetry install after {{ package_name }} generation?"
default: no
default: yes
when: "{{ setup_venv }}"

initial_commit:
type: bool
help: "Create an initial commit with the generated {{ package_name }}?"
default: no
default: yes

setup_github:
type: bool
help: "Setup GitHub repository (requires gh CLI)?"
default: no
default: yes
when: "{{ initial_commit }}"

setup_pre_commit:
type: bool
help: "Setup pre-commit hooks (requires pre-commit)?"
default: no
default: yes
when: "{{ setup_venv }}"

add_me_as_contributor:
type: bool
help: "Add me as a contributor?"
default: yes
when: "{{ initial_commit }}"

open_with_vscode:
type: bool
help: "Open with VSCode?"
default: no
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I don't use VSCode, so no. I'm not interested in making this question specific to a given IDE/editor either, but would be interesting if there is convention in env variable to open in the user's preferred editor: https://unix.stackexchange.com/q/4859

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to use $VISUAL


# Copier metadata
_min_copier_version: "9.0.0"
_subdirectory: "project"
_tasks:
# In Windows, command prompt
# Remove license file if no license
- "{% if open_source_license == 'Not open source' %}rm LICENSE{% endif %}"
# Cleanup docs
- "{% if not documentation %}rm -rf docs .readthedocs.yml{% endif %}"
# Setup venv
- "{% if setup_venv %}{% if 'Windows' in ''|env('OS') %}py -{% else %}python{% endif %}{{ venv_version }} -m venv venv{% endif %}"
# Run poetry install
- "{% if run_poetry_install %}poetry install{% endif %}"
# Initial commit
- "{% if initial_commit %}git init{% endif %}"
- "{% if initial_commit %}git add .{% endif %}"
- "{% if initial_commit %}git commit -m 'chore: initial commit'{% endif %}"
# Setup GitHub
- "{% if setup_github %}gh repo create {{ github_username }}/{{ project_slug }} -d '{{ project_short_description }}' --public --remote=origin --source=. --push{% endif %}"
- "{% if setup_github %}gh repo edit --delete-branch-on-merge --enable-projects=false --enable-wiki=false{% endif %}"
- "{% if setup_github %}gh secret set GH_PAT -b 'changeme'{% endif %}"
- "{% if setup_github %}gh secret set CODECOV_TOKEN -b 'changeme'{% endif %}"
- "{% if 'Windows' in ''|env('OS') %}{% set venv_folder = 'venv\\\\scripts\\\\' %}{% else %}{% set venv_folder = 'venv/bin/' %}{% endif %}
{% if run_poetry_install %}{{ venv_folder }}python -m pip install -U pip setuptools wheel && {{ venv_folder }}pip install -U poetry pre-commit && {{ venv_folder }}python -m poetry install --with dev{% endif %}"
# git init
- "git init && git add --all"
# Setup pre-commit
- "{% if setup_pre_commit %}pre-commit install{% endif %}"
- "{% if 'Windows' in ''|env('OS') %}{% set venv_folder = 'venv\\\\scripts\\\\' %}{% else %}{% set venv_folder = 'venv/bin/' %}{% endif %}
{% if setup_pre_commit %}{{ venv_folder }}pre-commit autoupdate && {{ venv_folder }}pre-commit install && {{ venv_folder }}pre-commit run -a || true{% endif %}"
# Initial commit
- '{% if initial_commit %}git add --all && git commit -m "chore: initial commit"{% endif %}'
# Replace true with false in .copier-answers.yml except "documentation: true" for faster copier update
# - "sed -i '/documentation: true/!s/true/false/g' .copier-answers.yml"
- "{% if 'Windows' in ''|env('OS') %}{% set venv_folder = 'venv\\\\scripts\\\\' %}{% else %}{% set venv_folder = 'venv/bin/' %}{% endif %}
{% if initial_commit %}{{ venv_folder }}python -c \"from pathlib import Path; import re; p = Path('.copier-answers.yml'); p.write_text(re.sub(r'((?:setup_venv|run_poetry_install|initial_commit|setup_github|setup_pre_commit|add_me_as_contributor): )true', r'\\g<1>false', p.read_text()))\"{% endif %}"
- '{% if initial_commit %}git add --all && git commit -m "chore: update .copier-answers.yml to avoid running commands again"{% endif %}'
# Add me as a contributor
- "{% if add_me_as_contributor %}npx all-contributors-cli add {{ github_username }} code,ideas,doc{% endif %}"
# Setup GitHub
- '{% if setup_github %}bash .github/setup-github.bash {{ github_username }} {{ project_slug }} "{{ project_short_description }}"{% endif %}'
# Open with vscode
- "{% if open_with_vscode %}code .{% endif %}"
_jinja_extensions:
- jinja2_time.TimeExtension
- jinja2_eval.EvalExtension
- jinja2_env.EnvExtension
Loading