Template for a new Python project.
To use this template:
- Read through this README.
- Start a new repository from this template by clicking "Use this template" on the repo's GitHub page.
- Name your repository source, this will be the name to import (e.g.
import aspect_py_project_template
).- Replace
./src/aspect_py_project_template
with your desired package name. - Replace all text instances of
aspect_py_project_template
with your desired package name.
- Replace
- Replace all text instances of
aspect-py-project-template
with your desired package name.- This is the name of the package that will be published to the internal PyPI repository. e.g. installable via
pip install aspect-py-project-template
.
- This is the name of the package that will be published to the internal PyPI repository. e.g. installable via
- Replace all text instances of
example_env
with your desired environment name for local development. - Stage the changes to be committed to git, and commit them.
- Add an initial git tag to the commit, this will be the first version of the package.
git tag v0.0.1
- Push the changes to the remote repository.
git push && git push --tags
If you want to create a new cloudfunction, look at the cloudfunctions_py_example_app
project instead as a starting point!
./env
: Python environment requirements and scripts to setup a local development environment../infra
: Docker containers and other project specific infrastructure code../notebooks
: Jupyter notebooks (if needed)./src
: Source code directory./src/aspect_py_project_template
: Package of this repo, changeaspect_py_project_template
to your desired package name../src/scripts
: Any scripts that should not be part of the installable package itself.
./test/
: Project specific tests./tools/
: Tools for code checking, formatting, and testing.
See each subdirectory for its specific documentation.
Please remove any directories and files from this template that are not needed for your project.
This repository defines an example installable Python package with example name aspect-py-project-template
.
To automatically publish new builds of this package to our internal PyPI repository, Set pythonPublish: true
in the Jenkinsfile
.
The package config is defined in the pyproject.toml file, which follows the pyproject.toml specification.
The package itself is build in our cloud platform by our CI/CD build system as configured in the Jenkinsfile
. The CI/CD pipeline uses the container provided at infra/Dockerfile to build the package using the the build tool.
If the tests and build passes, and the Jenkinsfile
has pythonPublish: true
a new version of the package is automatically published to our internal PyPI repository each time when successfully merged to the Git main
branch. It gets assigned a version according to an environment variable injected by the CI system which should correspond to the Git tag.
You can find more information on how to use the internal PyPI repositories in the Aspect-Analytics - Nexus3 docs, you might need VPN access to view this.
You can install packages build by the CI/CD system from pip. To do this you will have to setup credentials to the Aspect Analytics internal PyPI repository.
You can find the credentials under your Aspect Analytics 1Password, in the "Shared" vault under Aspect-Analytics - Nexus3 engineer
.
You can use these credentials to pip install a package using --extra-index-url
:
pip install --extra-index-url <url_with_credentials> <package_name>
Where <url_with_credentials>
is the URL to the internal PyPI repository with login credentials of Aspect-Analytics - Nexus3 engineer
, and is of the form https://<account_name>:<password>@nexus.aspect.aspect-analytics.ninja/repository/pypi-aspect-analytics-private/simple
.
Instead of always having to specify the credentials you can configure them by setting up your ~/.config/pip/pip.conf as detailed below.
If the ~/.config/pip/pip.conf
does not exist yet for you you can create it by running the following commands in your terminal:
mkdir -p ~/.config/pip/ && touch ~/.config/pip/pip.conf
Then open the file in your favorite text editor. Once in text editor, add the following credentials:
[global]
extra-index-url = <url_with_credentials>
To install this example package build by the CI/CD system run:
pip install aspect-py-project-template
You can get all available versions of the package by running: pip index versions aspect-py-project-template
with the latest version of pip.
A Python environment to locally develop on this repo is provided at ./env
, for more info see the env Readme. This environment can be setup locally by running:
./env/setup_local_conda_env.sh --name example_env
If you've chosen a different environment name, make sure to do a search and replace example_env
to the name of your choosing in .pre-commit-config.yaml
.
Tooling to standardize, check and test the code are provided at ./tools
. To run all checks:
./tools/run_all.sh
Pre-commit hooks are setup when setting up the local environment from ./env. Once setup, they will run automatically when committing changes to the repository. To run them manually run:
pre-commit run --all-files
The pre-commit hooks will mainly take care of standardizing the code by running formatters. By default they won't run tests. To ensure that your code works run tests locally before committing. Additionally, automated tests are part of the container build in ./infra and will run automatically on the CI/CD system.
This repository's container is automatically built as is defined in the Jenkinsfile
. The build includes running all checks and tests and should fail if any of these fail. The build results can be found in Jenkins (this requires VPN access).
An example Docker image including tests is provided at ./infra/Dockerfile
. More info on how to build and run this at ./infra/README.md
.
Use Conventional Commits git messages to version the repository automatically.