Skip to content

Commit

Permalink
refactor(rest_db): allow more local customization
Browse files Browse the repository at this point in the history
* devs may or may not want to reset their local DB
* devs may want to change which DB file is targeted
* devs may want to change which fixture file is loaded

update docs to reflect these changes
  • Loading branch information
thekaveman committed Feb 15, 2024
1 parent aeaa3aa commit 17f7164
Show file tree
Hide file tree
Showing 4 changed files with 104 additions and 49 deletions.
6 changes: 6 additions & 0 deletions .env.sample
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
DJANGO_SUPERUSER_USERNAME=benefits-admin
DJANGO_SUPERUSER_EMAIL=[email protected]
DJANGO_SUPERUSER_PASSWORD=superuser12345!

DJANGO_DB_RESET=true
DJANGO_DB_DIR=.
DJANGO_DB_FILE=django.db
DJANGO_DB_FIXTURES="benefits/core/migrations/local_fixtures.json"

testsecret=Hello from the local environment!
auth_provider_client_id=benefits-oauth-client-id
courtesy_card_verifier_api_auth_key=server-auth-token
Expand Down
54 changes: 32 additions & 22 deletions bin/reset_db.sh
Original file line number Diff line number Diff line change
@@ -1,25 +1,35 @@
#!/usr/bin/env bash
set -eux

# remove database file

# construct the path to the database file from environment or default
DB_DIR="${DJANGO_DB_DIR:-.}"
DB_FILE="${DB_DIR}/django.db"

# -f forces the delete (and avoids an error when the file doesn't exist)
rm -f "${DB_FILE}"

# run database migrations and other initialization

bin/init.sh

# create a superuser account for backend admin access
# set username, email, and password using environment variables
# DJANGO_SUPERUSER_USERNAME, DJANGO_SUPERUSER_EMAIL, and DJANGO_SUPERUSER_PASSWORD

python manage.py createsuperuser --no-input

# load sample data fixtures

python manage.py loaddata benefits/core/migrations/local_fixtures.json
# whether to reset database file, defaults to true
DB_RESET="${DJANGO_DB_RESET:-true}"
# optional fixtures to import
FIXTURES="${DJANGO_DB_FIXTURES}"

if [[ $DB_RESET = true ]]; then
# construct the path to the database file from environment or default
DB_DIR="${DJANGO_DB_DIR:-.}"
DB_FILE="${DJANGO_DB_FILE:-django.db}"
DB_PATH="${DB_DIR}/${DB_FILE}"

rm -f "${DB_PATH}"

# run database migrations and other initialization
bin/init.sh

# create a superuser account for backend admin access
# set username, email, and password using environment variables
# DJANGO_SUPERUSER_USERNAME, DJANGO_SUPERUSER_EMAIL, and DJANGO_SUPERUSER_PASSWORD
python manage.py createsuperuser --no-input
else
echo "DB_RESET is false, skipping"
fi

valid_fixtures=$( echo $FIXTURES | grep -e fixtures\.json$ )

if [[ -n "$valid_fixtures" ]]; then
# load data fixtures
python manage.py loaddata "$FIXTURES"
else
echo "No JSON fixtures to load"
fi
43 changes: 18 additions & 25 deletions docs/configuration/data.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,24 @@
# Configuration data

!!! example "Data migration file"
!!! example "Sample data fixtures"

[`benefits/core/migrations/0002_data.py`][data-migration]
[`benefits/core/migrations/local_fixtures.py`][sample-fixtures]

!!! tldr "Django docs"

[How to provide initial data for models][django-load-initial-data]

## Introduction

Django [data migrations](https://docs.djangoproject.com/en/4.0/topics/migrations/#data-migrations) are used to load the database with instances of the app's model classes, defined in [`benefits/core/models.py`][core-models].
The app's model classes are defined in [`benefits/core/models.py`][core-models].

Migrations are run as the application starts up. See the [`bin/init.sh`][init] script.

The sample values provided in the repository are sufficient to run the app locally and interact with e.g. the sample Transit
Agencies.
Agencies. [Django fixtures][django-fixtures] are used to load the database with sample data when running locally.

During the [deployment](../deployment/README.md) process, environment-specific values are set in environment variables and are read by the data migration file to build that environment's configuration database. See the [data migration file][data-migration] for the environment variable names.
During the [deployment](../deployment/README.md) process, some environment-specific values are set in environment variables and
read dynamically at runtime. Most configuration values are managed directly in the Django Admin interface at the `/admin` endpoint.

## Sample data

Expand All @@ -37,32 +38,24 @@ Some configuration data is not available with the samples in the repository:
- Payment processor configuration for the enrollment phase
- Amplitude configuration for capturing analytics events

### Sample transit agency: `ABC`
## Rebuilding the configuration database locally

- Presents the user a choice between two different eligibility pathways
- One eligibility verifier requires authentication
- One eligibility verifier does not require authentication
A local Django database will be initialized upon first startup of the devcontainer.

### Sample transit agency: `DefTL`

- Single eligibility pathway, no choice presented to the user
- Eligibility verifier does not require authentication

## Building the configuration database

When the data migration changes, the configuration database needs to be rebuilt.

The file is called `django.db` and the following commands will rebuild it.

Run these commands from within the repository root, inside the devcontainer:
To rebuild the local Django database, run the [`bin/reset_db.sh`][reset-db] script from within the repository root,
inside the devcontainer:

```bash
bin/init.sh
bin/reset_db.sh
```

See the [Django Environment Variables](environment-variables.md#django) section for details about how to configure the local
database rebuild.

[core-models]: https://github.com/cal-itp/benefits/blob/dev/benefits/core/models.py
[django-load-initial-data]: https://docs.djangoproject.com/en/4.0/howto/initial-data/
[django-fixtures]: https://docs.djangoproject.com/en/5.0/topics/db/fixtures/
[django-load-initial-data]: https://docs.djangoproject.com/en/5.0/howto/initial-data/
[eligibility-server]: https://docs.calitp.org/eligibility-server
[data-migration]: https://github.com/cal-itp/benefits/tree/dev/benefits/core/migrations/0002_data.py
[helper-migration]: https://github.com/cal-itp/benefits/tree/dev/benefits/core/migrations/0003_data_migration_order.py
[init]: https://github.com/cal-itp/benefits/blob/dev/bin/init.sh
[reset-db]: https://github.com/cal-itp/benefits/blob/dev/bin/reset_db.sh
[sample-fixtures]: https://github.com/cal-itp/benefits/tree/dev/benefits/core/migrations/local_fixtures.json
50 changes: 48 additions & 2 deletions docs/configuration/environment-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@

The first steps of the Getting Started guide mention [creating an `.env` file][getting-started_create-env].

The sections below outline in more detail the application environment variables that you may want to override, and their purpose. In App Service, this is more generally called the ["configuration"][app-service-config].
The sections below outline in more detail the application environment variables that you may want to override, and their purpose.
In Azure App Services, this is more generally called the ["configuration"][app-service-config].

See other topic pages in this section for more specific environment variable configurations.

Expand Down Expand Up @@ -71,6 +72,39 @@ writable by the Django process._

By default, the base project directory (i.e. the root of the repository).

### `DJANGO_DB_FILE`

!!! info "Local configuration"

This setting only affects the app running on localhost

The name of the Django database file to use locally (during both normal app startup and for resetting the database).

By default, `django.db`.

### `DJANGO_DB_FIXTURES`

!!! info "Local configuration"

This setting only affects the app running on localhost

A path, relative to the repository root, of Django data fixtures to load when resetting the database.

The file must end in `fixtures.json` for the script to process it correctly.

By default, `benefits/core/migrations/local_fixtures.json`.

### `DJANGO_DB_RESET`

!!! info "Local configuration"

This setting only affects the app running on localhost

Boolean:

- `True` (default): deletes the existing database file and runs fresh Django migrations.
- `False`: Django uses the existing database file.

### `DJANGO_DEBUG`

!!! warning "Deployment configuration"
Expand All @@ -79,7 +113,7 @@ By default, the base project directory (i.e. the root of the repository).

!!! tldr "Django docs"

[Settings: `DEBUG`](https://docs.djangoproject.com/en/4.0/ref/settings/#debug)
[Settings: `DEBUG`](https://docs.djangoproject.com/en/5.0/ref/settings/#debug)

Boolean:

Expand Down Expand Up @@ -128,14 +162,26 @@ Django's primary secret, keep this safe!

### `DJANGO_SUPERUSER_EMAIL`

!!! info "Local configuration"

This setting only affects the app running on localhost

The email address of the Django Admin superuser created when resetting the database.

### `DJANGO_SUPERUSER_PASSWORD`

!!! info "Local configuration"

This setting only affects the app running on localhost

The password of the Django Admin superuser created when resetting the database.

### `DJANGO_SUPERUSER_USERNAME`

!!! info "Local configuration"

This setting only affects the app running on localhost

The username of the Django Admin superuser created when resetting the database.

### `DJANGO_TRUSTED_ORIGINS`
Expand Down

0 comments on commit 17f7164

Please sign in to comment.