Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Dockerize the server and its supporting services #144

Open
wants to merge 5 commits into
base: develop
Choose a base branch
from

Conversation

berkes
Copy link
Contributor

@berkes berkes commented Sep 25, 2024

This is a PR to discuss the solution to the issue with migrations.

It is unclear why the migrations keep failing, but they won't pass if the recipient_identifier is
a varchar(712). At least not in the dockerized version of mysql 8.4. It fails on creating the
index, as the index is larger than 3072.

The database is utf8, and collation is correct.

When we set this column to be only 512 long, as it is at the end of running all migrations anyway,
the index can be created.

The failure is:

  Applying issuer.0043_auto_20180614_0949... OK
  Applying issuer.0044_auto_20180713_0658...Traceback (most recent call last):
  ...
MySQLdb.OperationalError: (1071, 'Specified key was too long; max key length is 3072 bytes')

This PR proposes two alternative solutions to this:

  1. Remove the index creation from issuer.0043_auto_20180614_0949 and add it in a new migration at the end
  2. Modify existing migrations and ensure the recipient_identifier is varchar(512).

The option 1 was reverted and can only be witnessed in the git history of this PR. It has a severe
downside: production builds, and other builds will run this new migration, but
will fail, as the index exsists. Neither MySQL nor django migrations have a
mechanism to "CREATE INDEX .. IF NOT EXISTS" like postgres or mariadb do.

Option 2 has downside that it alters migrations that have already been applied. Which is normally
a bad practice.

  • fix: Fix unclosed quote in example env var file
  • fix: Add compound index when the indexed columns have the right size
  • feat: Docker compose with required services
  • Revert "fix: Add compound index when the indexed columns have the right size"
  • fix: Allow migrations to create indexes that are guaranteed to be under 3072

At the place of migration 0043, the columns together are too long. So it
fails with
> 'Specified key was too long; max key length is 3072 bytes

After most migrations are ran, the columns are resized and do fit in
that index size limitation.

https://dev.mysql.com/doc/refman/8.0/en/innodb-limits.html
>  The index key prefix length limit is 3072 bytes for InnoDB tables that use DYNAMIC or COMPRESSED row format.

We already have the "max" size possible, going over the 3072 is not
possible.
This
* Builds an image for our app
* Uses mysql, mailhog and memcached services
* Exposes env vars to the app to connect to these services (WIP: db
  works, mailhog and memcached not
* Runs the migrations, then the app
* Mounts the host machine (dev machine) in the container as source code
* Runs a dev server in container in watch mode
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant