Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UUIDs Have 36 Characters Not 32 #132

Closed
kylie-sre opened this issue Jun 20, 2024 · 2 comments · Fixed by #138
Closed

UUIDs Have 36 Characters Not 32 #132

kylie-sre opened this issue Jun 20, 2024 · 2 comments · Fixed by #138
Assignees

Comments

@kylie-sre
Copy link

Subject: The mapping for a Django UUIDField to a Redshift field is varchar(32). UUID's have 36 characters including the dashes.

Problem

  • There is a length mismatch for UUIDFields

Procedure to reproduce the problem

  • Create a model and add a UUIDField
  • Generate and apply a migration
  • Attempt to insert a UUID in to the resulting table without using the Django model (we are using kinesis to do writes and using this package to manage redshift schemas)

Error logs / results

When using the folowing query to get the error from redshift

SELECT error_message, column_name, start_time, * FROM sys_load_error_detail ORDER BY start_time DESC;

the following error is returned

String length exceeds DDL length

Expected results

the migration results in a usable field

@shimizukawa
Copy link
Member

Actually, django's UUIDField is implemented as a varchar(32).
https://github.com/django/django/blob/c5d196a65264136ee6795356871a29f3d22ec52f/django/db/models/fields/__init__.py#L2658-L2660

For this reason, I think it is natural that django-redshift-backend also implements UUIDs as 32-character strings.

However, python's uuid.UUID(hex=value) can restore UUID values with a 36-character string including hyphens.
Considering that Redshift is used in conjunction with other systems on AWS, I think it is more flexible to be able to handle 36 characters, so I think it is reasonable to support it.

https://github.com/django/django/blob/c5d196a65264136ee6795356871a29f3d22ec52f/django/db/models/fields/__init__.py#L2684-L2688

$ python
Python 3.10.14 (main, May 14 2024, 08:39:53) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import uuid
>>> u = uuid.uuid4()
>>> u
UUID('993376e9-2ee7-4f29-8578-4fa38f889b8d')
>>> str(u)
'993376e9-2ee7-4f29-8578-4fa38f889b8d'
>>> len(str(u))
36
>>> str(u.hex)
'993376e92ee74f2985784fa38f889b8d'
>>> len(str(u.hex))
32
>>> uuid.UUID(hex=str(u))
UUID('993376e9-2ee7-4f29-8578-4fa38f889b8d')
>>> uuid.UUID(hex=str(u.hex))
UUID('993376e9-2ee7-4f29-8578-4fa38f889b8d')

@shimizukawa shimizukawa self-assigned this Jul 22, 2024
shimizukawa added a commit that referenced this issue Jul 22, 2024
shimizukawa added a commit that referenced this issue Jul 22, 2024
[#132] use 36 length for UUIDFields to support including hyphens
@shimizukawa
Copy link
Member

This issue was implemented in PR #138.
Thanks for your contribution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants