Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stage data adds duplicate key #614

Closed
jmgilman opened this issue Nov 1, 2023 · 4 comments
Closed

Stage data adds duplicate key #614

jmgilman opened this issue Nov 1, 2023 · 4 comments

Comments

@jmgilman
Copy link
Collaborator

jmgilman commented Nov 1, 2023

During the migration, stage-specific data is added. Using the dev cluster as an example, the SQL files that get imported are here. The test fund event that is added in this section uses a row_id of 0 which conflicts with the historical data that was added in a previous step.

The result is the migration fails with:

Adding stage specific data from ./stage_data/dev/00001_testfund_event.sql
psql:./stage_data/dev/00001_testfund_event.sql:54: ERROR:  duplicate key value violates unique constraint "event_pkey"

We should give the test event a very high row ID (i.e., 9999). If we do a UPSERT, it will end up overwriting some of the historical data, and the API will return incorrect data.

@kukkok3
Copy link
Contributor

kukkok3 commented Nov 2, 2023

@jmgilman could this be due to my PR with the fund 11 data that was not merged yet when you did the migration?

@kukkok3
Copy link
Contributor

kukkok3 commented Nov 2, 2023

Pr with fund11 entry #612
right now this is failing for what I think is a bug in this earthfile where stage data are copied even when data=test. Not sure how to handle this, I think I'll move coping the stage data with the historic data. A ARG array type would be perfect to manage this but it is not supported (yet earthly/earthly#1890) by earthly

@jmgilman
Copy link
Collaborator Author

jmgilman commented Nov 8, 2023

There's still an issue here, as at the moment when the migration is run on a eventdb with existing data, it fails because of duplicate row IDs. The SQL script needs to be modified to handle this situation.

@FelipeRosa
Copy link
Contributor

Fixed that in #624

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants