Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: pass if table is already removed on upgrade #30017

Merged
merged 3 commits into from
Sep 3, 2024
Merged

fix: pass if table is already removed on upgrade #30017

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
13cdcb3
pass if table is already removed on upgrade
sadpandajoe Aug 26, 2024
39e5cfa
update migration scripts to be cleaner and use has_table
sadpandajoe Aug 27, 2024
4061377
pr change requests
sadpandajoe Sep 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions superset/migrations/versions/2024-08-13_15-17_39549add7bfc_remove_sl_table_columns_table.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,13 @@

def upgrade():
connection = op.get_bind()
if connection.dialect.name != "sqlite":
drop_fks_for_table("sl_table_columns")
op.drop_table("sl_table_columns")
try:
if connection.dialect.name != "sqlite":
drop_fks_for_table("sl_table_columns")
op.drop_table("sl_table_columns")
except sa.exc.NoSuchTableError:
# Table doesn't exist, so we can safely ignore this error
pass


def downgrade():
Expand Down
11 changes: 8 additions & 3 deletions superset/migrations/versions/2024-08-13_15-23_38f4144e8558_remove_sl_dataset_tables.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,14 @@

def upgrade():
connection = op.get_bind()
if connection.dialect.name != "sqlite":
drop_fks_for_table("sl_dataset_tables")
op.drop_table("sl_dataset_tables")

try:
if connection.dialect.name != "sqlite":
drop_fks_for_table("sl_dataset_tables")
op.drop_table("sl_dataset_tables")
except sa.exc.NoSuchTableError:
# Table doesn't exist, so we can safely ignore this error
pass


def downgrade():
Expand Down
11 changes: 8 additions & 3 deletions superset/migrations/versions/2024-08-13_15-27_e53fd48cc078_remove_sl_dataset_users.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,14 @@

def upgrade():
connection = op.get_bind()
if connection.dialect.name != "sqlite":
drop_fks_for_table("sl_dataset_users")
op.drop_table("sl_dataset_users")

try:
if connection.dialect.name != "sqlite":
drop_fks_for_table("sl_dataset_users")
op.drop_table("sl_dataset_users")
except sa.exc.NoSuchTableError:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sadpandajoe Would you mind adding a method called has_table to https://github.com/apache/superset/blob/master/superset/migrations/shared/utils.py to be reused in these and future migrations?

def has_table(table_name: str) -> bool:
    """
    Check if a table exists in the database.

    :param table_name: The table name
    :returns: True iff the table exists
    """

    insp = inspect(op.get_context().bind)
    return insp.has_table(table_name)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, and/or change drop_fks_for_table to receive a fail_when_missing=False

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even if we updated drop_fks_for_table we'd have to write something to drop table too, else it'll fail won't it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whatever makes sense, maybe drop_table(table_name, including_fks=[], only_if_it_exists=True) or maybe it's for another PR, but a common issue is a migration that fails at step 2 and when you rerun it it fails at step one since step one can only run once (say a create table). Personally think we could use more of the "drop-table-but-only-if-it-exists" and "create-table-but-drop-it-first-if-it-exists" type utils to avoid getting people tangled up. We've had many people open issue with the message of "step 1 one of the migration fails, here's the stacktrace", only to realize that the issue was with a subsequent step.

# Table doesn't exist, so we can safely ignore this error
pass


def downgrade():
Expand Down
11 changes: 8 additions & 3 deletions superset/migrations/versions/2024-08-13_15-29_a6b32d2d07b1_remove_sl_columns.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,14 @@

def upgrade():
connection = op.get_bind()
if connection.dialect.name != "sqlite":
drop_fks_for_table("sl_columns")
op.drop_table("sl_columns")

try:
if connection.dialect.name != "sqlite":
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also seems like the special handling for sqlite could be handled within drop_fks_for_table. More generally would be great to have migrations that are simpler / self-healing / free-of-engine-specific-logic through better utils.

drop_fks_for_table("sl_columns")
op.drop_table("sl_columns")
except sa.exc.NoSuchTableError:
# Table doesn't exist, so we can safely ignore this error
pass


def downgrade():
Expand Down
11 changes: 8 additions & 3 deletions superset/migrations/versions/2024-08-13_15-31_007a1abffe7e_remove_sl_tables.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,14 @@

def upgrade():
connection = op.get_bind()
if connection.dialect.name != "sqlite":
drop_fks_for_table("sl_tables")
op.drop_table("sl_tables")

try:
if connection.dialect.name != "sqlite":
drop_fks_for_table("sl_tables")
op.drop_table("sl_tables")
except sa.exc.NoSuchTableError:
# Table doesn't exist, so we can safely ignore this error
pass


def downgrade():
Expand Down
11 changes: 8 additions & 3 deletions superset/migrations/versions/2024-08-13_15-33_48cbb571fa3a_remove_sl_datasets.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,14 @@

def upgrade():
connection = op.get_bind()
if connection.dialect.name != "sqlite":
drop_fks_for_table("sl_datasets")
op.drop_table("sl_datasets")

try:
if connection.dialect.name != "sqlite":
drop_fks_for_table("sl_datasets")
op.drop_table("sl_datasets")
except sa.exc.NoSuchTableError:
# Table doesn't exist, so we can safely ignore this error
pass


def downgrade():
Expand Down
Loading