upgrade to do all tables, not just system tables

[grooverdan]

Might get some startup timeout on those that have a huge number of tables during upgrade.

WDYT @WhoAmI0501

Closes #562

[WhoAmI0501]

Hi and sorry for the late response.

What kind of timeoutes are we talking about? Docker Health Checks?

[grooverdan]

Delay starting - https://jira.mariadb.org/browse/MDEV-27060

While bitnami --force was the cause of this the measures of upgrade time are still relevant.

[WhoAmI0501]

Delay starting - https://jira.mariadb.org/browse/MDEV-27060

While bitnami --force was the cause of this the measures of upgrade time are still relevant.

I read the bug story multiple times and still dont really get it. The official MariaDB container should fix a problem with the entrypoint script of a inofficial third party container?

Well... on the one hand, imho this shouldn't be our problem here. On the other hand, its already fixed and it may be kind of asocial towards to the bitnami community, to remove the fix now.

What about adding a additional environment variable, to control, whether user tables should be also covered and leaving the system table parameter as the default behavior?

[WhoAmI0501]

Just seen that the bug story is also a few years old.

So i checked the current Bitnami repository and didnt found any --force parameter in the scripts (but may i also overseen something). You may also have a look on here: https://github.com/bitnami/containers/tree/main/bitnami/mariadb/11.2/debian-12

EDIT: They are actually not using this image anymore... so i guess its maximum unlikely, that they are using the entrypoint of this project here.

EDIT 2: But i nevermind stay on my suggestion from the previous post, since also users of this project may could experience a (breaking?) change, based on their used storage and table sizes.

[grooverdan]

Ignore the --force aspect. Because of the large number of user tables, 2hrs where required to perform the upgrade in this user's case.

What about adding a additional environment variable, to control, whether user tables should be also covered and leaving the system table parameter as the default behavior?

While it solves the immediate there are two cases (lots of user tables, and those that don't know), the remaining user question is how does a user ever now they need to upgrade their user tables?

There's an aspect that its not a new problem and providing an env variable is just like putting a upgrade-system-tables (or its inverse) in a configuration file, and abandonment of responsibility to the user. Its like the opt in version problem described in the issue, you don't want to find X years later that the version is incompatible (unlikely but mistakes happen when not tested).

So in short the competing goals are:

• achieve a quick startup time
• make users not worry their user tables are getting neglected in mariadb-upgrade

I'd rather:
a) the server covered the mariadb-upgrade of non-system tables automaticly because inline takes far too long.
b) optimizing / parallelizing of mariadb-upgrade of non-system tables to get down to a reasonable time to work (probably more achievable)

[WhoAmI0501]

I absolutely understand and support your point of view, as well as your goals.

b) optimizing / parallelizing of mariadb-upgrade of non-system tables to get down to a reasonable time to work (probably more achievable)

As we already read, theres sometimes an disk I/O bottleneck, which leads the upgrade task to take very long. So based on that fact, i guess, that parallelizing may wouldnt make it better in any way.

In either way, it seems like this topic should be discussed in the MariaDB repository/project itself? Unfortunately, i cannot contribute anything than my view as a user to this problem, since i never had any contact with the code of MariaDB and its tooling.

[grooverdan]

I absolutely understand and support your point of view, as well as your goals.

b) optimizing / parallelizing of mariadb-upgrade of non-system tables to get down to a reasonable time to work (probably more achievable)

As we already read, theres sometimes an disk I/O bottleneck, which leads the upgrade task to take very long. So based on that fact, i guess, that parallelizing may wouldnt make it better in any way.

It reads a lot of tables definitions but doesn't dive into the entire table contents.

In either way, it seems like this topic should be discussed in the MariaDB repository/project itself?

yep.

Thankfully @vaintroub provided me with a reference implementation.

Unfortunately, i cannot contribute anything than my view as a user to this problem, since i never had any contact with the code of MariaDB and its tooling.

Your view as a user is valuable enough :-)