Replace cloud-init validation with external script #2079
Conversation
I ran test installs on jammy, focal, noble, and oracular injected with a snap with the proposed changes. I tested with some userdata that would produce deprecation warnings on newer version of cloud-init e.g., chpasswd:
list: ""to make sure it would result in correct warnings in the debug log: I also tested that autoinstall with invalid userdata e.g., ch:
list: ""would cause failures: The strictness of the validation is unfortunately not consistent across all versions and relies on the behavior of the shipped cloud-init version. For example, the above faulty userdata section validates fine on the focal ISOs I tested. Likely due to not enforcing strict top level keys yet. I also confirmed that top-level key validation we perform when extracting autoinstall is skipped on older releases that don't support the but still works on later releases: Note: interactive installs may fail on Oracular due to an unrelated UI bug when passing autoinstall and having an interactive section. I just discovered it when re-running tests to get output. I'll open a bug shortly. |
Chris-Peterson444
force-pushed
the
refactor-cloud-init-validation
branch
3 times, most recently
from
935897b
to
02430e4
Compare
PR and bug for the issue: #2085 |
We need to define the SNAP environment variable in dry-run mode so the `system_scripts_env` function can setup a path to the system_scripts directory. This is necessary for the subiquity-legacy-cloud-init -validate script.
Rename validate_cloud_init_schema to validate_cloud_init_top_level_keys and rename get_schema_failure_keys to get_unknown_keys. validate_cloud_init_schema wasn't really validating the whole cloud-init schema, it's really just parsing the output for any top-level keys that failed to validate. Let's rename it to what it actually is doing: checking for bad top level keys. Also renames the original CloudInitSchemaValidationError to CloudInitSchemaTopLevelKeyError to reflect this change.
Chris-Peterson444
force-pushed
the
refactor-cloud-init-validation
branch
from
02430e4
to
cde2140
Compare
Extract cloud-init schema validation to a new system_script `subiquity-legacy-cloud-init-validate`. This is called "legacy" because, in the future, we likely want to rely on the cloud-init CLI directly to do schema validation, but there isn't a pressing need to rely on it now. For now, all releases will rely on the "legacy" script. Also let `get_unknown_keys` use `system_scripts_env` for the command environment to rely on the system cloud-init version.
The "schema" subcommand was introduced in cloud-init 22.2 (see cloud-init commit [0] ), which was released April 27, 2022. This means older releases of Focal and Jammy, which are currently supported LTSes, will fail this code path after we unstage cloud-init from the Subiquity snap. Add a check to skip this validation if a newer version of cloud-init is detected. [0] 3bcffacb216d683241cf955e4f7f3e89431c1491
Chris-Peterson444
force-pushed
the
refactor-cloud-init-validation
branch
from
cde2140
to
087d023
Compare
Slightly more changes for one PR than I'd like but everything sort of fell together:
Extract cloud-init schema validation to a new system_script
subiquity-legacy-cloud-init-validate. This is called "legacy" because, in the future, we likely want to rely on the cloud-init CLI directly to do schema validation, but there isn't a pressing need to rely on it now. For now, all releases will rely on the "legacy" script.Rename the old
validate_cloud_init_schemafunction tovalidate_cloud_init_top_level_keys(along with some other function and type renaming). The function wasn't really validating the whole cloud-init schema. It's really just parsing the output for any top-level keys that failed to validate.Skip the top-level key validation during autoinstall extraction in cloud-config on releases in which the "schema" subcommand wasn't available.
Add a default value of the SNAP environment variable for dry-run and tests. This lets code paths that rely on
system_scripts_envto not fail in dry-run or testing, which is mandatory now thatsubiquity-legacy-cloud-init-validateis used for cloud-init schema validation.