Image refresh for rhel-9-6

[cockpituous]

Image refresh for rhel-9-6

• image-refresh rhel-9-6

[cockpituous]

image-refresh rhel-9-6 done: https://github.com/cockpit-project/bots/commits/image-refresh-rhel-9-6-20241017-225558

[cockpituous]

Success. Log: https://cockpit-logs.us-east-1.linodeobjects.com/image-refresh-rhel-9-6-e014a6d3-20241017-224107/log.html

[martinpitt]

This kdump nfs failure may be real -- there's no new kdump, but a new kernel. Retrying once to compare, then investigating.

[martinpitt]

There's indeed a regression. On the main image, kdump.service fails properly with an invalid config:

× kdump.service - Crash recovery kernel arming
     Loaded: loaded (/usr/lib/systemd/system/kdump.service; enabled; preset: enabled)
     Active: failed (Result: exit-code) since Fri 2024-10-18 02:26:05 EDT; 3min 14s ago
   Duration: 5.194s
    Process: 1911 ExecStart=/usr/bin/kdumpctl start (code=exited, status=1/FAILURE)
   Main PID: 1911 (code=exited, status=1/FAILURE)
        CPU: 464ms

Oct 18 02:26:03 rhel-9-6-127-0-0-2-2201 systemd[1]: Starting Crash recovery kernel arming...
Oct 18 02:26:04 rhel-9-6-127-0-0-2-2201 kdumpctl[1934]: kdump: Detected change(s) in the following file(s):  /etc/kdump.conf
Oct 18 02:26:05 rhel-9-6-127-0-0-2-2201 kdumpctl[1934]: kdump: Rebuilding /boot/initramfs-5.14.0-516.el9.x86_64kdump.img
Oct 18 02:26:05 rhel-9-6-127-0-0-2-2201 kdumpctl[2426]: kdump: Dump path "dumps" does not exist in dump target "10.111.113.2:/srv/kdump"
Oct 18 02:26:05 rhel-9-6-127-0-0-2-2201 kdumpctl[1934]: kdump: mkdumprd: failed to make kdump initrd
Oct 18 02:26:05 rhel-9-6-127-0-0-2-2201 kdumpctl[1934]: kdump: Starting kdump: [FAILED]
Oct 18 02:26:05 rhel-9-6-127-0-0-2-2201 systemd[1]: kdump.service: Main process exited, code=exited, status=1/FAILURE
Oct 18 02:26:05 rhel-9-6-127-0-0-2-2201 systemd[1]: kdump.service: Failed with result 'exit-code'.
Oct 18 02:26:05 rhel-9-6-127-0-0-2-2201 systemd[1]: Failed to start Crash recovery kernel arming.

but on the new image it succeeds despite seeing the error:

● kdump.service - Crash recovery kernel arming
     Loaded: loaded (/usr/lib/systemd/system/kdump.service; enabled; preset: enabled)
     Active: active (exited) since Fri 2024-10-18 02:28:51 EDT; 30s ago
    Process: 1763 ExecStart=/usr/bin/kdumpctl start (code=exited, status=0/SUCCESS)
   Main PID: 1763 (code=exited, status=0/SUCCESS)
        CPU: 900ms

Oct 18 02:28:48 rhel-9-6-127-0-0-2-2201 systemd[1]: Starting Crash recovery kernel arming...
Oct 18 02:28:49 rhel-9-6-127-0-0-2-2201 kdumpctl[1786]: kdump: Detected change(s) in the following file(s):  /etc/kdump.conf
Oct 18 02:28:50 rhel-9-6-127-0-0-2-2201 kdumpctl[1786]: kdump: Rebuilding /boot/initramfs-5.14.0-518.el9.x86_64kdump.img
Oct 18 02:28:51 rhel-9-6-127-0-0-2-2201 kdumpctl[2276]: kdump: Dump path "dumps" does not exist in dump target "10.111.113.2:/srv/kdump"
Oct 18 02:28:51 rhel-9-6-127-0-0-2-2201 kdumpctl[1786]: kdump: mkdumprd: failed to make kdump initrd
Oct 18 02:28:51 rhel-9-6-127-0-0-2-2201 kdumpctl[1786]: kdump: kexec: loaded kdump kernel
Oct 18 02:28:51 rhel-9-6-127-0-0-2-2201 kdumpctl[1786]: kdump: Starting kdump: [OK]
Oct 18 02:28:51 rhel-9-6-127-0-0-2-2201 kdumpctl[1786]: kdump: Notice: No vmcore creation test performed!
Oct 18 02:28:51 rhel-9-6-127-0-0-2-2201 systemd[1]: Finished Crash recovery kernel arming.

Same result with kdumpctl start, it exits with 0 on the new image now.

This doesn't actually need a running NFS server, can just be tested with nfs unknown.local:/srv/kdump. Reproducer on clean image:

echo 'nfs unknown.local:/srv/kdump' >> /etc/kdump.conf 
kdumpctl stop
kdumpctl start; echo $?

With old kernel-core-5.14.0-516.el9.x86_64 and kexec-tools-2.0.27-16.el9_5.1.x86_64 it correctly fails:

kdump: Detected change(s) in the following file(s):  /etc/kdump.conf
kdump: Rebuilding /boot/initramfs-5.14.0-516.el9.x86_64kdump.img
mount.nfs: Failed to resolve server unknown.local: Name or service not known
kdump: Failed to mount unknown.local:/srv/kdump for kdump preflight check. Please make sure nfs-utils has been installed.
kdump: mkdumprd: failed to make kdump initrd
kdump: Starting kdump: [FAILED]
1

with kexec-tools-2.0.27-17.el9.x86_64 it wrongly succeeds:

kdump: Detected change(s) in the following file(s):  /etc/kdump.conf
kdump: Rebuilding /boot/initramfs-5.14.0-518.el9.x86_64kdump.img
mount.nfs: Failed to resolve server unknown.local: Name or service not known
kdump: Failed to mount unknown.local:/srv/kdump for kdump preflight check. Please make sure nfs-utils has been installed.
kdump: mkdumprd: failed to make kdump initrd
kdump: kexec: loaded kdump kernel
kdump: Starting kdump: [OK]
kdump: Notice: No vmcore creation test performed!
0

It's enough to just update that one package.

This was changed in https://gitlab.com/redhat/centos-stream/rpms/kexec-tools/-/merge_requests/245 and it only refers to https://issues.redhat.com/browse/RHEL-32060 which is unrelated. So this doesn't look like an intentional behaviour change, but a bad backport.

Reported to https://issues.redhat.com/browse/RHEL-63047