Fixes #31257 - support incremental update for debs

[m-bucher]

What are the changes introduced in this pull request?

Support copying of deb-packages in incremental Update of ContentViews .

Considerations taken when implementing this change?

Adding deb-packages does not support dependency-solving. Trying to do that will fail the incremental update.
#10734 ensures the dependency-solving option is ignored for deb-packages/-repositories.

What are the testing steps for this pull request?

No UI support; use the Content_view_versions / incremental_update REST-API endpoint for testing:

POST /katello/api/content_view_versions/incremental_update
...
{
  "add_content": {
    "deb_ids": [ <pkg_ids> ]
  }
}

[martux69]

For test reason I implement the changes manually on my test system and a first test was successfull, though the hammer cli output was the same as before:

[root@foreman ~]# hammer content-view version incremental-update --content-view-version-id 12 --lifecycle-environment-ids 2 --deb-ids 1685
[..........................................................................................................................................................................................................] [100%]
Content View: CV Debian 12 version 1.8
Added Content:
[root@foreman ~]#

But in the GUI the added package is seen now. It would be nice when the CLI output could be corrected.

I would appreciate a quick merge into the next possible release branch :-D

[ianballou]

Mind doing a rebase to get it back up to speed with the main branch?

[m-bucher]

@ianballou rebase is done
@martux69 I also managed to fix the output issue 😁

While testing, I saw that hammer does support specifiying the deb-package names with --debs.
But there is a problem where hammer tries to find the IDs for these packages by doing an API-GET with parameter name, which is not supported by itself. It has to be within the search-parameter.
It works for RPMs. No clue why not for Deb.
Is this an issue in hammer-cli-katello or with the /debs API endpoint implementation?

NOK (hammer): /katello/api/debs?name=dbus
OK: /katello/api/debs?search=name%3Ddbus

[martux69]

@ianballou
I would kindly ask for the reason of delay?

[ianballou]

@martux69 folks have been busy fixing bugs and prepping for the latest Foreman and Katello release. Plus once we got close to the 4.14 branching, we didn't want to introduce anything that would risk stability. Broken nightlies means skipped testing, which is no fun.

Now that we're branched though, we should be able to get it in soon. The changes aren't too complicated.

[ianballou]

One corner case that comes to mind that needs testing is incrementally updating a content view version with both Debs and RPMs.

[martux69]

Thanks for the information

[ianballou]

I've discovered a bug. I haven't determined why, but the changes here are causing content view publishing to fail. Specifically, it's triggering the following error:

The repository's publication is missing. Please run a 'complete sync' on Zoo. 

The change that likely caused this is how debs are now being added to the repo mapping for multi-copy workflows.

Here's the reproducer setup:

• Sync https://jlsherrill.fedorapeople.org/fake-repos/needed-errata/
• Sync https://fixtures.pulpproject.org/debian and use the ragnarok Release/Distribution
• Create a content view with both repositories that has dependency solving enabled.
  • This is crucial to reproducing the issue -- without dependency solving enabled, the CV publish succeeds.
• Create an RPM filter that only includes walrus - noarch - less that version 5
• Create a deb filter that only includes one of the packages (like frigg)
• Publish the content view

Hopefully the steps above reproduce it in your environment. There may be a simpler reproducer out there, but it's hard to say until we know what's going wrong.

I confirmed that removing these changes and republishing creates a working content view version.

[ianballou]

I found an issue when incrementally adding packages and debs together. The debs and packages requested do get added, but all of the remaining debs in my test repo got added even when trying to add only one of them.
To test it, I filtered out all of the debs but one in the test repo, and tried adding back only one of the other debs via hammer. The result was that 3 debs were added instead of one.

[ianballou]

It appears that incrementally adding RPMs still works as expected.

[m-bucher]

@ianballou That was the problem. It only did MultiCloneToVersion on either deb or yum, when it has to run for both.
Should be fixed now.

Thanks for testing that was a nice catch 👍

[ianballou]

Publishing with dependency solving is working again, awesome.

I'm still seeing an issue with extra debs being pulled in during the incremental update. Instead of just the one deb I want, all of the missing debs get pulled in.

[m-bucher]

@ianballou, sorry for the delay.

I'm still seeing an issue with extra debs being pulled in during the incremental update. Instead of just the one deb I want, all of the missing debs get pulled in.

I only had this behaviour when using hammer content-view version incremental-update with the --debs parameter. There seems to be an issue within hammer-cli, which does not correctly resolve the deb-packages (maybe it also has something to do with Katello's debs API-endpoint.
It works for me, if I specify the --deb-ids.

[m-bucher]

I only had this behaviour when using hammer content-view version incremental-update with the --debs parameter.

... and I found the issue for that 😎 Katello/hammer-cli-katello#960

[maximiliankolb]

docs: theforeman/foreman-documentation#3403

[ianballou]

Working great! Do you mind if we wait to merge until we're out of stabilization week and Katello 4.15 is branched?