Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes and improvements for package installation during image builds #77

Open
wants to merge 26 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
50eeca5
packages: Restore separate block for client base image tasks in Ubuntu
dpward Mar 10, 2024
d3aa604
packages: Only run installation tasks when creating base images
dpward Mar 10, 2024
640ac6f
packages: Make tasks for client base image consistent
dpward Mar 10, 2024
89ad3e7
packages: Make tasks for client development image consistent
dpward Mar 10, 2024
4488a74
packages: Fix configuration of backports repository in Debian
dpward Mar 10, 2024
3f1e76a
packages: Only update apt cache when needed
dpward Mar 10, 2024
8d9702c
facts: Import tasks for Debian into Ubuntu
dpward Mar 10, 2024
791df6e
packages: Import tasks for Debian into Ubuntu
dpward Mar 10, 2024
b8163fe
packages: Enable GPG signature check for base RPM repositories
dpward Mar 10, 2024
551db92
packages: Remove set_fact task for extended_packageset
dpward Mar 10, 2024
63aa542
packages: Remove unneeded test for dnf.conf
dpward Mar 10, 2024
e684f50
packages: Modify dnf.conf in ground base image only
dpward Mar 10, 2024
d0513fe
packages: Remove redundant task to install core DNF plugins in CentOS
dpward Mar 10, 2024
a049403
packages: Fix configuration of additional repositories in CentOS and …
dpward Mar 10, 2024
8d3ce4d
packages: Use yum_repository module to configure CentOS buildroot
dpward Mar 10, 2024
b0e2cd0
facts: Consolidate tasks for CentOS 8 and 9
dpward Mar 10, 2024
28dfab3
packages: Consolidate tasks for CentOS 8 and 9
dpward Mar 10, 2024
c67ce38
packages: Import tasks for CentOS into RHEL
dpward Mar 10, 2024
8d2ea58
ansible: Replace group variables containing IPA package names
dpward Mar 10, 2024
0e780c4
ansible: Remove unneeded filter plugin
dpward Mar 10, 2024
3996c76
build: Do not remove dnf-5
dpward Mar 10, 2024
4c976ba
build: Stop the base-ground container cleanly
dpward Mar 10, 2024
72c04a5
build: Detect package manager from base image
dpward Mar 10, 2024
5a5da45
build: Re-use package manager cache when creating base images
dpward Mar 10, 2024
fd7125b
build: Include both devel images in comment describing layers
dpward Mar 10, 2024
ac5cf15
build: Remove obsolete variable from output
dpward Mar 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 0 additions & 19 deletions src/ansible/filter_plugins/distro.py

This file was deleted.

10 changes: 0 additions & 10 deletions src/ansible/group_vars/all
Original file line number Diff line number Diff line change
Expand Up @@ -57,16 +57,6 @@ user: {
}
}

freeipa_packages: {
server: [ freeipa-server, freeipa-server-dns, freeipa-server-trust-ad ],
client: [ freeipa-client, ]
}

ipa_packages: {
server: [ ipa-server-dns, ipa-server, ipa-server-trust-ad ],
client: [ ipa-client, ]
}

join_samba: yes
join_ipa: yes
join_ldap: yes
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
- name: 'Facts are the same as in Fedora'
include_tasks: 'Fedora.yml'

- name: Set distribution specific facts
- name: Set distribution specific facts for CentOS 8
set_fact:
passkey_support: No
when: ansible_distribution_major_version < '9'
6 changes: 0 additions & 6 deletions src/ansible/roles/facts/tasks/CentOS9.yml

This file was deleted.

2 changes: 1 addition & 1 deletion src/ansible/roles/facts/tasks/Fedora.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,6 @@
buildroot: yes
debuginfo: yes
passkey_support: Yes
ipa: '{{ freeipa_packages }}'
ipa_package: freeipa
ca_trust_dir: /etc/pki/ca-trust/source/anchors/
ca_trust_update: update-ca-trust
15 changes: 3 additions & 12 deletions src/ansible/roles/facts/tasks/RedHat.yml
Original file line number Diff line number Diff line change
@@ -1,17 +1,8 @@
- name: 'Facts are the same as in CentOS {{ ansible_distribution_major_version }}'
include_tasks: '{{ include_centos }}'
loop_control:
loop_var: include_centos
with_first_found:
- files: '{{ "CentOS" | distro_includes(ansible_distribution_major_version) }}'
- name: 'Facts are the same as in CentOS'
include_tasks: 'CentOS.yml'

- name: Set distribution specific facts
set_fact:
buildroot: no
debuginfo: no
ipa: '{{ ipa_packages }}'

- name: Set distribution specific facts for RHEL 7
set_fact:
passkey_support: No
when: ansible_distribution_major_version == '7'
ipa_package: ipa
10 changes: 3 additions & 7 deletions src/ansible/roles/facts/tasks/Ubuntu.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,6 @@
- name: 'Facts are the same as in Debian'
include_tasks: 'Debian.yml'

- name: Set distribution specific facts
set_fact:
systemd:
services:
kadmin: krb5-admin-server.service
krb5kdc: krb5-kdc.service
sshd: ssh.service
passkey_support: Yes
ca_trust_dir: /etc/ssl/certs
ca_trust_update: update-ca-certificates
8 changes: 2 additions & 6 deletions src/ansible/roles/facts/tasks/main.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,2 @@
- name: 'Include distribution specific tasks [{{ ansible_distribution }} {{ ansible_distribution_major_version }}]'
include_tasks: '{{ include_file }}'
loop_control:
loop_var: include_file
with_first_found:
- files: '{{ ansible_distribution | distro_includes(ansible_distribution_major_version) }}'
- name: 'Include distribution specific tasks [{{ ansible_distribution }}]'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, keep the plugin. It may not be needed now, but it most certainly will be needed in the future.

include_tasks: '{{ ansible_distribution }}.yml'
24 changes: 24 additions & 0 deletions src/ansible/roles/packages/tasks/CentOS.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
- name: Install additional repositories
block:
- name: Install buildroot repository
yum_repository:
name: buildroot
description: "CentOS Stream $releasever - Buildroot"
baseurl: "https://kojihub.stream.centos.org/kojifiles/repos/c{{ ansible_distribution_major_version }}s-build/latest/$basearch"
gpgcheck: false
when: buildroot

- name: Enable IdM module
command: dnf -y module enable idm:DL1
when: "ansible_distribution_major_version == '8'"

- name: Install EPEL repository
dnf:
state: present
name: "https://dl.fedoraproject.org/pub/epel/epel-release-latest-{{ ansible_distribution_major_version }}.noarch.rpm"
disable_gpg_check: yes
when: extended_packageset
when: "'base_ground' in group_names"

- name: 'Packages are the same as in Fedora'
include_tasks: 'Fedora.yml'
32 changes: 0 additions & 32 deletions src/ansible/roles/packages/tasks/CentOS8.yml

This file was deleted.

28 changes: 0 additions & 28 deletions src/ansible/roles/packages/tasks/CentOS9.yml

This file was deleted.

61 changes: 31 additions & 30 deletions src/ansible/roles/packages/tasks/Debian.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
- name: Install systemd and common tools
apt:
state: present
update_cache: yes
cache_valid_time: 3600
name:
- bash-completion
- bind9utils
Expand Down Expand Up @@ -39,73 +39,75 @@
when: "'base_ground' in group_names"

- name: Enable backports repo to install freeipa
block:
- name: Enable backports repo
copy:
dest: /etc/apt/sources.list.d/backports.list
content: "deb http://deb.debian.org/debian bullseye-backports main"
owner: root
group: root
mode: '0644'
apt_repository:
repo: deb http://deb.debian.org/debian bullseye-backports main
state: present
when:
- ansible_distribution == 'Debian'
- ansible_distribution_major_version == '11'

- name: Install packages for client base image
block:
- name: Install SSSD and its dependencies
- name: Install IPA client
apt:
state: present
cache_valid_time: 3600
name: freeipa-client
- name: Install SSSD
apt:
state: present
update_cache: yes
cache_valid_time: 3600
name:
- adcli
- freeipa-client
- nfs-common
- nslcd
- packagekit
- realmd
- slapd
- sssd
- sssd-*
- name: Install test dependencies on client
apt:
state: present
update_cache: yes
cache_valid_time: 3600
name:
- adcli
- augeas-tools
when: "'base_client' in group_names or 'client' in group_names"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This and similar ones are unwanted change, this is there specifically for use in other ci where this is applied to VMs instead of containers and mixing in the base_xxx and xxx.
we got inventory that has group just client.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's change the other CI for the VMs, please, so that the VM host is in both the base_xxx and xxx groups. We do not need this task to run twice for the containers; it slows down the build.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would it run twice?

- nfs-common
- nslcd
- packagekit
- realmd
- slapd
when: "'base_client' in group_names"

- name: Install packages for NFS base image
block:
- name: Install NFS
apt:
state: present
update_cache: yes
cache_valid_time: 3600
name:
- nfs-kernel-server
when: "'base_nfs' in group_names or 'nfs' in group_names"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This and similar ones are unwanted change, this is there specifically for use in other ci where this is applied to VMs instead of containers and mixing in the base_xxx and xxx.
we got inventory that has group just nfs.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's change the other CI for the VMs, please, so that the VM host is in both the base_xxx and xxx groups. We do not need this task to run twice for the containers; it slows down the build.

when: "'base_nfs' in group_names"

- name: Install packages for KDC base image
block:
- name: Install KDC
apt:
state: present
update_cache: yes
cache_valid_time: 3600
name:
- krb5-admin-server
- krb5-config
- krb5-kdc
when: "'base_kdc' in group_names or 'kdc' in group_names"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This and similar ones are unwanted change, this is there specifically for use in other ci where this is applied to VMs instead of containers and mixing in the base_xxx and xxx.
we got inventory that has group just kdc.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's change the other CI for the VMs, please, so that the VM host is in both the base_xxx and xxx groups. We do not need this task to run twice for the containers; it slows down the build.

when: "'base_kdc' in group_names"

- name: Install packages for Keycloak base image
block:
- name: Install ca-certificates-java first to avoid dep issues
apt:
state: present
update_cache: yes
cache_valid_time: 3600
name:
- ca-certificates-java
- name: Install Keycloak dependencies
apt:
state: present
update_cache: yes
cache_valid_time: 3600
name:
- openssl
- unzip
Expand All @@ -117,10 +119,10 @@

- name: Install additional packages for client development image
block:
- name: Install SSSD build and integration tests dependencies
- name: Install packages required for integration tests
apt:
state: present
update_cache: yes
cache_valid_time: 3600
name:
- adcli
- autoconf
Expand Down Expand Up @@ -175,7 +177,6 @@
- libsemanage-dev
- libsmbclient-dev
- libssl-dev
- libssl-dev
- libsystemd-dev
- libtalloc-dev
- libtdb-dev
Expand Down Expand Up @@ -206,7 +207,7 @@
- name: Install additional python packages
apt:
state: present
update_cache: yes
cache_valid_time: 3600
name:
- python3-flaky
when: "'client_devel' in group_names"
Loading