Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use podman #16

Closed
wants to merge 23 commits into from
Closed
Show file tree
Hide file tree
Changes from 22 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
all: sm srcds deploy relay-deploy

base:
@ansible-playbook --limit prod playbooks/base-tf2server.yml --extra-vars "clean=$(CLEAN)"
@ansible-playbook --limit prod playbooks/base-tf2server.yml

sm:
@ansible-playbook --limit prod playbooks/sourcemod.yml
Expand Down
29 changes: 12 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,17 +10,13 @@ Ansible requires Linux. If you're running Windows, you'll need to set up **[WSL]
These playbooks only work with [systemd](https://systemd.io/)-based hosts, which is the default for most Linux distributions.

### Mandatory setup
1. Assuming you're using Ubuntu (WSL default), install Python, Ansible, and Docker using `sudo apt install -y python3 ansible docker.io`
1. Assuming you're using Ubuntu (WSL default), install Ansible using `sudo apt install ansible`
> NixOS users: a flake is provided for convenience, just run `nix develop`.

2. On all `tf2` hosts (dedicated servers):
- 1. `sudo apt install -y docker.io docker-compose-v2` - Install Docker and Docker-Compose.
- 2. `sudo useradd -UmG docker tf2server` - Create the `tf2server` user with the `docker` role.
- 3. `sudo loginctl enable-linger tf2server` - Enable systemd service lingering.

3. On your `metrics` host:
- 1. `sudo apt install -y docker.io` - Install Docker.
- 2. `sudo useradd -UmG docker tf2server` - Create the `tf2server` user with the `docker` role.
2. On all hosts (dedicated servers):
- 1. `sudo apt install -y podman kubernetes` - Install Podman and Kubernetes.
- 2. `sudo useradd -Um tf2server` - Create the `tf2server`.
- 3. `sudo loginctl enable-linger tf2server` - Enable systemd service lingering on the user.

Of course, don't forget to put your SSH public key in `/home/tf2server/.ssh/authorized_keys`.

Expand All @@ -42,7 +38,7 @@ There is a pre-commit hook that you should enable to ensure you don't commit any
> Configure the rest to your liking.
3. `make cycle` - Generate initial ssh keys for secure SB++ DB connection.
-- This directive also (re-)starts the SB++ network, so no need to run `make sbpp`.
4. `make base` - Build the base Team Fortress 2 server Docker image on every `tf2` host.
4. `make base` - Build the base Team Fortress 2 server image on every `tf2` host.
5. `make sm` - Distribute and build SourceMod.
6. `make srcds` - Build instance images.
7. `make deploy` - Start containers & setup crontab for the TF2 servers.
Expand All @@ -54,9 +50,9 @@ There is a pre-commit hook that you should enable to ensure you don't commit any
**`make base` can take a long time!**
This is because it's downloading the full TF2 server.<br>
**It is possible that Ansible will *SILENTLY* time out while waiting if it takes too long!**<br>
Watch for `jackavery/base-tf2server` to show up in `docker image ls`. If it's there, you can Ctrl+C Ansible.
Watch for `jackavery/base-tf2server` to show up in `podman image ls`. If it's there, you can Ctrl+C Ansible.

**Do not re-run `sbpp-install` once you've installed SB++!**
**Do not re-run `make sbpp-install` once you've installed SB++!**
This is because it starts the container with intent to install SourceBans++.<br>
**This also means it will wipe your existing data, including user punishments and server configuration.**

Expand All @@ -71,14 +67,13 @@ Using the `discord_relay` plugin (depends on `discord` plugin, uses a webhook in
### 👏 Hands-off Maintenance
This playbook set comes with a robust and simple auto-update script that ensures your servers update as soon as a new version of Team Fortress 2 is available. This is done by rebuilding from *scratch*, instead of updating the existing image, so as to maintain [image immutability](https://kubernetes.io/blog/2018/03/principles-of-container-app-design/). Servers are restarted once daily at a time set per-host as to prevent [server clock errors](https://www.youtube.com/watch?v=RdTJHVG_IdU). The relay plugin facilitates chat logs with user IDs for use by moderators for moderation decisions. This leads to a seamless 24/7 server experience with quality-of-life for your moderation team.

### 🛠️ Docker and Ansible, confined scope
**ansible-tf2network** uses Ansible to provide a user-friendly and extensive configuration interface, and Docker to make your deploys consistent regardless of host. If you upgrade or move hosts, all you need to do is point your host record in `inventory.yml` at the new IP.
### 🛠️ Podman and Ansible, confined scope, highly secure
**ansible-tf2network** uses Ansible to provide a user-friendly and extensive configuration interface, and Podman to make your deploys consistent regardless of host. If you upgrade or move hosts, all you need to do is point your host record in `inventory.yml` at the new IP. Multiple measures have been put in place to ensure

Since the playbooks keep their activity contained within the `tf2server` user folder with *no* actions performed as root, cleaning up a host after using **ansible-tf2network** can be done with these commands:
1. `userdel -r tf2server` - Delete their user
2. `docker stop [...]` - Stop the containers (do this for all servers)
3. `docker container prune` - Remove the containers
4. `docker image prune -a` - Remove all unused Docker images
2. `systemctl daemon-reload` - Reload systemd (to teardown their containers)
3. `podman system prune -a` - Prune Podman

### 📚 Default, Ruleset, and Instance level configuration
**ansible-tf2network** server configuration has 3 scopes: default, ruleset, and instance. Overriding configuration from outer scopes is possible within inner scopes, e.g., ruleset config overrides default config, and instance config overrides both.
Expand Down
34 changes: 16 additions & 18 deletions manifest.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,7 @@ def load_ansible_variables() -> dict:
logging.info("Discovering secrets...")
secrets: list[str] = []
secrets += [
file
for file in os.listdir("host_vars")
if not file.endswith(".sample")
file for file in os.listdir("host_vars") if not file.endswith(".sample")
]
logging.debug(secrets)

Expand Down Expand Up @@ -94,9 +92,7 @@ def load_ansible_variables() -> dict:
secrets_file.write(original)

else:
instances = {
instance["name"]: instance for instance in vars["instances"]
}
instances = {instance["name"]: instance for instance in vars["instances"]}
del vars["instances"]
host_vars[host]["host"] = vars
host_vars[host]["vars"] = instances
Expand Down Expand Up @@ -126,18 +122,20 @@ def create_manifest(inventory: dict, globals: dict, vars: dict) -> dict:
else:
stv_enabled = globals["stv_enabled"]

instances.append({
"internal_name": instance_name,
"hostname": vars[host]["vars"][instance_name]["hostname"],
"ip": f"{ip}:{base_port + (vars[host]['host']['srcds_reserve_ports'] * i)}",
"rcon_pass": vars[host]["secret"][instance_name]["rcon_pass"],
"relay_channel": vars[host]["vars"][instance_name].get("relay_channel", None),
"stv_enabled": stv_enabled
})

manifest = {
"hosts": instances
}
instances.append(
{
"internal_name": instance_name,
"hostname": vars[host]["vars"][instance_name]["hostname"],
"ip": f"{ip}:{base_port + (vars[host]['host']['srcds_reserve_ports'] * i)}",
"rcon_pass": vars[host]["secret"][instance_name]["rcon_pass"],
"relay_channel": vars[host]["vars"][instance_name].get(
"relay_channel", None
),
"stv_enabled": stv_enabled,
}
)

manifest = {"hosts": instances}

logging.debug(manifest)
return manifest
Expand Down
2 changes: 1 addition & 1 deletion playbooks/base-tf2server.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
- name: Build (or update) base server image on host
- name: Build base server image on host
hosts: tf2
roles:
- base-tf2server
27 changes: 17 additions & 10 deletions playbooks/relay-deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,18 +9,25 @@
ansible.posix.synchronize:
src: "{{ playbook_dir }}/../manifest.yml"
dest: ~/{{ network_shortname }}-manifest.yml
- name: Start container
community.docker.docker_container:

- name: Create quadlet
containers.podman.podman_container:
name: "{{ network_shortname }}_relay"
image: rust:1.74-slim-bookworm
state: started
restart: true
restart_policy: "unless-stopped"
image: docker.io/library/rust:1.74-slim-bookworm
state: quadlet
env:
DISCORD_TOKEN: "{{ discord_relay_bot_token }}"
RCON_USERS: "{{ discord_relay_rcon_users | map('string') | join(':') }}"
volumes:
- /home/tf2server/build-relay/target/release/ansible-tf2network-relay:/main
- /home/tf2server/{{ network_shortname }}-manifest.yml:/manifest.yml
command: /main
- "/home/tf2server/build-relay/target/release/ansible-tf2network-relay:/main"
- "/home/tf2server/{{ network_shortname }}-manifest.yml:/manifest.yml"
command: /main
quadlet_options:
- |
[Install]
WantedBy=multi-user.target default.target

- name: systemd user daemon reload
ansible.builtin.systemd_service:
scope: user
daemon_reload: true
2 changes: 1 addition & 1 deletion playbooks/relay.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
- name: Deploy Discord relay
- name: Build Discord relay
hosts: metrics
roles:
- relay
6 changes: 3 additions & 3 deletions roles/admins/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@
loop: "{{ targets.instances }}"

- name: Move into container
community.docker.docker_container_copy_into:
containers.podman.podman_container_copy:
container: "srcds-{{ item.name }}"
path: ~/build/{{ item.name }}/tf/addons/sourcemod/configs/admins_simple.ini
container_path: /home/steam/tf-dedicated/tf/addons/sourcemod/configs/admins_simple.ini
src: ~/build/{{ item.name }}/tf/addons/sourcemod/configs/admins_simple.ini
dest: /home/steam/tf-dedicated/tf/addons/sourcemod/configs/admins_simple.ini
loop: "{{ targets.instances }}"
delegate_to: "{{ ansible_host }}"

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM cm2network/steamcmd:root
FROM docker.io/cm2network/steamcmd:root

# Install dependencies
RUN dpkg --add-architecture i386 && \
Expand Down
16 changes: 5 additions & 11 deletions roles/base-tf2server/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,17 +18,11 @@
recursive: true

- name: Build base image
community.docker.docker_image:
containers.podman.podman_image:
name: jackavery/base-tf2server
tag: latest
source: build
force_tag: true
force_source: true
path: ~/build/base
state: build
build:
nocache: true
pull: true
rm: true
dockerfile: Dockerfile
platform: linux/amd64
path: ~/build/base
state: present
cache: false
force_rm: true
15 changes: 8 additions & 7 deletions roles/deploy/files/updater.sh
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@ main() {

echo ""
echo "[$(date +'%D %H:%M:%S')] ($$) Checking..."
LATEST=$(docker run --quiet --pull always --rm --entrypoint=/bin/bash cm2network/steamcmd -c "timeout $UPDATE_TIMEOUT ./steamcmd.sh +login anonymous +app_info_update 232250 +app_info_print 232250 +quit" | perl -n -e'/"buildid"\s+"([ 0-9]+)"/ && print "$1\n"' | head -n1)

LATEST=$(podman run --quiet --pull always --rm --entrypoint=/bin/bash cm2network/steamcmd -c "timeout $UPDATE_TIMEOUT ./steamcmd.sh +login anonymous +app_info_update 232250 +app_info_print 232250 +quit" | perl -n -e'/"buildid"\s+"([ 0-9]+)"/ && print "$1\n"' | head -n1)

if [ ! -f current_tf2_buildid ]; then
echo "[$(date +'%D %H:%M:%S')] No current version saved, saving latest as current and doing nothing!"
Expand All @@ -52,7 +53,7 @@ main() {
elif [[ "$CURRENT" -ne "$LATEST" ]]; then
# rebuild (update) the base image
echo "[$(date +'%D %H:%M:%S')] Updating base image..."
docker buildx build --no-cache -t jackavery/base-tf2server:latest /home/tf2server/build/base
podman buildx build --no-cache -t jackavery/base-tf2server:latest /home/tf2server/build/base

get_networks

Expand All @@ -65,16 +66,16 @@ main() {
for server in ${SERVERS[@]} ; do
echo "[$(date +'%D %H:%M:%S')] Rebuilding $network/$server..."
cd /home/tf2server/build/servers/$network/$server
docker buildx build -t srcds-$network-$server:latest .
podman buildx build -t srcds-$network-$server:latest .
cd /home/tf2server
done

echo "[$(date +'%D %H:%M:%S')] Docker recomposing $network..."
docker compose -f /home/tf2server/docker-compose_$network.yml up -d
echo "[$(date +'%D %H:%M:%S')] Podman recomposing $network..."
podman compose -f /home/tf2server/podman-compose_$network.yml up -d
done

echo "[$(date +'%D %H:%M:%S')] Pruning Docker..."
docker system prune -f
echo "[$(date +'%D %H:%M:%S')] Pruning Podman..."
podman system prune -af

echo $LATEST > current_tf2_buildid

Expand Down
35 changes: 22 additions & 13 deletions roles/deploy/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,6 @@
- name: Perform setup
ansible.builtin.include_tasks: setup.yml

- name: debug
debug:
var: targets

- name: Announce update
ignore_errors: true
rcon:
Expand All @@ -19,26 +15,39 @@
delegate_to: localhost
timeout: 10

- name: Create Docker network
community.docker.docker_network:
- name: Create Podman network
containers.podman.podman_network:
name: net-{{ network_shortname }}
state: quadlet
quadlet_options:
- |
[Install]
WantedBy=ansible-tf2network_{{ network_shortname }}.service multi-user.target default.target

- name: Template kube.j2
ansible.builtin.template:
src: "kube.yaml.j2"
dest: "~/{{ network_shortname }}.yaml"
mode: 0755

- name: Generate docker-compose
- name: Template quadlet.j2
ansible.builtin.template:
src: docker-compose.yml.j2
dest: /home/tf2server/docker-compose_{{ network_shortname }}.yml
mode: 0700
src: "quadlet.kube.j2"
dest: "~/.config/containers/systemd/ansible-tf2network_{{ network_shortname }}.kube"
mode: 0755

- name: docker compose up -d
shell: docker compose -f /home/tf2server/docker-compose_{{ network_shortname }}.yml up -d
- name: systemd user daemon reload
ansible.builtin.systemd_service:
scope: user
daemon_reload: true

- name: Install restart cronjob
ansible.builtin.cron:
name: "srcds-restart-{{ network_shortname }}"
weekday: "*"
minute: "0"
hour: "{{ daily_restart_hour_utc }}"
job: "docker compose -f /home/tf2server/docker-compose_{{ network_shortname }}.yml restart"
job: "podman pod restart ansible-tf2network_{{ network_shortname }}"

- name: Copy updater script
copy:
Expand Down
29 changes: 0 additions & 29 deletions roles/deploy/templates/docker-compose.yml.j2

This file was deleted.

45 changes: 45 additions & 0 deletions roles/deploy/templates/kube.yaml.j2
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
apiVersion: v1
kind: Pod
metadata:
name: ansible-tf2network_{{ network_shortname }}
spec:
containers:
{% for item in targets.instances %}
- name: "srcds-{{ network_shortname }}-{{ item.name }}"
image: "srcds-{{ network_shortname }}-{{ item.name }}:latest"
ports:
- containerPort: {{ item.port }}
hostPort: {{ item.port }}
protocol: TCP
- containerPort: {{ item.port }}
hostPort: {{ item.port }}
protocol: UDP
- containerPort: {{ item.port + 5 }}
hostPort: {{ item.port + 5}}
protocol: UDP
args:
- -cpus
- "1"
Comment on lines +21 to +22
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make this use cpuset

{% endfor %}
{% if not targets.is_metrics %}
- name: "ssh-{{ network_shortname }}"
image: docker.io/kroniak/ssh-client
restart: always
volumes:
- "/home/tf2server/.ssh/sbpp_key_{{ network_shortname }}:/root/id_rsa"
command:
- ssh
- -4NTCp
- "{{ sbpp_host_ssh_port }}"
- -i
- /root/id_rsa
- -o
- StrictHostKeyChecking=no
- -o
- ServerAliveInterval=60
- -o
- ExitOnForwardFailure=yes
- -L
- "0.0.0.0:3306:db:3306"
- "sbpp_user@{{ sbpp_host }}"
{% endif %}
Loading