Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"swarm exec" support #425

Open
zibellon opened this issue Jun 1, 2024 · 1 comment
Open

"swarm exec" support #425

zibellon opened this issue Jun 1, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@zibellon
Copy link

zibellon commented Jun 1, 2024

Is your feature request related to a problem? Please describe.
Yes. The problem was described in issue 423. previous issue

Main points of problem

  1. Swarm cluster. 2 Nodes. MASTER_1, WORKER_1
  2. Run postgres with docker on WORKER_1
  3. Need to backup postgres. Before backup need to run script: pg_dump ...
  4. Up offen-backup on WORKER_1, add labels to deploy section of postgres stack (service)
  5. Catch the error: no master node. Can exec labels only on master

Describe the solution you'd like
Solution is

  1. Run offen-backup service ONLY on master node
  2. For exec labels - run one-time container (docker:25.0.5-cli-alpine) on EACH node when we found tasks (containers) with lables
  3. For volumes - run one-time container (offen-backup, for example) on EACH node when we found tasks (containers) with labels: volume-list (You can see it in my .sh script)
  4. If we want to use different timers for different exec-lables or different volume-list-labels - we can also use your idea of exec-label=database, but add support for exec and for volume-list

Additional context
I'm a big fan of docker-swarm. And use swarm cluster in production.

I spent some time and write .sh script for backup WHOLE swarm cluster, depends on labels and use your open-source project
swarm-backuper.sh

  1. GET all volumes from all nodes. Start docker dind container on each node and get volume list
  2. GET all services with label volume-list. in this label - we enter volumes, which we want to backup for this service. JOIN this volumes with volumes from step one. Result: MAP<node-name, [volume1, volume2, etc]>
  3. RUN all labels: exec-pre. Run separate container on each node, where we need to run exec-pre and run docker exec ...
  4. STOP all services with label: stop. Important: stop === scale to 0, Stop only services in replicated mode
  5. RUN offen-backup on each node with all volumes from step 2
  6. RESTORE all services with label: stop. Important: we need to use the JSON from step 4. Scale to original replicas number, before step 4
  7. RUN all labels: exec-post

How we can run it ?
swarm-backuper-stack.yaml


All of this functions can be write on GoLang and add to ypur project (offen-backup)

@m90
Copy link
Member

m90 commented Jun 2, 2024

I'm still not entirely sure why you're unable to execute commands on the same node (this should work no matter if it's a master or a worker node, albeit setup can be a bit tricky, see moby/moby#27552).

That being said, the described approach of using one-off containers that mount volumes themselves has also been discussed here already #329 maybe this use case can be considered if this is going to be implemented.

@m90 m90 added the enhancement New feature or request label Jun 2, 2024
@m90 m90 changed the title Swarm support "swarm exec" support Jun 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants