Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

iptables locking issues if --wait is not available #2625

Open
mm-dlwe opened this issue Mar 12, 2021 · 0 comments
Open

iptables locking issues if --wait is not available #2625

mm-dlwe opened this issue Mar 12, 2021 · 0 comments

Comments

@mm-dlwe
Copy link

mm-dlwe commented Mar 12, 2021

If an iptables version is used that does not support or expose (in case a wrapper script is used arround iptables) the --wait flag issues occur if multiple containers are started at the same time. As only one iptables instance can run at any time, parallel calls to iptables fail and the rules for the dns resolver are not added to the container (in case a docker network is used).

Symptom:
Containers can not access other containers or the internet due to the DNS resolution failing.

Steps to reproduce:

  • Use an iptables wrapper or version without --wait
  • Create a docker network
  • Add multiple containers to this network that are restarted on system start
  • Restart docker or the system to start all containers at once

Cause for this issue:
Docker will detect that the xtables lock (iptables/iptables.go:121) is not supported and use its internal locking mechanism (bestEffortLock) instead (iptables/iptables.go:537).

The issue is that this lock is a mutex and can only provide locking within the same process. This becomes an issue if docker networks are used. In this case a new resolver process for each container is spawned via "reexec" to add the required iptables rules into the container network namespace (resolver_unix.go:29). If multiple containers start in parallel, multiple of these resolver setup processes are spawned which each have their own independent bestEffortLock and thus try to execute iptables in parallel to each other - and fail while attempting to do this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant