Long hangs when os.sysconf('SC_OPEN_MAX') is large

[originalsouth]

The line return os.sysconf('SC_OPEN_MAX') in

billiard/billiard/compat.py

Lines 95 to 112 in 1392223

	def get_fdmax(default=None):
	"""Return the maximum number of open file descriptors
	on this system.
	:keyword default: Value returned if there's no file
	descriptor limit.
	"""
	try:
	return os.sysconf('SC_OPEN_MAX')
	except:
	pass
	if resource is None: # Windows
	return default
	fdmax = resource.getrlimit(resource.RLIMIT_NOFILE)[1]
	if fdmax == resource.RLIM_INFINITY:
	return default
	return fdmax

causes hangs when SC_OPEN_MAX is very large for instance in Docker containers.

In said Docker container:

root@8fd7761924b0:/app/octopoes# python
Python 3.11.7 (main, Jan 11 2024, 10:33:01) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.sysconf('SC_OPEN_MAX')
1073741816

the line returns a large number. Resulting

billiard/billiard/compat.py

Lines 144 to 153 in 1392223

	def close_open_fds(keep=None): # noqa
	keep = [maybe_fileno(f)
	for f in (keep or []) if maybe_fileno(f) is not None]
	for fd in reversed(range(get_fdmax(default=2048))):
	if fd not in keep:
	try:
	os.close(fd)
	except OSError as exc:
	if exc.errno != errno.EBADF:
	raise

to try to close many non-existent files while hanging. Which can be observed using strace on the process (snip):

...
close(971554778)                        = -1 EBADF (Bad file descriptor)
close(971554777)                        = -1 EBADF (Bad file descriptor)
close(971554776)                        = -1 EBADF (Bad file descriptor)
close(971554775)                        = -1 EBADF (Bad file descriptor)
close(971554774)                        = -1 EBADF (Bad file descriptor)
close(971554773)                        = -1 EBADF (Bad file descriptor)
close(971554772)                        = -1 EBADF (Bad file descriptor)
close(971554771)                        = -1 EBADF (Bad file descriptor)
close(971554770)                        = -1 EBADF (Bad file descriptor)
...

This bug possibly causes issues like celery/celery#8306 and related.