Re-enable OSG 23 development images (SOFTWARE-5673)

[brianhlin]

No description provided.

[matyasselmeci]

LGTM; once pelican-osdf-compat is in the htcondor 23.x daily repos (and therefore osg-23-upcoming-development repos) then you should rerun that failed job and merge.

[matyasselmeci]

Brian, if this PR is still relevant, can you rerun the jobs and merge if they pass?

[brianhlin]

I couldn't find a button to rerun tests for CI this old so I pushed a dummy commit then removed it and force pushed

[brianhlin]

Hrm, I'm not sure I understand the failures:

1. Testing singularity detection inside the backfill container.

INFO   Determining extra Singularity capabilities...
INFO   Sourcing /pilot/osgvo-pilot-ufwmtr/add_config_line.source
INFO  Error testing for SIF support; output:
INFO  FATAL:   container creation failed: image driver mount failure: image driver squashfuse_ll instance exited with error: squashfuse_ll exited: fuse: device not found, try 'modprobe fuse' first

2. Similarly in the startd

<149>1 2024-05-21T17:49:05.320Z b8df5ed7ea4c condor_starter 4543 - [cat="D_ALWAYS:2" slot="testing" GLIDEIN_ResourceName="'None'" GLIDEIN_Site="'None'"] Attempting to run: '/usr/bin/apptainer exec -C /usr/libexec/condor/exit_37.sif /exit_37'.
<149>1 2024-05-21T17:49:05.364Z b8df5ed7ea4c condor_starter 4543 - [cat="D_ALWAYS" slot="testing" GLIDEIN_ResourceName="'None'" GLIDEIN_Site="'None'"] '/usr/bin/apptainer exec -C /usr/libexec/condor/exit_37.sif /exit_37' did not exit successfully (code 65280); stderr is :
<149>1 2024-05-21T17:49:05.364Z b8df5ed7ea4c condor_starter 4543 - [cat="D_ALWAYS" slot="testing" GLIDEIN_ResourceName="'None'" GLIDEIN_Site="'None'"] [singularity stderr]: FATAL:   container creation failed: image driver mount failure: image driver squashfuse_ll instance exited with error: squashfuse_ll exited: fuse: device not found, try 'modprobe fuse' first
<29>1 2024-05-21T17:49:05.364Z b8df5ed7ea4c supervisord 1 - [level="INFO" GLIDEIN_ResourceName="'None'" GLIDEIN_Site="'None'"] reaped unknown pid 4607 (terminated by SIGKILL)

[matyasselmeci]

Possibly #171 fixes this. Brian, can you rebase on master and try again?

[matyasselmeci]

I see the problem. I was expecting 23.10.0 to fix the issue and wrote the version comparison accordingly (>=23.8.0, <23.10.0), but the daily is technically called 23.10.0 and doesn't yet fix it.