AbstractFFTs failing to precompile on a server

[votroto]

Hi, I'm having trouble getting CUDA.jl to run on a server because AbstractFFTs.jl fails during precompilation. Any ideas what might be causing it or how to fix it?

(scut) pkg> add AbstractFFTs
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
  No Changes to `~/scut/Project.toml`
  No Changes to `~/scut/Manifest.toml`
  0 dependencies successfully precompiled in 0 seconds. 5 already precompiled. 1 skipped during auto due to previous errors.
  1 dependency errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the package

(scut) pkg> precompile
Precompiling project...
  ✗ AbstractFFTs
  0 dependencies successfully precompiled in 1 seconds. 5 already precompiled.

ERROR: The following 1 direct dependency failed to precompile:

AbstractFFTs [621f4979-c628-5d54-868e-fcf4e3e8185c]

Failed to precompile AbstractFFTs [621f4979-c628-5d54-868e-fcf4e3e8185c] to /home/votroto1/.julia/compiled/v1.8/AbstractFFTs/jl_jh9mgp.
ERROR: LoadError: ArgumentError: Package ChainRulesCore [d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

Stacktrace:
 [1] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1306
 [2] _require_prelocked(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1200
 [3] macro expansion
   @ ./loading.jl:1180 [inlined]
 [4] macro expansion
   @ ./lock.jl:223 [inlined]
 [5] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1144
 [6] include
   @ ./Base.jl:419 [inlined]
 [7] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
   @ Base ./loading.jl:1554
 [8] top-level scope
   @ stdin:1
in expression starting at /home/votroto1/.julia/packages/AbstractFFTs/Wg2Yf/src/AbstractFFTs.jl:1
in expression starting at stdin:1

(scut) pkg> instantiate
  0 dependencies successfully precompiled in 0 seconds. 5 already precompiled. 1 skipped during auto due to previous errors.
  1 dependency errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the package

julia> using AbstractFFTs
[ Info: Precompiling AbstractFFTs [621f4979-c628-5d54-868e-fcf4e3e8185c]
ERROR: LoadError: ArgumentError: Package ChainRulesCore [d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

Stacktrace:
 [1] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1306
 [2] _require_prelocked(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1200
 [3] macro expansion
   @ ./loading.jl:1180 [inlined]
 [4] macro expansion
   @ ./lock.jl:223 [inlined]
 [5] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1144
 [6] include
   @ ./Base.jl:419 [inlined]
 [7] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
   @ Base ./loading.jl:1554
 [8] top-level scope
   @ stdin:1
in expression starting at /home/votroto1/.julia/packages/AbstractFFTs/Wg2Yf/src/AbstractFFTs.jl:1
in expression starting at stdin:1
ERROR: Failed to precompile AbstractFFTs [621f4979-c628-5d54-868e-fcf4e3e8185c] to /home/votroto1/.julia/compiled/v1.8/AbstractFFTs/jl_sCH8yP.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
   @ Base ./loading.jl:1707
 [3] compilecache
   @ ./loading.jl:1651 [inlined]
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1337
 [5] _require_prelocked(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1200
 [6] macro expansion
   @ ./loading.jl:1180 [inlined]
 [7] macro expansion
   @ ./lock.jl:223 [inlined]
 [8] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1144

Here are two affected systems

julia> versioninfo()
Julia Version 1.8.2
Commit 36034abf260 (2022-09-29 15:21 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 128 × AMD EPYC 7543 32-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, znver3)
  Threads: 1 on 128 virtual cores
Environment:
  LD_LIBRARY_PATH = /mnt/appl/software/OpenBLAS/0.3.20-GCC-11.3.0/lib:/mnt/appl/software/binutils/2.38-GCCcore-11.3.0/lib:/mnt/appl/software/zlib/1.2.12-GCCcore-11.3.0/lib:/mnt/appl/software/GCCcore/11.3.0/lib64:/mnt/appl/software/CUDA/11.7.0/nvvm/lib64:/mnt/appl/software/CUDA/11.7.0/extras/CUPTI/lib64:/mnt/appl/software/CUDA/11.7.0/lib:/mnt/appl/software/Julia/1.8.2-linux-x86_64/lib

and

julia> versioninfo()
Julia Version 1.8.0
Commit 5544a0fab76 (2022-08-17 13:38 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 72 × Intel(R) Xeon(R) Gold 6150 CPU @ 2.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, skylake-avx512)
  Threads: 1 on 72 virtual cores
Environment:
  LD_LIBRARY_PATH = /mnt/appl/software/LAPACK/3.9.1-GCC-10.2.0/lib:/mnt/appl/software/binutils/2.35-GCCcore-10.2.0/lib:/mnt/appl/software/zlib/1.2.11-GCCcore-10.2.0/lib:/mnt/appl/software/GCCcore/10.2.0/lib64:/mnt/appl/software/GCCcore/10.2.0/lib:/mnt/appl/software/OpenBLAS/0.3.20-GCC-11.3.0/lib:/mnt/appl/software/Julia/1.8.0-linux-x86_64/lib:/mnt/appl/software/CUDAcore/11.4.1/nvvm/lib64:/mnt/appl/software/CUDAcore/11.4.1/extras/CUPTI/lib64:/mnt/appl/software/CUDAcore/11.4.1/lib:/mnt/appl/software/CUDA/11.7.0/nvvm/lib64:/mnt/appl/software/CUDA/11.7.0/extras/CUPTI/lib64:/mnt/appl/software/CUDA/11.7.0/lib