Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[clang] clang driver fails to find headers/libraries when specified --gcc-install-dir or --gcc-toolchain or --sysroot to a cross-compiler directory #114210

Open
lbmeng opened this issue Oct 30, 2024 · 1 comment
Labels
clang:driver 'clang' and 'clang++' user-facing binaries. Not 'clang-cl'

Comments

@lbmeng
Copy link

lbmeng commented Oct 30, 2024

I am trying to use clang 18 to cross compile a program for Arm 32-bit target.
The clang 18 was installed from the official LLVM Ubuntu repository following the instructions here.

The GCC toolchain I was using is from bootlin.

The rough GCC toolchain directory layout is:

/path/to/bootlin/toolchain
├── arm-buildroot-linux-gnueabihf
│   ├── bin
│   ├── include
│   │   └── c++
│   ├── lib
│   └── sysroot
│       ├── bin
│       ├── dev
│       ├── etc
│       ├── lib
│       ├── lib32 -> lib
│       ├── media
│       ├── mnt
│       ├── opt
│       ├── proc
│       ├── root
│       ├── run
│       ├── sbin
│       ├── sys
│       ├── tmp
│       ├── usr
│       └── var
├── bin
├── etc
├── include
├── lib
│   ├── gcc
│   │   └── arm-buildroot-linux-gnueabihf
...
├── lib64 -> lib
├── libexec
└── share

Compiling a simple program using:

$ clang-18 --target=arm-eabi -mcpu=cortex-a7 -mthumb -mfpu=neon -mfloat-abi=hard -Wall --gcc-toolchain=/path/to/bootlin/toolchain -c test.c -o test.o
clang-18: warning: argument unused during compilation: '--gcc-toolchain=/path/to/bootlin/toolchain' [-Wunused-command-line-argument]
test.c:1:10: fatal error: 'stdio.h' file not found
    1 | #include <stdio.h>
      |          ^~~~~~~~~
1 error generated.

clang-18 complains --gcc-toolchain is unused. Changing to use --gcc-install-dir does not make it better. The same unused argument warning was thrown. By looking at the documentation, the doc does not mention how exactly these 2 options affects what behavior of clang.

Changing to use--sysroot failed too. The sysroot value was obtained from the result of /path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-gcc --print-sysroot.

Then I found a workaround to help locate the header file by creating a symbolic link include to point to usr/include in the sysroot directory when using

/path/to/bootlin/toolchain/arm-buildroot-linux-gnueabihf/sysroot$ ln -s usr/include include

Then I removed -c to see if I can compile and link the program correctly, I got:

$ clang-18 --target=arm-eabi -mcpu=cortex-a7 -mthumb -mfpu=neon -mfloat-abi=hard -Wall --sysroot=/path/to/bootlin/toolchain/arm-buildroot-linux-gnueabihf/sysroot test.c -o test.o
ld.lld: error: unable to find library -lc
ld.lld: error: unable to find library -lm
ld.lld: error: unable to find library -lgcc
clang-18: error: ld.lld command failed with exit code 1 (use -v to see invocation)

I realized the new clang compiler driver defaults to use ld.lld, so I added -fuse-ld=/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld in the command line, which made clang to successfully locate libc and libm libraries, but not libgcc.

$ clang-18 --target=arm-eabi -mcpu=cortex-a7 -mthumb -mfpu=neon -mfloat-abi=hard -Wall --sysroot=/path/to/bootlin/toolchain/arm-buildroot-linux-gnueabihf/sysroot -fuse-ld=/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld test.c -o test.o
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: warning: library search path "/usr/lib/llvm-18/lib/clang/18/lib/baremetal" is unsafe for cross-compilation
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: cannot find -lgcc: No such file or directory
clang-18: error: ld.lld command failed with exit code 1 (use -v to see invocation)

The libgcc.a is located in the bootlin gcc toolchain directorty here:

$ /path/to/bootlin/toolchain$ find . -name libgcc.a
./lib/gcc/arm-buildroot-linux-gnueabihf/12.3.0/libgcc.a

So I added -L/path/to/bootlin/toolchain/lib/gcc/arm-buildroot-linux-gnueabihf/12.3.0 in the command line, then libgcc was located by clang, but the final linking procedure still complained:

$ clang-18 --target=arm-eabi -mcpu=cortex-a7 -mthumb -mfpu=neon -mfloat-abi=hard -Wall --sysroot=/path/to/bootlin/toolchain/arm-buildroot-linux-gnueabihf/sysroot -fuse-ld=/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld test.c -L/path/to/bootlin/toolchain/lib/gcc/arm-buildroot-linux-gnueabihf/12.3.0 -o test.o
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: warning: library search path "/usr/lib/llvm-18/lib/clang/18/lib/baremetal" is unsafe for cross-compilation
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: warning: cannot find entry symbol _start; defaulting to 00010140
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: /path/to/bootlin/toolchain/bin/../arm-buildroot-linux-gnueabihf/sysroot/usr/lib/libc.a(printf.o):(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr1'
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: /path/to/bootlin/toolchain/bin/../arm-buildroot-linux-gnueabihf/sysroot/usr/lib/libc.a(getsysstats.o):(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr0'
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: /path/to/bootlin/toolchain/bin/../arm-buildroot-linux-gnueabihf/sysroot/usr/lib/libc.a(getsysstats.o):(.ARM.exidx+0x8): undefined reference to `__aeabi_unwind_cpp_pr1'
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: /path/to/bootlin/toolchain/bin/../arm-buildroot-linux-gnueabihf/sysroot/usr/lib/libc.a(madvise.o):(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr0'
...

I gave up here. So what is the expected way of using clang to cross-compile a program and using libraries from a gcc cross-toolchain?

@github-actions github-actions bot added the clang Clang issues not falling into any other category label Oct 30, 2024
@EugeneZelenko EugeneZelenko added clang:driver 'clang' and 'clang++' user-facing binaries. Not 'clang-cl' and removed clang Clang issues not falling into any other category labels Oct 30, 2024
@llvmbot
Copy link
Collaborator

llvmbot commented Oct 30, 2024

@llvm/issue-subscribers-clang-driver

Author: Bin Meng (lbmeng)

I am trying to use clang 18 to cross compile a program for Arm 32-bit target. The clang 18 was installed from the official LLVM Ubuntu repository following the instructions [here](https://apt.llvm.org/).

The GCC toolchain I was using is from bootlin.

The rough GCC toolchain directory layout is:

/path/to/bootlin/toolchain
├── arm-buildroot-linux-gnueabihf
│   ├── bin
│   ├── include
│   │   └── c++
│   ├── lib
│   └── sysroot
│       ├── bin
│       ├── dev
│       ├── etc
│       ├── lib
│       ├── lib32 -&gt; lib
│       ├── media
│       ├── mnt
│       ├── opt
│       ├── proc
│       ├── root
│       ├── run
│       ├── sbin
│       ├── sys
│       ├── tmp
│       ├── usr
│       └── var
├── bin
├── etc
├── include
├── lib
│   ├── gcc
│   │   └── arm-buildroot-linux-gnueabihf
...
├── lib64 -&gt; lib
├── libexec
└── share

Compiling a simple program using:

$ clang-18 --target=arm-eabi -mcpu=cortex-a7 -mthumb -mfpu=neon -mfloat-abi=hard -Wall --gcc-toolchain=/path/to/bootlin/toolchain -c test.c -o test.o
clang-18: warning: argument unused during compilation: '--gcc-toolchain=/path/to/bootlin/toolchain' [-Wunused-command-line-argument]
test.c:1:10: fatal error: 'stdio.h' file not found
    1 | #include &lt;stdio.h&gt;
      |          ^~~~~~~~~
1 error generated.

clang-18 complains --gcc-toolchain is unused. Changing to use --gcc-install-dir does not make it better. The same unused argument warning was thrown. By looking at the documentation, the doc does not mention how exactly these 2 options affects what behavior of clang.

Changing to use--sysroot failed too. The sysroot value was obtained from the result of /path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-gcc --print-sysroot.

Then I found a workaround to help locate the header file by creating a symbolic link include to point to usr/include in the sysroot directory when using

/path/to/bootlin/toolchain/arm-buildroot-linux-gnueabihf/sysroot$ ln -s usr/include include

Then I removed -c to see if I can compile and link the program correctly, I got:

$ clang-18 --target=arm-eabi -mcpu=cortex-a7 -mthumb -mfpu=neon -mfloat-abi=hard -Wall --sysroot=/path/to/bootlin/toolchain/arm-buildroot-linux-gnueabihf/sysroot test.c -o test.o
ld.lld: error: unable to find library -lc
ld.lld: error: unable to find library -lm
ld.lld: error: unable to find library -lgcc
clang-18: error: ld.lld command failed with exit code 1 (use -v to see invocation)

I realized the new clang compiler driver defaults to use ld.lld, so I added -fuse-ld=/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld in the command line, which made clang to successfully locate libc and libm libraries, but not libgcc.

$ clang-18 --target=arm-eabi -mcpu=cortex-a7 -mthumb -mfpu=neon -mfloat-abi=hard -Wall --sysroot=/path/to/bootlin/toolchain/arm-buildroot-linux-gnueabihf/sysroot -fuse-ld=/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld test.c -o test.o
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: warning: library search path "/usr/lib/llvm-18/lib/clang/18/lib/baremetal" is unsafe for cross-compilation
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: cannot find -lgcc: No such file or directory
clang-18: error: ld.lld command failed with exit code 1 (use -v to see invocation)

The libgcc.a is located in the bootlin gcc toolchain directorty here:

$ /path/to/bootlin/toolchain$ find . -name libgcc.a
./lib/gcc/arm-buildroot-linux-gnueabihf/12.3.0/libgcc.a

So I added -L/path/to/bootlin/toolchain/lib/gcc/arm-buildroot-linux-gnueabihf/12.3.0 in the command line, then libgcc was located by clang, but the final linking procedure still complained:

$ clang-18 --target=arm-eabi -mcpu=cortex-a7 -mthumb -mfpu=neon -mfloat-abi=hard -Wall --sysroot=/path/to/bootlin/toolchain/arm-buildroot-linux-gnueabihf/sysroot -fuse-ld=/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld test.c -L/path/to/bootlin/toolchain/lib/gcc/arm-buildroot-linux-gnueabihf/12.3.0 -o test.o
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: warning: library search path "/usr/lib/llvm-18/lib/clang/18/lib/baremetal" is unsafe for cross-compilation
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: warning: cannot find entry symbol _start; defaulting to 00010140
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: /path/to/bootlin/toolchain/bin/../arm-buildroot-linux-gnueabihf/sysroot/usr/lib/libc.a(printf.o):(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr1'
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: /path/to/bootlin/toolchain/bin/../arm-buildroot-linux-gnueabihf/sysroot/usr/lib/libc.a(getsysstats.o):(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr0'
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: /path/to/bootlin/toolchain/bin/../arm-buildroot-linux-gnueabihf/sysroot/usr/lib/libc.a(getsysstats.o):(.ARM.exidx+0x8): undefined reference to `__aeabi_unwind_cpp_pr1'
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: /path/to/bootlin/toolchain/bin/../arm-buildroot-linux-gnueabihf/sysroot/usr/lib/libc.a(madvise.o):(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr0'
...

I gave up here. So what is the expected way of using clang to cross-compile a program and using libraries from a gcc cross-toolchain?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clang:driver 'clang' and 'clang++' user-facing binaries. Not 'clang-cl'
Projects
None yet
Development

No branches or pull requests

3 participants