Compiling a cmake project with zig c++ doesn't resolve to and absolute path

[Sid911]

Xmake Version

2.9.4+20240729

Operating System Version and Architecture

Ubuntu 22.04.3 LTS x86_64

Describe Bug

Setting zig as a default toolchain for entire script and including a cmake based project for this instance snitch 1.2.5 results in cmake not recognizing the path as an absolute path

add_requires("snitch 1.2.5", {system=false})

set_toolchains("zig")
-- ...

target("test")
  -- ...

This results in this error

/snap/bin/cmake -DSNITCH_DO_TEST=OFF -DSNITCH_CREATE_HEADER_ONLY=OFF -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DSNITCH_DEFINE_MAIN=ON -DCMAKE_INSTALL_PREFIX=/home/sid/.xmake/packages/s/snitch/1.2.5/14ad55b1337f4c69825194568c3595cb -DCMAKE_INSTALL_LIBDIR:PATH=lib -G "Unix Makefiles" -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=BOTH -DCMAKE_OSX_SYSROOT= -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=BOTH "-DCMAKE_C_COMPILER=/usr/local/bin/zig cc" -DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER -DCMAKE_AR=/usr/local/bin/zig -DCMAKE_FIND_USE_INSTALL_PREFIX=0 -DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=BOTH "-DCMAKE_CXX_COMPILER=/usr/local/bin/zig c++" -DCMAKE_FIND_USE_CMAKE_SYSTEM_PATH=0 -DHAVE_FLAG_SEARCH_PATHS_FIRST=0 -DCMAKE_STATIC_LINKER_FLAGS= /home/sid/.xmake/cache/packages/2409/s/snitch/1.2.5/source
-- The CXX compiler identification is unknown
CMake Error at CMakeLists.txt:3 (project):
  The CMAKE_CXX_COMPILER:

    /usr/local/bin/zig c++

  is not a full path to an existing compiler tool.

  Tell CMake where to find the compiler by setting either the environment
  variable "CXX" or the CMake cache entry CMAKE_CXX_COMPILER to the full path
  to the compiler, or to the compiler name if it is in the PATH.


-- Configuring incomplete, errors occurred!
error: execv(/snap/bin/cmake -DSNITCH_DO_TEST=OFF -DSNITCH_CREATE_HEADER_ONLY=OFF -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DSNITCH_DEFINE_MAIN=ON -DCMAKE_INSTALL_PREFIX=/home/sid/.xmake/packages/s/snitch/1.2.5/14ad55b1337f4c69825194568c3595cb -DCMAKE_INSTALL_LIBDIR:PATH=lib -G "Unix Makefiles" -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=BOTH -DCMAKE_OSX_SYSROOT= -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=BOTH "-DCMAKE_C_COMPILER=/usr/local/bin/zig cc" -DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER -DCMAKE_AR=/usr/local/bin/zig -DCMAKE_FIND_USE_INSTALL_PREFIX=0 -DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=BOTH "-DCMAKE_CXX_COMPILER=/usr/local/bin/zig c++" -DCMAKE_FIND_USE_CMAKE_SYSTEM_PATH=0 -DHAVE_FLAG_SEARCH_PATHS_FIRST=0 -DCMAKE_STATIC_LINKER_FLAGS= /home/sid/.xmake/cache/packages/2409/s/snitch/1.2.5/source) failed(1)
  => install snitch 1.2.5 .. failed
error: install failed!

This was discussed in ziglang/zig#8716, but the conclusion was that its basically glue code which depends per platform.

Expected Behavior

zig should be able to be used by default as compilation for dependent projects. This will allow cross compilation more easily.

It would be amazing this could be setup. The workaround for linux where you basically make a warpper script doesn't work with xmake as it results in cannot find known tool script for zig-cxx etc.

Project Configuration

add_rules("mode.debug", "mode.release")
add_rules("plugin.compile_commands.autoupdate", {outputdir = "./build"})
includes("@builtin/check")
includes("toolchains/*.lua")

set_languages("c++2c")

add_requires("flux ~0.4")
add_requires("snitch 1.2.5", {system=false})
add_requires("fmt", {system=false, configs = {header_only = true}})

-- version 015de6adee287917a9ee10afcbfcb8e560f4d9aa 
-- https://github.com/Sid911/xrepo Add my xrepo because this isn't usually available in official xrepo
add_requires("argparse") 
add_requires("simdutf", "utfcpp")
add_requires("cmake::LLVM 19", {alias = "llvm", system=true, configs = {search_mode = "config"}})

-- set_toolchains("clang", "gcc", "icx")
set_toolchains("zig")

option("static_bin")
    set_default(true)
    set_showmenu(true)
    add_ldflags("-static", {force = true})
option_end()

target("scions_dsl")
    set_version("0.0.1", {build = "%Y%m%d%H%M"})
    set_kind("binary")
    set_policy("build.optimization.lto", true)
    set_policy("build.warning", true)

    set_warnings("all", "extra")
    add_options("static_bin")

    -- Used for __declspec extensions
    -- add_cxxflags("-fms-extensions", {force = true})

    -- on_config(function(target)
    --     -- basically don't support gcc
    --     if target:has_cxxflags("-fdeclspec") then
    --         target:add("defines", "HAS_DECLSPEC_PROPERTY")
    --     end
    -- end)

    if(is_mode("debug"))then
        add_defines("SC_DEBUG")
        set_symbols("debug")
        set_optimize("none")
    end

    if(is_mode("release", "profile")) then
        if is_mode("release") then
            set_symbols("hidden")
            set_strip("all")
            add_cxxflags("-fomit-frame-pointer")
            set_optimize("fastest")
        else
            set_symbols("debug")
            set_optimize("fast")
        end
        add_vectorexts("avx", "avx2")
    end

    add_includedirs("$(projectdir)/include")
    add_files("src/**.cpp")

    add_packages("flux", "argparse")
    add_packages("llvm")
    add_packages("fmt", "simdutf")

    -- Adding hard link shouldn't be there but for the sake for my sanity I will do it for now
    -- Todo: Remove hardlink
    add_linkdirs("/lib/llvm-19/lib")
    -- add_links("LLVMCore","LLVMRemarks", "LLVMBitstreamReader", "LLVMBinaryFormat", "LLVMTargetParser") -- LLVM core
    -- add_links("LLVMIRReader", "LLVMBitReader", "LLVMAsmParser") -- LLVM irreader
    -- add_links("LLVMSupport", "LLVMDemangle") -- LLVM SUpport

    add_defines("SCIONS_TRUE")
    -- before_build(function (target)
        -- os.cp("$(projectdir)/hello.txt", target:targetdir() .. "/inc.txt")
        -- cprint('${bright}CPU : ')
        -- print(target:orderpkgs())
    -- end)
target_end()

-- This is primarily for having all the stuff actually 
target("scions_lib")
    set_kind("static")
    set_default(false)
    set_targetdir("$(buildir)/$(host)/$(arch)/$(mode)/lib/")

    -- set_policy("build.merge_archive", true)
    set_policy("build.warning", true)
    set_warnings("all", "extra")    

    if(is_mode("debug"))then
        add_defines("SC_DEBUG")
        set_symbols("debug")
        set_optimize("none")
    end

    if(is_mode("release", "profile")) then
        if is_mode("release") then
            set_symbols("hidden")
            set_strip("all")
            add_cxxflags("-fomit-frame-pointer")
            set_optimize("fastest")
        else
            set_symbols("debug")
            set_optimize("fast")
        end
        add_vectorexts("avx", "avx2")
    end

    add_packages("flux")
    add_packages("fmt", "simdutf")

    add_includedirs("$(projectdir)/include", {public = true})
    add_files("src/**.cpp")
    remove_files("src/main.cpp")
target_end()


for _, file in ipairs(os.files("tests/**.cpp")) do
    -- print("relative_path: " .. relative_path .. "\n")
    -- local src_path = relative_path:gsub("^tests", "src", 1)
    local name = path.basename(file)
    target(name)
        set_kind("binary")
        set_default(false)

        add_includedirs("$(projectdir)/include")

        add_packages("snitch", "flux")
        add_deps("scions_lib")
        set_targetdir("$(buildir)/$(host)/$(arch)/$(mode)/tests/")

        on_config(function (target)
            -- utility functions : I am not good enough for lua yet
            -- there is porbably is a better way to do this
            local function read_first_line(file_path)
                local file = io.open(file_path, "r")
                if not file then return nil end
                local first_line = file:read("*line")
                file:close()
                return first_line
            end

            local function extract_test_name(line)
                -- Remove leading and trailing whitespace, then remove "//" if present
                return line:match("^%s*//?(.*)")
                       :gsub("^%s*(.-)%s*$", "%1")  -- Trim whitespace
            end

            local relative_path = path.relative(file, ".")
            local first_line = read_first_line(file)
            local test_name = first_line and extract_test_name(first_line) or "Unknown Test"

            target:add("files",relative_path)
            target:add("tests",test_name)
        end)
    target_end()
end

Additional Information and Error Logs

sid@sid-ubuntu /m/s/P/C/S/scions_dsl (master)> xmake -j 8 -vD
checking for gcc ... /usr/bin/gcc
checking for zig ... /usr/local/bin/zig
checkinfo: cannot runv(nim --version), No such file or directory
checking for nim ... no
checkinfo: cannot runv(nim --version), No such file or directory
checking for nim ... no
checking for unzip ... /usr/bin/unzip
checking for git ... /usr/bin/git
checking for gzip ... /usr/bin/gzip
checking for tar ... /usr/bin/tar
/usr/bin/git rev-parse HEAD
/usr/bin/git rev-parse HEAD
finding flux from xmake ..
checking for xmake::flux ... flux 0.4.0
checking for cmake ... no
checking for cmake ... /snap/bin/cmake
finding snitch from xmake ..
checking for xmake::snitch ... no
finding fmt from xmake ..
checking for xmake::fmt ... no
finding argparse from xmake ..
checking for xmake::argparse ... argparse @default
finding simdutf from xmake ..
checking for xmake::simdutf ... no
checkinfo: cannot runv(pacman --version), No such file or directory
checking for pacman ... no
checkinfo: cannot runv(emerge --version), No such file or directory
checking for emerge ... no
finding simdutf from brew ..
checking for brew ... no
finding simdutf from vcpkg ..
finding simdutf from conan ..
finding simdutf from pkgconfig ..
checking for pkg-config ... /usr/bin/pkg-config
finding simdutf from system ..
checking for simdutf ... no
finding utfcpp from xmake ..
checking for xmake::utfcpp ... utfcpp v4.0.5
finding LLVM from cmake ..
checking for cmake ... /snap/bin/cmake
finding it from the generated CMakeLists.txt:
cmake_minimum_required(VERSION 3.30.3)
project(find_package)
find_package(LLVM 19 CONFIG REQUIRED )
if(LLVM_FOUND)
   add_executable(test_LLVM test.cpp)
   target_include_directories(test_LLVM PRIVATE LLVM_INCLUDE_DIR LLVM_INCLUDE_DIRS LLVM_INCLUDE_DIR LLVM_INCLUDE_DIRS)
   target_include_directories(test_LLVM PRIVATE LLVM_CXX_INCLUDE_DIRS)
   target_link_libraries(test_LLVM PRIVATE LLVM_LIBRARY LLVM_LIBRARIES LLVM_LIBS LLVM_LIBRARY LLVM_LIBRARIES LLVM_LIBS)
endif(LLVM_FOUND)

/snap/bin/cmake /tmp/.xmake1000/240912/_1EC6F4C960BC4A108176066389346630.dir
-- The C compiler identification is GNU 14.2.0
-- The CXX compiler identification is GNU 14.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test HAVE_FFI_CALL
-- Performing Test HAVE_FFI_CALL - Success
-- Found FFI: /usr/lib/x86_64-linux-gnu/libffi.so
-- Looking for histedit.h
-- Looking for histedit.h - found
-- Found LibEdit: /usr/include (found version "2.11")
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.11")
-- Found zstd: /usr/lib/x86_64-linux-gnu/libzstd.so
-- Found LibXml2: /usr/lib/x86_64-linux-gnu/libxml2.so (found version "2.9.13")
-- Found CURL: /usr/lib/x86_64-linux-gnu/libcurl.so (found version "7.81.0")
-- Configuring done (0.5s)
-- Generating done (0.0s)
-- Build files have been written to: /tmp/.xmake1000/240912/_1EC6F4C960BC4A108176066389346630.dir
finding includes from /tmp/.xmake1000/240912/_1EC6F4C960BC4A108176066389346630.dir/CMakeFiles/test_LLVM.dir/flags.make
# CMAKE generated file: DO NOT EDIT!
# Generated by "Unix Makefiles" Generator, CMake Version 3.30

# compile CXX with /usr/bin/c++
CXX_DEFINES =

CXX_INCLUDES = -I/usr/lib/llvm-19/include

CXX_FLAGS = -O3 -DNDEBUG


finding links from /tmp/.xmake1000/240912/_1EC6F4C960BC4A108176066389346630.dir/CMakeFiles/test_LLVM.dir/link.txt
/usr/bin/c++ -O3 -DNDEBUG CMakeFiles/test_LLVM.dir/test.cpp.o -o test_LLVM

checking for cmake::LLVM ... LLVM
note: install or modify (m) these packages (pass -y to skip confirm)?
in xmake-repo:
  -> snitch 1.2.5 [toolchains:"zig"]
  -> fmt 11.0.2 [toolchains:"zig", header_only:y]
  -> simdutf v5.5.0 [toolchains:"zig"]
please input: y (y/n/m)
y
checking for ping ... /usr/bin/ping
pinging the host(github.com) ... 44 ms
/usr/bin/tar -xf v1.2.5.tar.gz -C source.tmp
checking for /usr/local/bin/zig cc ... ok
checking for the c compiler (cc) ... zig cc
checking for /usr/local/bin/zig c++ ... ok
checking for the c++ compiler (cxx) ... zig c++
checking for /usr/local/bin/zig ... ok
checking for the static library archiver (ar) ... zig
checking for /usr/local/bin/zig c++ ... ok
checking for the linker (ld) ... zig c++
checking for /usr/local/bin/zig cc ... ok
checking for flags (-fPIC) ... ok
> zig cc "-fPIC" "-target" "x86_64-linux-gnu"
checking for /usr/local/bin/zig c++ ... ok
checking for flags (-fPIC) ... ok
> zig c++ "-fPIC" "-target" "x86_64-linux-gnu"
checking for gcc ... /usr/bin/gcc
checking for the assember (as) ... gcc
checking for /usr/bin/gcc ... ok
checking for flags (-fPIC) ... ok
> gcc "-fPIC"
checking for flags (-fPIC) ... ok
> zig c++ "-fPIC" "-target" "x86_64-linux-gnu" "-target" "x86_64-linux-gnu"
checking for /usr/local/bin/zig c++ ... ok
checking for the shared library linker (sh) ... zig c++
checking for flags (-fPIC) ... ok
> zig c++ "-fPIC" "-shared" "-target" "x86_64-linux-gnu" "-target" "x86_64-linux-gnu"
/snap/bin/cmake -DSNITCH_DO_TEST=OFF -DSNITCH_CREATE_HEADER_ONLY=OFF -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DSNITCH_DEFINE_MAIN=ON -DCMAKE_INSTALL_PREFIX=/home/sid/.xmake/packages/s/snitch/1.2.5/14ad55b1337f4c69825194568c3595cb -DCMAKE_INSTALL_LIBDIR:PATH=lib -G "Unix Makefiles" -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=BOTH -DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER -DCMAKE_AR=/usr/local/bin/zig -DCMAKE_FIND_USE_INSTALL_PREFIX=0 "-DCMAKE_C_COMPILER=/usr/local/bin/zig cc" -DCMAKE_FIND_USE_CMAKE_SYSTEM_PATH=0 -DCMAKE_STATIC_LINKER_FLAGS= -DHAVE_FLAG_SEARCH_PATHS_FIRST=0 -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=BOTH -DCMAKE_OSX_SYSROOT= "-DCMAKE_CXX_COMPILER=/usr/local/bin/zig c++" -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=BOTH /home/sid/.xmake/cache/packages/2409/s/snitch/1.2.5/source
-- The CXX compiler identification is unknown
CMake Error at CMakeLists.txt:3 (project):
  The CMAKE_CXX_COMPILER:

    /usr/local/bin/zig c++

  is not a full path to an existing compiler tool.

  Tell CMake where to find the compiler by setting either the environment
  variable "CXX" or the CMake cache entry CMAKE_CXX_COMPILER to the full path
  to the compiler, or to the compiler name if it is in the PATH.


-- Configuring incomplete, errors occurred!
error: @programdir/core/sandbox/modules/os.lua:378: execv(/snap/bin/cmake -DSNITCH_DO_TEST=OFF -DSNITCH_CREATE_HEADER_ONLY=OFF -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DSNITCH_DEFINE_MAIN=ON -DCMAKE_INSTALL_PREFIX=/home/sid/.xmake/packages/s/snitch/1.2.5/14ad55b1337f4c69825194568c3595cb -DCMAKE_INSTALL_LIBDIR:PATH=lib -G "Unix Makefiles" -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=BOTH -DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER -DCMAKE_AR=/usr/local/bin/zig -DCMAKE_FIND_USE_INSTALL_PREFIX=0 "-DCMAKE_C_COMPILER=/usr/local/bin/zig cc" -DCMAKE_FIND_USE_CMAKE_SYSTEM_PATH=0 -DCMAKE_STATIC_LINKER_FLAGS= -DHAVE_FLAG_SEARCH_PATHS_FIRST=0 -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=BOTH -DCMAKE_OSX_SYSROOT= "-DCMAKE_CXX_COMPILER=/usr/local/bin/zig c++" -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=BOTH /home/sid/.xmake/cache/packages/2409/s/snitch/1.2.5/source) failed(1)
stack traceback:
    [C]: in function 'error'
    [@programdir/core/base/os.lua:973]:
    [@programdir/core/sandbox/modules/os.lua:378]:
    [@programdir/core/sandbox/modules/os.lua:291]: in function 'vrunv'
    [@programdir/modules/package/tools/cmake.lua:1176]: in function 'install'
    [...make/repositories/xmake-repo/packages/s/snitch/xmake.lua:22]: in function 'script'
    [...dir/modules/private/action/require/impl/utils/filter.lua:114]: in function 'call'
    [.../modules/private/action/require/impl/actions/install.lua:404]:

  => install snitch 1.2.5 .. failed
error: @programdir/core/main.lua:329: @programdir/core/sandbox/modules/import/core/base/task.lua:65: @programdir/modules/async/runjobs.lua:325: .../modules/private/action/require/impl/actions/install.lua:506: install failed!
stack traceback:
    [C]: in function 'error'
    [@programdir/core/base/os.lua:973]:
    [.../modules/private/action/require/impl/actions/install.lua:506]: in function 'catch'
    [@programdir/core/sandbox/modules/try.lua:123]: in function 'try'
    [.../modules/private/action/require/impl/actions/install.lua:370]:
    [...modules/private/action/require/impl/install_packages.lua:491]: in function 'jobfunc'
    [@programdir/modules/async/runjobs.lua:241]:

stack traceback:
        [C]: in function 'error'
        @programdir/core/base/os.lua:973: in function 'base/os.raiselevel'
        (...tail calls...)
        @programdir/core/main.lua:329: in upvalue 'cotask'
        @programdir/core/base/scheduler.lua:406: in function <@programdir/core/base/scheduler.lua:399>

[waruqi]

Although I can support it with a wrapper script, I don't currently have the time to support the zig cc toolchain for all packages, which requires a lot of work. Not just support for cmake packages, but also for autoconf/meson packages. And there is no CI test for zig cc on xmake-repo at the moment, and there is no guarantee that all packages will support it.

Also, I think this should be a cmake issue, and it might be better to have cmake support zig c++ compiler detection with spaces.

[Sid911]

Hi, @waruqi thanks for looking at it, it makes sense that it should technically be CMake issue.

On the note of being able to run wrapper scripts for compilers, I think it would be a useful feature in my opinion although very niche use case (and probably non standard way of doing things). I was looking at xmake code and maybe I could make it happen.

Does the project want to support any similar feature in that direction?

[waruqi]

you can add it in your custom toolchain first.

like this

xmake/xmake/toolchains/xcode/xmake.lua

Line 62 in 4786482

toolchain:set("toolset", "as", path.join(os.programdir(), "scripts", "gas-preprocessor.pl " .. xc_clang))