🚀 Replace vcpkg with conan #641

AnotherFoxGuy · 2023-09-24T14:28:32Z

This PR is still a work in progress, but the interface should built and run on both Windows and Linux

TODO

How to build on Windows with conan
Add the remote
conan remote add overte https://git.anotherfoxguy.com/api/packages/overte/conan
Install the deps with conan
conan install . -b missing -pr:b=tools/conan-profiles/vs-19-release -pr=tools/conan-profiles/vs-19-release -of build
Generate the VS project files
cmake --preset conan-default
(Note: Currently only the Release build can be built)

How to build on Linux with conan
Let conan detect the build environment
conan profile detect
Add the remote
conan remote add overte https://git.anotherfoxguy.com/api/packages/overte/conan
Install the deps with conan
conan install . -s build_type=Release -b missing -of build
Generate the make files
cmake --preset conan-default
Build the interface
cd build && make

AnotherFoxGuy · 2023-09-24T15:12:27Z

For some reason avatars are broken in this build, no idea why

JulianGro

A little comment on vhacd-util: As far as I know this is somewhat legacy. Our engine will automatically generate collision hulls and doesn't use vhacd-util for that. vhacd-util could be very useful for generating performant collision hulls for more complicated objects, but I don't think anyone does that. In fact, I have never even tried using it, and don't even know if it works.

JulianGro · 2023-09-30T11:49:22Z

conan-recipes/nvidia-texture-tools/all/conanfile.py

+        tc.variables["BUILD_TOOLS"] = "OFF"
+        tc.variables["USE_CUDA"] = "OFF"
+        if self.settings.os == "Linux":
+            tc.variables["CMAKE_CXX_FLAGS"] = "-march=msse3 -fPIC"


This would break on aarch64, which technically Overte can currently run on.

cmake/macros/AutoScribeShader.cmake

JulianGro · 2023-09-30T11:53:51Z

cmake/macros/PackageLibrariesForDeployment.cmake

-    set(QTAUDIO_PATH "$<TARGET_FILE_DIR:${TARGET_NAME}>/audio")
-    set(QTAUDIO_WIN7_PATH "$<TARGET_FILE_DIR:${TARGET_NAME}>/audioWin7/audio")
-    set(QTAUDIO_WIN8_PATH "$<TARGET_FILE_DIR:${TARGET_NAME}>/audioWin8/audio")
+    # TODO: Is this still needed? needs testing


Windows 7 and Windows 8(.1) are End Of Life and therefore also not supported by us anymore.

JulianGro · 2023-09-30T11:56:27Z

cmake/macros/TargetJson.cmake

@@ -7,5 +7,9 @@
 # 
 macro(TARGET_JSON)
 	# We use vcpkg for both json and glm, so we just re-use the target_glm macro here


We will have to remember to remove this.

JulianGro · 2023-09-30T12:00:15Z

CMakeLists.txt

@@ -15,67 +15,32 @@ endif()
 # 3.14 is the minimum version that supports symlinks on Windows
 cmake_minimum_required(VERSION 3.14)


Don't be shy about bumping this version if there is a reason to. VCPKG required a much higher CMake version than this anyway.

JulianGro · 2023-09-30T12:11:15Z

conan-recipes/libnode/config.yml

@@ -0,0 +1,3 @@
+versions:
+  "2019.02":


Does this version the conan-recipe?

JulianGro · 2023-09-30T12:20:27Z

conanfile.py

+        "qt*:qtxmlpatterns": "True",
+        "glad*:spec": "gl",
+        "glad*:gl_profile": "core",
+        "glad*:gl_version": "4.6",


Not sure if you know, but Overte supports multiple OpenGL versions. For example, it will fall back to OpenGL 4.1 if OpenGL 4.6 is not available. It can also be built for OpenGL ES 3.2.
I don't know how this part of the Conan code works though.

This is just the targeted version for glad, I just copied the settings from the vcpkg package

JulianGro · 2023-09-30T12:21:54Z

interface/CMakeLists.txt

@@ -68,8 +68,8 @@ foreach(EXTERNAL ${OPTIONAL_EXTERNALS})
  endif ()
 endforeach()

-find_package(Qt5LinguistTools REQUIRED)
-find_package(Qt5LinguistToolsMacros)
+# find_package(Qt5LinguistTools REQUIRED)


As you might know, we would like to add (more) localization support to Overte, but this is currently unused.

JulianGro · 2023-09-30T12:25:00Z

tools/conan-profiles/vs-19-debug

@@ -0,0 +1,8 @@
+[settings]
+os=Windows
+arch=x86_64


We don't currently have any plans for an aarch64 Windows version, but would it make sense to have the profile name reflect the architecture?

JulianGro · 2023-09-30T12:35:03Z

tools/shadergen.py

@@ -217,7 +217,7 @@ def processCommand(line):
        executeSubprocess(scribeArgs)

        # Generate the un-optimized output
-        executeSubprocess([glslangExec, '-V110', '-o', upoptSpirvFile, unoptGlslFile])
+        executeSubprocess([glslangExec, '-V100', '-o', upoptSpirvFile, unoptGlslFile])


Could you explain what this change is doing? Looking it up online it seems like this switches to an older Visual Studio toolchain?

The glslangValidator from the conan package is too old to support that version, it needs to be updated

JulianGro · 2023-10-01T05:54:51Z

A comment on building this on Linux:
Since Conan is not available as a distribution package (at least not on Debian), but rather a pip package, the installation procedure is not that trivial.

# Install python3 virtual environment
sudo apt install python3-venv
# Create a virtual environment in the Overte codebase
python3 -m venv .venv
# Install pip dependencies
.venv/bin/pip3 install conan
# Now one can run conan
.venv/bin/conan profile detect

JulianGro · 2023-10-01T16:53:18Z

Am I understanding it right that Conan is supposed to only get dependencies of its own if the system doesn't already provide them?
If yes, we should probably always depend on a version range, from the oldest version that we know works, to the newest version that we know works. With libopus we would get version 1.3.1 from the system on Ubuntu 20.04.
Or does Conan "only" provide the tools for us to conditionally use system packages on our own?

AnotherFoxGuy · 2023-10-01T17:16:34Z

Am I understanding it right that Conan is supposed to only get dependencies of its own if the system doesn't already provide them?

That is really dependent on how the conan recipes are written.
The recipes provided by the conan-center are always using conan packages to make packaging easier and more reproducible.
But it is possible to script conan to install system packages, for example the xorg recipe just installs the appropriate system packages https://github.com/conan-io/conan-center-index/blob/master/recipes/xorg/all/conanfile.py#L26

JulianGro · 2023-10-02T08:30:31Z

It would be nice if you could put a README into the conan-recipes directory on how to consume those and how to update them.

It seems like I can build node for example using a command like

../../.venv/bin/conan create all/conanfile.py --version 18.17.1

JulianGro · 2023-10-02T21:04:59Z

Something to figure out later: The resulting binary seems to be searching for the Qt platform plugins in the wrong directory.

juliangro@x299-desktop:~/git/overte/build$ QT_DEBUG_PLUGINS=1 ./interface/interface --disable-displays "OpenVR (Vive)" --disable-inputs "OpenVR (Vive)","Sdl2" --allowMultipleInstances
[10/02 23:02:35] [DEBUG] [hifi.shared] Settings file: "/home/juliangro/.config/Overte - dev/Interface.json"
[10/02 23:02:35] [DEBUG] [settings.manager] Initializing settings write thread
[10/02 23:02:35] [WARNING] [default] QEventLoop: Cannot be used without QApplication
[10/02 23:02:35] [DEBUG] [default] UserActivityLogger is enabled: false
[10/02 23:02:35] [INFO] [default] Disabling following display plugins: ("OpenVR (Vive)")
[10/02 23:02:35] [WARNING] [default] DependencyManager::get(): No instance available for 13PluginManager
[10/02 23:02:35] [INFO] [default] Disabling following input plugins: ("OpenVR (Vive)", "Sdl2")
[10/02 23:02:35] [WARNING] [default] DependencyManager::get(): No instance available for 13PluginManager
[10/02 23:02:35] [DEBUG] [default] QFactoryLoader::QFactoryLoader() checking directory path "/home/juliangro/git/overte/build/interface/platforms" ...
[10/02 23:02:35] [WARNING] [qt.qpa.plugin] Could not find the Qt platform plugin "xcb" in ""
[10/02 23:02:35] [FATAL] [default] This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
[10/02 23:02:35] [FATAL] [default] 
Abgebrochen (Speicherabzug geschrieben)
juliangro@x299-desktop:~/git/overte/build$

JulianGro · 2023-10-02T21:41:03Z

Building libnode through conan fails on Ubuntu 20.04.
profile detect seems to have chosen C++14, while libnode seems to require C++17 or higher to build.
Log: https://bin.linux.pizza/?3ad49c3deaebf232#5NxEbsMUNagxtaMvnFJxm3dpWXTt2Ja5CJQrYio89wAR

JulianGro · 2023-10-07T18:33:22Z

Just tried on aarch64 postmarketOS. (I cannot select text or pipe it to file for some reason.)

It looks like it is trying to install gl through the Alpine Package manager, which is not a valid package name. Unfortunately, Conan Center is having issues, so I don't know how to take a look at the source code.

AnotherFoxGuy · 2023-10-07T18:58:14Z

I don't know how to take a look at the source code.

The source for the package is here:
https://github.com/conan-io/conan-center-index/blob/master/recipes/opengl/all/conanfile.py

JulianGro · 2023-10-07T20:14:48Z

Installing a bunch of packages fixed the gl issue.

This uses a temporary prebuild package to build, it would be better to build WebRTC from sources

This breaks the interface

)

AnotherFoxGuy force-pushed the conan branch from f10011e to c7a34e6 Compare September 24, 2023 15:02

AnotherFoxGuy force-pushed the conan branch 2 times, most recently from 1f6bfe1 to 705a375 Compare September 27, 2023 11:55

AnotherFoxGuy marked this pull request as ready for review September 27, 2023 14:04

AnotherFoxGuy requested a review from JulianGro as a code owner September 27, 2023 14:04

JulianGro reviewed Sep 30, 2023

View reviewed changes

This comment was marked as resolved.

Sign in to view

AnotherFoxGuy force-pushed the conan branch from 7c7d9d8 to e2fadc3 Compare October 2, 2023 21:18

JulianGro force-pushed the conan branch from 114c5d2 to 2bec5c7 Compare October 9, 2023 07:34

AnotherFoxGuy force-pushed the conan branch from 2bec5c7 to 752e0f2 Compare November 8, 2023 10:02

AnotherFoxGuy force-pushed the conan branch 3 times, most recently from becf99d to 8d93025 Compare November 24, 2023 09:16

daleglass mentioned this pull request Dec 4, 2023

Upgrade openexr #752

Merged

AnotherFoxGuy force-pushed the conan branch 2 times, most recently from e98cdb9 to c699930 Compare December 7, 2023 10:32

AnotherFoxGuy and others added 26 commits September 25, 2024 10:58

🐛 Fixed missing WIN32 definition when building with ninja

6706841

🐛 Fixed OpenVR failing to link on windows

dc0f03f

🔧 Reenabled build with WebRTC

7943457

This uses a temporary prebuild package to build, it would be better to build WebRTC from sources

📦 Added SteamWorks Conan package

a31b78d

📦 Added OVR Conan package

6998f72

📦 Added NeuronDataReader Conan package

259030b

📦 Added GifCreator Conan package

8dcec58

🔥 Removed external cmake packages

d3e441f

🔧 Use conan packages instead of cmake external projects

79daf1f

🐛 Fixed dependencies on Linux

727681b

🔧 Don't generate a qt.conf on Linux

b733a6e

This breaks the interface

👷 Updated gh actions

f8e64b2

🐛 Fixed building manual tests

5181ba3

🐛 Fixed building tests

d11aa3f

📦 Added cgltf Conan package

346092c

🐛 Fixed polyvox trying to build bindings

b5cf984

🐛 Fixed missing libraries on Linux

eb05679

🔧 Simplify FixupBundlePostBuild

db44995

⬆️ Updated liblo and node package

2688256

⬇️ Downgraded GLM

df05172

🔧 Disable WebRTC by default

071c177

Fix hang on Linux due to some sort of bug in Qt's SSL initialization (#1

3aaef6e

)

⬆️ Updated SDL to 2.30.3

1f5900d

⬆️ Updated cgltf to 1.14

9251c71

🔧 Added artery-font-format

e35e1bc

⬆️ Updated google-webrtc to 124

a9190fc

AnotherFoxGuy force-pushed the conan branch from 94392c2 to e7e0529 Compare September 30, 2024 09:05

AnotherFoxGuy added 2 commits October 20, 2024 10:59

🔧 Use system OpenSSL on Linux

35f2231

🔧 Switch back to prebuild WebRTC for now

e26b0c6

AnotherFoxGuy force-pushed the conan branch from a63773d to e26b0c6 Compare October 20, 2024 08:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🚀 Replace vcpkg with conan #641

🚀 Replace vcpkg with conan #641

AnotherFoxGuy commented Sep 24, 2023 •

edited

Loading

AnotherFoxGuy commented Sep 24, 2023 •

edited

Loading

JulianGro left a comment

JulianGro Sep 30, 2023

JulianGro Sep 30, 2023

JulianGro Sep 30, 2023

JulianGro Sep 30, 2023

JulianGro Sep 30, 2023

JulianGro Sep 30, 2023

AnotherFoxGuy Sep 30, 2023

JulianGro Sep 30, 2023

JulianGro Sep 30, 2023

JulianGro Sep 30, 2023

AnotherFoxGuy Sep 30, 2023

JulianGro commented Oct 1, 2023

This comment was marked as resolved.

This comment was marked as resolved.

JulianGro commented Oct 1, 2023

AnotherFoxGuy commented Oct 1, 2023

This comment was marked as resolved.

JulianGro commented Oct 2, 2023 •

edited

Loading

This comment was marked as resolved.

This comment was marked as resolved.

JulianGro commented Oct 2, 2023

JulianGro commented Oct 2, 2023

JulianGro commented Oct 7, 2023

AnotherFoxGuy commented Oct 7, 2023

JulianGro commented Oct 7, 2023

		@@ -15,67 +15,32 @@ endif()
		# 3.14 is the minimum version that supports symlinks on Windows
		cmake_minimum_required(VERSION 3.14)

🚀 Replace vcpkg with conan #641

Are you sure you want to change the base?

🚀 Replace vcpkg with conan #641

Conversation

AnotherFoxGuy commented Sep 24, 2023 • edited Loading

AnotherFoxGuy commented Sep 24, 2023 • edited Loading

JulianGro left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JulianGro commented Oct 1, 2023

This comment was marked as resolved.

This comment was marked as resolved.

JulianGro commented Oct 1, 2023

AnotherFoxGuy commented Oct 1, 2023

This comment was marked as resolved.

JulianGro commented Oct 2, 2023 • edited Loading

This comment was marked as resolved.

This comment was marked as resolved.

JulianGro commented Oct 2, 2023

JulianGro commented Oct 2, 2023

JulianGro commented Oct 7, 2023

AnotherFoxGuy commented Oct 7, 2023

JulianGro commented Oct 7, 2023

AnotherFoxGuy commented Sep 24, 2023 •

edited

Loading

AnotherFoxGuy commented Sep 24, 2023 •

edited

Loading

JulianGro commented Oct 2, 2023 •

edited

Loading