onnxruntime installation in conda environment

[Prithvi-Shah]

Hi onnx community! Good day, hope all is well!

I am trying to install onnxruntime on the following machine to include onnx inference support for C/C++.

Host Details -
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

From what I understand, I can install onnxruntime by using docker, dotnet, using the precompiled shared object or building from source. I am currently trying out with the precompiled .so file and building from source via a conda environment.

PREBUILT SO
Steps taken -
Downloaded onnxruntime-linux-x64-1.15.0 from here https://github.com/microsoft/onnxruntime/releases/download/v1.15.0/onnxruntime-linux-x64-1.15.0.tgz

Followed this thread - #6489
When I try to compile the model_explorer.cpp from here - https://github.com/microsoft/onnxruntime-inference-examples/blob/main/c_cxx/model-explorer/model-explorer.cpp

I get the following error on my machine -
g++ -o model-explorer model-explorer.cpp -I/var/tellme/pshah/Installations/ONNX/onnxruntime-linux-x64-1.15.0/include -L/var/tellme/pshah/Installations/ONNX/onnxruntime-linux-x64-1.15.0/lib -lonnxruntime

ERROR HERE
/var/tellme/pshah/Installations/anaconda3/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /var/tellme/pshah/Installations/ONNX/onnxruntime-linux-x64-1.15.0/lib/libonnxruntime.so: undefined reference to memcpy@GLIBC_2.14' /var/tellme/pshah/Installations/anaconda3/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /var/tellme/pshah/Installations/ONNX/onnxruntime-linux-x64-1.15.0/lib/libonnxruntime.so: undefined reference to aligned_alloc@GLIBC_2.16'
collect2: error: ld returned 1 exit status

I am not sure how to resolve this GLIBC issue on the conda environment. There are similar discussions here on the glibc issue around memcpy - https://stackoverflow.com/questions/12286460/undefined-reference-to-mempcyglibc-2-14-when-compiling-on-linux

Should I try to override the memcpy function/install parallel glibc versions to fix this issue or is there another way? How would one do that in their conda environment?

When I try to build from source for inference as mentioned here - https://onnxruntime.ai/docs/build/inferencing.html for linux, centos 7, I run into the same issue - (again doing this from the conda environment)

Build command -
./build.sh --config RelWithDebInfo --build_shared_lib --parallel --compile_no_warning_as_error --skip_submodule_sync --cmake_extra_defines CMAKE_OSX_ARCHITECTURES=x86_64

ERROR HERE
[ 33%] Linking CXX executable protoc
/var/tellme/pshah/Installations/anaconda3/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib64/libz.so: undefined reference to `memcpy@GLIBC_2.14'
collect2: error: ld returned 1 exit status
gmake[2]: *** [_deps/protobuf-build/protoc-3.21.12.0] Error 1
gmake[1]: *** [_deps/protobuf-build/CMakeFiles/protoc.dir/all] Error 2
gmake: *** [all] Error 2
Traceback (most recent call last):
File "/var/tellme/pshah/Installations/onnxruntime/onnxruntime/tools/ci_build/build.py", line 2634, in
sys.exit(main())
File "/var/tellme/pshah/Installations/onnxruntime/onnxruntime/tools/ci_build/build.py", line 2527, in main
build_targets(args, cmake_path, build_dir, configs, num_parallel_jobs, args.target)
File "/var/tellme/pshah/Installations/onnxruntime/onnxruntime/tools/ci_build/build.py", line 1454, in build_targets
run_subprocess(cmd_args, env=env)
File "/var/tellme/pshah/Installations/onnxruntime/onnxruntime/tools/ci_build/build.py", line 791, in run_subprocess
return run(*args, cwd=cwd, capture_stdout=capture_stdout, shell=shell, env=my_env)
File "/var/tellme/pshah/Installations/onnxruntime/onnxruntime/tools/python/util/run.py", line 49, in run
completed_process = subprocess.run(
File "/var/tellme/pshah/Installations/anaconda3/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/var/tellme/pshah/Installations/anaconda3/bin/cmake', '--build', '/var/tellme/pshah/Installations/onnxruntime/onnxruntime/build/Linux/RelWithDebInfo', '--config', 'RelWithDebInfo', '--', '-j4']' returned non-zero exit status 2.

I am trying out dotnet and docker alternatives, but I was wondering why this fails.
Apologies if this question is redundant, any help appreciated. Thank you very much.

Side questions -
I pulled the docker image from here - (https://hub.docker.com/_/microsoft-azureml-onnxruntimefamily/)
docker pull mcr.microsoft.com/azureml/onnxruntime
How do I use it? Was unable to find much information around how to use it for inference, apologies if this is redundant.

For using dotnet, I downloaded dotnet 7 for linux from here - https://dotnet.microsoft.com/en-us/download/dotnet/7.0
I then tarred it and set the DOTNET_ROOT and PATH variables to work with dotnet.
tar -xvf dotnet-sdk-7.0.302-linux-x64.tar.gz

Did this -
dotnet new console --name onnxruntime
cd onnxruntime/
dotnet add package Microsoft.ML.OnnxRuntime --version 1.15.0

Usual dotnet development is in C# but is it possible to extend development to C/C++ like how we do it on Visual Studio but on a remote host?

Similar threads -
#6021