Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

can't install k2-sherpa #127

Open
JaheimLee opened this issue Sep 20, 2022 · 9 comments
Open

can't install k2-sherpa #127

JaheimLee opened this issue Sep 20, 2022 · 9 comments

Comments

@JaheimLee
Copy link
Contributor

I installed all other dependencies using conda. And when I install sherpa using pip install --verbose k2-sherpa, I encountered the errors like k2-fsa/k2#1031 .

[ 61%] Linking CXX shared library ../../lib/libsherpa_core.so
  /usr/bin/ld: cannot find -lmkl_intel_ilp64
  /usr/bin/ld: cannot find -lmkl_core
  /usr/bin/ld: cannot find -lmkl_intel_thread
  collect2: error: ld returned 1 exit status
  sherpa/csrc/CMakeFiles/sherpa_core.dir/build.make:293: recipe for target 'lib/libsherpa_core.so' failed
  make[2]: *** [lib/libsherpa_core.so] Error 1
  CMakeFiles/Makefile2:279: recipe for target 'sherpa/csrc/CMakeFiles/sherpa_core.dir/all' failed
  make[1]: *** [sherpa/csrc/CMakeFiles/sherpa_core.dir/all] Error 2
  Makefile:135: recipe for target 'all' failed
  make: *** [all] Error 2

I tried to add path of libmkl by LD_LIBRARY_PATH like:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/data/miniconda3/envs/k2_py38/lib/

Still not work.
@csukuangfj
Copy link
Collaborator

csukuangfj commented Sep 20, 2022 •
edited
Loading

I installed all other dependencies using conda

What is the command you used to install the dependencies?

Also, what is your OS?

@JaheimLee
Copy link
Contributor Author

conda install

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/data/miniconda3/envs/k2_py38/lib/

My OS is Ubuntu18.04. And my Python version is 3.9.13. I use conda like:

conda install pytorch=1.12.0 torchvision torchaudio cudatoolkit=10.2 -c pytorch
conda install \
  -c k2-fsa \
  -c k2-fsa-sherpa \
  -c kaldifeat \
  -c kaldi_native_io \
  k2 \
  sherpa \
  kaldifeat \
  kaldi_native_io
  websockets
  sentencepiece

And here is all my packages:

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                  2_kmp_llvm    conda-forge
aiofiles                  0.8.0              pyhd8ed1ab_0    conda-forge
asttokens                 2.0.8              pyhd8ed1ab_0    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                        py_2    conda-forge
backports.functools_lru_cache 1.6.4              pyhd8ed1ab_0    conda-forge
blas                      2.116                       mkl    conda-forge
blas-devel                3.9.0            16_linux64_mkl    conda-forge
brotlipy                  0.7.0           py39hb9d737c_1004    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
ca-certificates           2022.9.14            ha878542_0    conda-forge
certifi                   2022.9.14          pyhd8ed1ab_0    conda-forge
cffi                      1.15.1           py39he91dace_0    conda-forge
charset-normalizer        2.1.1              pyhd8ed1ab_0    conda-forge
click                     8.1.3            py39hf3d152e_0    conda-forge
cryptography              37.0.4           py39hd97740a_0    conda-forge
cudatoolkit               10.2.89             h713d32c_10    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
executing                 1.0.0              pyhd8ed1ab_0    conda-forge
ffmpeg                    4.3                  hf484d3e_0    pytorch
freetype                  2.12.1               hca18f0e_0    conda-forge
gmp                       6.2.1                h58526e2_0    conda-forge
gnutls                    3.6.13               h85f3911_1    conda-forge
gunicorn                  20.1.0           py39hf3d152e_2    conda-forge
h11                       0.13.0             pyhd8ed1ab_1    conda-forge
httptools                 0.5.0            py39hb9d737c_0    conda-forge
idna                      3.3                pyhd8ed1ab_0    conda-forge
ipython                   8.5.0              pyh41d4057_1    conda-forge
jedi                      0.18.1             pyhd8ed1ab_2    conda-forge
jpeg                      9e                   h166bdaf_2    conda-forge
k2                        1.19.dev20220823 cuda10.2_py3.9_torch1.12.0    k2-fsa
kaldi_native_io           1.14                      py3.9    kaldi_native_io
kaldifeat                 1.19.2          cuda10.2_py3.9_torch1.12.0    kaldifeat
lame                      3.100             h7f98852_1001    conda-forge
lcms2                     2.12                 hddcbb42_0    conda-forge
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libblas                   3.9.0            16_linux64_mkl    conda-forge
libcblas                  3.9.0            16_linux64_mkl    conda-forge
libdeflate                1.14                 h166bdaf_0    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 12.1.0              h8d9b700_16    conda-forge
libgfortran-ng            12.1.0              h69a702a_16    conda-forge
libgfortran5              12.1.0              hdcd56e2_16    conda-forge
libgomp                   12.1.0              h8d9b700_16    conda-forge
libiconv                  1.17                 h166bdaf_0    conda-forge
liblapack                 3.9.0            16_linux64_mkl    conda-forge
liblapacke                3.9.0            16_linux64_mkl    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libpng                    1.6.37               h753d276_4    conda-forge
libsqlite                 3.39.3               h753d276_0    conda-forge
libstdcxx-ng              12.1.0              ha89aaad_16    conda-forge
libtiff                   4.4.0                h55922b4_4    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libuv                     1.44.2               h166bdaf_0    conda-forge
libwebp-base              1.2.4                h166bdaf_0    conda-forge
libxcb                    1.13              h7f98852_1004    conda-forge
libzlib                   1.2.12               h166bdaf_3    conda-forge
llvm-openmp               14.0.4               he0ac6c6_0    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
mkl                       2022.1.0           h84fe81f_915    conda-forge
mkl-devel                 2022.1.0           ha770c72_916    conda-forge
mkl-include               2022.1.0           h84fe81f_915    conda-forge
multidict                 6.0.2            py39hb9d737c_1    conda-forge
ncurses                   6.3                  h27087fc_1    conda-forge
nettle                    3.6                  he412f7d_0    conda-forge
numpy                     1.23.3           py39hba7629e_0    conda-forge
openh264                  2.1.1                h780b84a_0    conda-forge
openjpeg                  2.5.0                h7d73246_1    conda-forge
openssl                   1.1.1q               h166bdaf_0    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
pexpect                   4.8.0              pyh9f0ad1d_2    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    9.2.0            py39hd5dbb17_2    conda-forge
pip                       22.2.2             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.31             pyha770c72_0    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pydub                     0.25.1             pyhd8ed1ab_0    conda-forge
pygments                  2.13.0             pyhd8ed1ab_0    conda-forge
pyopenssl                 22.0.0             pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
python                    3.9.13          h9a8a25e_0_cpython    conda-forge
python_abi                3.9                      2_cp39    conda-forge
pytorch                   1.12.0          py3.9_cuda10.2_cudnn7.6.5_0    pytorch
pytorch-mutex             1.0                        cuda    pytorch
readline                  8.1.2                h0f457ee_0    conda-forge
requests                  2.28.1             pyhd8ed1ab_1    conda-forge
sanic                     22.6.2           py39hf3d152e_0    conda-forge
sanic-routing             22.3.0             pyhd8ed1ab_0    conda-forge
sentencepiece             0.1.96           py39hf939315_1    conda-forge
setuptools                65.3.0             pyhd8ed1ab_1    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
sqlite                    3.39.3               h4ff8645_0    conda-forge
stack_data                0.5.0              pyhd8ed1ab_0    conda-forge
tbb                       2021.5.0             h924138e_2    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
torchaudio                0.12.0               py39_cu102    pytorch
torchvision               0.13.0               py39_cu102    pytorch
traitlets                 5.4.0              pyhd8ed1ab_0    conda-forge
typing_extensions         4.3.0              pyha770c72_0    conda-forge
tzdata                    2022c                h191b570_0    conda-forge
ujson                     5.5.0            py39h5a03fae_0    conda-forge
urllib3                   1.26.11            pyhd8ed1ab_0    conda-forge
uvicorn                   0.18.3           py39hf3d152e_0    conda-forge
uvloop                    0.17.0           py39hb9d737c_0    conda-forge
wcwidth                   0.2.5              pyh9f0ad1d_2    conda-forge
websockets                10.3             py39hb9d737c_0    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
xorg-libxau               1.0.9                h7f98852_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
zlib                      1.2.12               h166bdaf_3    conda-forge
zstd                      1.5.2                h6239696_4    conda-forge

@JaheimLee
Copy link
Contributor Author

The above env 'k2_py38' is not correct, just ignore it

@csukuangfj
Copy link
Collaborator 

  -c k2-fsa \
  -c k2-fsa-sherpa \
  -c kaldifeat \
  -c kaldi_native_io \
  k2 \
  sherpa \
  kaldifeat \
  kaldi_native_io
  websockets
  sentencepiece

I see that you have installed sherpa using conda install. But the error seems to be from pip install.
Is there something wrong?

@JaheimLee
Copy link
Contributor Author 

  -c k2-fsa \
  -c k2-fsa-sherpa \
  -c kaldifeat \
  -c kaldi_native_io \
  k2 \
  sherpa \
  kaldifeat \
  kaldi_native_io
  websockets
  sentencepiece

I see that you have installed sherpa using conda install. But the error seems to be from pip install. Is there something wrong?

Sorry, no sherpa installed. I copied it from your doc here. I forgot to remove it. There is no sherpa in my env.

@JaheimLee
Copy link
Contributor Author

By the way, I tried to install sherpa using conda, and it did work, even running on GPU. But you said conda installation of sherpa is to be done according to your doc.

@csukuangfj
Copy link
Collaborator

But you said conda installation of sherpa is to be done according to your doc.

The installation commands using conda install for CPU and GPU are different. For instance, for the GPU version, you need to install a GPU version of PyTorch, which needs to select which cudatoolkit to use.

The doc has not been updated to describe how to install a GPU version of sherpa.

@csukuangfj
Copy link
Collaborator

By the way, I just tried

conda install -c k2-fsa -c kaldifeat -c pytorch cpuonly k2 kaldifeat pytorch=1.12.0 python=3.8
pip install --verbose k2-sherpa

It works like a charm and I cannot reproduce your issue.

@JaheimLee
Copy link
Contributor Author

By the way, I just tried

conda install -c k2-fsa -c kaldifeat -c pytorch cpuonly k2 kaldifeat pytorch=1.12.0 python=3.8
pip install --verbose k2-sherpa

It works like a charm and I cannot reproduce your issue.

I didn't use cpuonly pytorch. I use it with cuda 10.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@csukuangfj @JaheimLee