sherpa-onnx 能否使用 cuda-12.4.0 、 cudnn-8.9.x 实现 GPU 运算

使用 GPU  运行模型，执行命令如下



>./sherpa-onnx/build/bin/sherpa-onnx-offline \

  --tokens=./models/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-2024-07-17/tokens.txt \

  --sense-voice-model=./models/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-2024-07-17/model.onnx \

  --num-threads=2 \

  --debug=1 \

  --provider=cuda \

  ./models/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-2024-07-17/test_wavs/zh.wav \

  ./models/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-2024-07-17/test_wavs/en.wav



报错如下：



Creating recognizer ...

**terminate called after throwing an instance of 'Ort::Exception'

  what():  /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1209 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.11: cannot open shared object file: No such file or directory**



cuda 和 cudnn 安装方式参考文档是： https://k2-fsa.github.io/k2/installation/cuda-cudnn.html#cuda-12-4 。



`which nvcc`



/home/voice11/repositories/cuda_env/cuda-12.4.0/bin/nvcc



`nvcc --version`



nvcc: NVIDIA (R) Cuda compiler driver

Copyright (c) 2005-2024 NVIDIA Corporation

Built on Tue_Feb_27_16:19:38_PST_2024

Cuda compilation tools, release 12.4, V12.4.99

Build cuda_12.4.r12.4/compiler.33961263_0



因为报错的代码来自 onnxruntime_src， 我先去 onnxruntime github 中查找了相关问题：

- https://github.com/microsoft/onnxruntime/issues/21684

- https://github.com/microsoft/onnxruntime/issues/20944



看描述，cuda12.x 中的库文件是 libcublasLt.so.12 , 但是  libonnxruntime_providers_cuda.so 要找的是 libcublasLt.so.11



查看 ldd /home/voice11/repositories/sherpa-onnx/build/_deps/onnxruntime-src/lib/libonnxruntime_providers_cuda.so ，的确有不少库文件丢失



>linux-vdso.so.1 (0x00007ffc3e1b4000)

libcublasLt.so.11 => not found

libcublas.so.11 => not found

libcudnn.so.8 => /home/voice11/repositories/cuda_env/cuda-12.4.0/lib/libcudnn.so.8 (0x00007b6d5de00000)

libcurand.so.10 => /home/voice11/repositories/cuda_env/cuda-12.4.0/lib64/libcurand.so.10 (0x00007b6d57600000)

libcufft.so.10 => not found

libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007b6d7b525000)

librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007b6d7b520000)

libcudart.so.11.0 => not found

libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007b6d57200000)

libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007b6d7b439000)

libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007b6d7b417000)

libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007b6d56e00000)

/lib64/ld-linux-x86-64.so.2 (0x00007b6d7b53c000)

libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007b6d7b412000)



再查看 /home/voice11/repositories/sherpa-onnx/build/_deps/onnxruntime-src/lib/libonnxruntime.so



>linux-vdso.so.1 (0x00007ffc1e2de000)

libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x000074a40d2ca000)

librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x000074a40d2c5000)

libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x000074a40d2a9000)

libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x000074a40d2a4000)

libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x000074a40b800000)

libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x000074a40d1bb000)

libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x000074a40d19b000)

libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x000074a40b400000)

/lib64/ld-linux-x86-64.so.2 (0x000074a40d2df000)

 

并没有使用  libonnxruntime_providers_cuda.so，所以并不清楚 libonnxruntime_providers_cuda.so  是怎么跑出来的，有一个相关代码改动是： https://github.com/k2-fsa/sherpa-onnx/pull/1210 。



至此，排查就不知道如何进行下去了。

[]