ImportError: Невозможно найти общую библиотеку: libhdfs3.so с Python Anaconda - PullRequest
Новая
       34

ImportError: Невозможно найти общую библиотеку: libhdfs3.so с Python Anaconda

1 голос
/ 05 ноября 2019

Работа с приведенной ниже версией Python: 

   (base) [xxxx@xxx-xxx-000 lib]# python
    Python 3.7.3 (default, Mar 27 2019, 22:11:17)
    [GCC 7.3.0] :: Anaconda, Inc. on linux
    Type "help", "copyright", "credits" or "license" for more information.

Доступные пакеты conda: 

(base) [xxxx@xxx-xxx-000 lib]# conda list hdfs3
WARNING: The conda.compat module is deprecated and will be removed in a future release.
# packages in environment at /opt/anaconda3:
#
# Name                    Version                   Build  Channel
hdfs3                     0.3.1                    py37_0    <unknown>
libhdfs3                  2.3               h311b756_1006    <unknown>

Импорт пакетов Python: 

>>> from hdfs3 import HDFileSystem
>>> hdfs = HDFileSystem(host='localhost', port=8020)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/anaconda3/lib/python3.7/site-packages/hdfs3/core.py", line 88, in __init__
    self.connect()
  File "/opt/anaconda3/lib/python3.7/site-packages/hdfs3/core.py", line 106, in connect
    get_lib()
  File "/opt/anaconda3/lib/python3.7/site-packages/hdfs3/core.py", line 670, in get_lib
    from .lib import _lib as l
  File "/opt/anaconda3/lib/python3.7/site-packages/hdfs3/lib.py", line 24, in <module>
    raise ImportError("Can not find the shared library: libhdfs3.so\n"
ImportError: Can not find the shared library: libhdfs3.so

Какое местоположение он на самом деле проверяетобщая библиотека? Я настроил расположение LD_LIBRARY_PATH и PATH с указанными ниже подробностями. 

(base) [xxxx@xxx-xxx-000 lib]# python -c "import sys; print(sys.path)"
['', '/opt/anaconda3/lib/python37.zip', '/opt/anaconda3/lib/python3.7', '/opt/anaconda3/lib/python3.7/lib-dynload', '/opt/anaconda3/lib/python3.7/site-packages']

(base) [xxxx@xxx-xxx-000 lib]# echo $LD_LIBRARY_PATH
/opt/anaconda3/lib/:/home/zeppelin/.conda/pkgs/glog-0.4.0-he1b5a44_1/lib/:/opt/anaconda3/pkgs/glog-0.4.0-he1b5a44_1/lib/:/usr/local/lib/:/usr/local/lib:/opt/anaconda3/lib/:/home/zeppelin/.conda/pkgs/glog-0.4.0-he1b5a44_1/lib/:/opt/anaconda3/pkgs/glog-0.4.0-he1b5a44_1/lib/

Расположение libhdfs3.so на моем сервере ниже: 

    (base) [xxxx@xxx-xxx-000 lib]# sudo find / -name libhdfs3.so
    /opt/anaconda3/lib/libhdfs3.so
    /opt/anaconda3/pkgs/libhdfs3-2.3-h311b756_1006/lib/libhdfs3.so
    /opt/anaconda2/lib/libhdfs3.so
    /opt/anaconda2/pkgs/libhdfs3-2.3-h311b756_1006/lib/libhdfs3.so

Вот выводкоманда ldd: 

(base) [xxxx@xxx-xxx-000 pandas]# ldd /opt/anaconda3/lib/libhdfs3.so
/opt/anaconda3/lib/libhdfs3.so: /opt/anaconda3/lib/./libuuid.so.1: no version information available (required by /opt/anaconda3/lib/libhdfs3.so)
        linux-vdso.so.1 =>  (0x00007fffb18bc000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f369138f000)
        libuuid.so.1 => /opt/anaconda3/lib/./libuuid.so.1 (0x00007f369118b000)
        libprotobuf.so.19 => not found
        libxml2.so.2 => /opt/anaconda3/lib/./libxml2.so.2 (0x00007f3691987000)
        libkrb5.so.3 => /opt/anaconda3/lib/./libkrb5.so.3 (0x00007f36910af000)
        libgsasl.so.7 => not found
        libcrypto.so.1.1 => /opt/anaconda3/lib/./libcrypto.so.1.1 (0x00007f3690db7000)
        libcurl.so.4 => /opt/anaconda3/lib/./libcurl.so.4 (0x00007f3690d2f000)
        libstdc++.so.6 => /opt/anaconda3/lib/./libstdc++.so.6 (0x00007f3690bee000)
        libgcc_s.so.1 => /opt/anaconda3/lib/./libgcc_s.so.1 (0x00007f3691970000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f3690821000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f36918e1000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f369061d000)
        libicuuc.so.58 => /opt/anaconda3/lib/././libicuuc.so.58 (0x00007f369026f000)
        libz.so.1 => /opt/anaconda3/lib/././libz.so.1 (0x00007f369194e000)
        liblzma.so.5 => /opt/anaconda3/lib/././liblzma.so.5 (0x00007f3690049000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f368fd47000)
        libk5crypto.so.3 => /opt/anaconda3/lib/././libk5crypto.so.3 (0x00007f369192e000)
        libcom_err.so.3 => /opt/anaconda3/lib/././libcom_err.so.3 (0x00007f3691928000)
        libkrb5support.so.0 => /opt/anaconda3/lib/././libkrb5support.so.0 (0x00007f3691919000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f368fb2e000)
        libssh2.so.1 => /opt/anaconda3/lib/././libssh2.so.1 (0x00007f368fafd000)
        libssl.so.1.1 => /opt/anaconda3/lib/././libssl.so.1.1 (0x00007f368fa67000)
        libgssapi_krb5.so.2 => /opt/anaconda3/lib/././libgssapi_krb5.so.2 (0x00007f368fa18000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f368f810000)
        libicudata.so.58 => /opt/anaconda3/lib/./././libicudata.so.58 (0x00007f368dd10000)
...