Ошибка использования Tensorflow GPU на кластере с использованием Conda - PullRequest
0 голосов
/ 28 июня 2019

Я пытаюсь запустить программу на компьютерном кластере, который использует графический процессор Tensorflow. Я установил кондо и все необходимые файлы, как показано ниже

# packages in environment at /software/anaconda3:
#
# Name                    Version                   Build  Channel
_ipyw_jlab_nb_ext_conf    0.1.0            py36he11e457_0  
_mutex_mxnet              0.0.20                  gpu_mkl  
alabaster                 0.7.10           py36h306e16b_0  
anaconda                  custom           py36hbbc8b67_0  
anaconda-client           1.6.5            py36h19c0dcd_0  
anaconda-navigator        1.6.9            py36h11ddaaa_0  
anaconda-project          0.8.0            py36h29abdf5_0  
appdirs                   1.4.3                     <pip>
asn1crypto                0.22.0           py36h265ca7c_1  
astroid                   1.5.3            py36hbdb9df2_0  
astropy                   2.0.2            py36ha51211e_4  
babel                     2.5.0            py36h7d14adf_0  
backports                 1.0              py36hfa02d7e_1  
backports.shutil_get_terminal_size 1.0.0            py36hfea85ff_2  
beautifulsoup4            4.6.0            py36h49b8c8c_1  
bitarray                  0.8.1            py36h5834eb8_0  
bkcharts                  0.2              py36h735825a_0  
blaze                     0.11.3           py36h4e06776_0  
bleach                    1.5.0                    py36_0    conda-forge
bokeh                     0.12.10          py36hbb0e44a_0  
boto                      2.48.0           py36h6e4cd66_1  
bottleneck                1.2.1            py36haac1ea0_0  
bzip2                     1.0.6                h0376d23_1  
ca-certificates           2018.8.24            ha4d7672_0    conda-forge
cairo                     1.14.10              haa5651f_5  
certifi                   2018.8.24                py36_1    conda-forge
cffi                      1.10.0           py36had8d393_1  
chardet                   3.0.4            py36h0f667ec_1  
click                     6.7              py36h5253387_0  
cloudpickle               0.4.0            py36h30f8c20_0  
clyent                    1.2.2            py36h7e57e65_1  
colorama                  0.3.9            py36h489cec4_0  
conda                     4.5.11                   py36_0    conda-forge
conda-build               3.0.27           py36h940a66d_0  
conda-env                 2.6.0                         0    conda-forge
conda-verify              2.0.0            py36h98955d8_0  
contextlib2               0.5.5            py36h6c84a62_0  
cryptography              2.0.3            py36ha225213_1  
cudatoolkit               8.0                           3  
cudnn                     7.0.5                 cuda8.0_0  
curl                      7.55.1               hcb0b314_2  
cycler                    0.10.0           py36h93f1223_0  
cython                    0.26.1           py36h21c49d0_0  
cytoolz                   0.8.2            py36h708bfd4_0  
dask                      0.15.3           py36hdc2c8aa_0  
dask-core                 0.15.3           py36h10e6167_0  
datashape                 0.5.4            py36h3ad6b5c_0  
dbus                      1.10.22              h3b5a359_0  
decorator                 4.1.2            py36hd076ac8_0  
distributed               1.19.1           py36h25f3894_0  
docutils                  0.14             py36hb0f60f5_0  
entrypoints               0.2.3            py36h1aec115_2  
et_xmlfile                1.0.1            py36hd6bccc3_0  
expat                     2.2.4                hc00ebd1_1  
fastcache                 1.0.2            py36h5b0c431_0  
ffmpeg                    4.0                  h04d0a96_0  
filelock                  2.0.12           py36hacfa1f5_0  
flask                     0.12.2           py36hb24657c_0  
flask-cors                3.0.3            py36h2d857d3_0  
fontconfig                2.12.4               h88586e7_1  
freetype                  2.8                  h52ed37b_0  
get_terminal_size         1.0.0                haa9412d_0  
gevent                    1.2.2            py36h2fe25dc_0  
glib                      2.53.6               hc861d11_1  
glob2                     0.5              py36h2c1b292_1  
gmp                       6.1.2                hb3b607b_0  
gmpy2                     2.0.8            py36h55090d7_1  
graphite2                 1.3.10               hc526e54_0  
greenlet                  0.4.12           py36h2d503a6_0  
gsl                       2.2.1                h0c605f7_3  
gst-plugins-base          1.12.2               he3457e5_0  
gstreamer                 1.12.2               h4f93127_0  
h5py                      2.7.1                    py36_1    conda-forge
harfbuzz                  1.5.0                h2545bd6_0  
hdf4                      4.2.13                        0    conda-forge
hdf5                      1.8.18                        3    conda-forge
heapdict                  1.0.0            py36h79797d7_0  
html5lib                  0.9999999                py36_0    conda-forge
icu                       58.2                 h211956c_0  
idna                      2.6              py36h82fb2a8_1  
imageio                   2.2.0            py36he555465_0  
imagesize                 0.7.1            py36h52d8127_0  
intel-openmp              2018.0.3                      0  
ipykernel                 4.6.1            py36hbf841aa_0  
ipython                   6.1.0            py36hc72a948_1  
ipython_genutils          0.2.0            py36hb52b0d5_0  
ipywidgets                7.0.0            py36h7b55c3a_0  
isort                     4.2.15           py36had401c0_0  
itsdangerous              0.24             py36h93cc618_1  
jasper                    1.900.1                       4    conda-forge
jbig                      2.1                  hdba287a_0  
jdcal                     1.3              py36h4c697fb_0  
jedi                      0.10.2           py36h552def0_0  
jinja2                    2.9.6            py36h489bce4_1  
jpeg                      9c                   h470a237_0    conda-forge
jsonschema                2.6.0            py36h006f8b5_0  
jupyter                   1.0.0            py36h9896ce5_0  
jupyter_client            5.1.0            py36h614e9ea_0  
jupyter_console           5.2.0            py36he59e554_1  
jupyter_core              4.3.0            py36h357a921_0  
jupyterlab                0.27.0           py36h86377d0_2  
jupyterlab_launcher       0.4.0            py36h4d8058d_0  
krb5                      1.14.2                        0    conda-forge
lazy-object-proxy         1.3.1            py36h10fcdad_0  
libedit                   3.1                  heed3624_0  
libffi                    3.2.1                h4deb6c0_3  
libgcc                    7.2.0                h69d50b8_2  
libgcc-ng                 7.2.0                h7cc24e2_2  
libgfortran               3.0.0                         1  
libgfortran-ng            7.2.0                h9f7466a_2  
libmklml                  2018.0.3                      0  
libmxnet                  1.2.1           gpu_mkl_h3d71631_1  
libnetcdf                 4.4.1.1                       9    conda-forge
libopenblas               0.2.20               h9ac9557_7  
libopencv                 3.4.1                h62359dd_1  
libopus                   1.2.1                hb9ed12e_0  
libpng                    1.6.34                        0    conda-forge
libprotobuf               3.5.2                hd28b015_1    conda-forge
libsodium                 1.0.13               h31c71d8_2  
libssh2                   1.8.0                h8c220ad_2  
libstdcxx-ng              7.2.0                h7a57d05_2  
libtiff                   4.0.9                he6b73bb_1    conda-forge
libtool                   2.4.6                hd50d1a6_0  
libvpx                    1.7.0                h439df22_0  
libxcb                    1.12                 h84ff03f_3  
libxml2                   2.9.4                h6b072ca_5  
libxslt                   1.1.29               hcf9102b_5  
llvmlite                  0.20.0                   py36_0  
locket                    0.2.0            py36h787c0ad_1  
lxml                      4.1.0            py36h5b66e50_0  
lzo                       2.10                 h1bfc0ba_1  
Mako                      1.0.7                     <pip>
markdown                  2.6.9                    py36_0  
markupsafe                1.0              py36hd9260cd_1  
matplotlib                2.1.0            py36hba5de38_0  
mccabe                    0.6.1            py36h5ad9710_1  
mistune                   0.7.4            py36hbab8784_0  
mkl                       2018.0.0             hb491cac_4  
mkl-dnn                   0.14                 h6bb024c_0  
mkl-service               1.1.2            py36h17a0993_4  
mpc                       1.0.3                hf803216_4  
mpfr                      3.1.5                h12ff648_1  
mpmath                    0.19             py36h8cc018b_2  
msgpack-python            0.4.8            py36hec4c5d1_0  
multipledispatch          0.4.9            py36h41da3fb_0  
mxnet                     1.2.1                h8cc8929_0  
mxnet-gpu                 1.2.1                hf82a2c8_0  
navigator-updater         0.1.0            py36h14770f7_0  
nbconvert                 5.3.1            py36hb41ffb7_0  
nbformat                  4.4.0            py36h31c9010_0  
nccl                      1.3.4                 cuda8.0_1  
ncurses                   6.0                  h06874d7_1  
netcdf-fortran            4.4.4                         5    conda-forge
networkx                  2.0              py36h7e96fb8_0  
ninja                     1.8.2                h2d50403_1    conda-forge
nltk                      3.2.4            py36h1a0979f_0  
nose                      1.3.7            py36hcdf7029_2  
notebook                  5.0.0            py36h0b20546_2  
numba                     0.35.0             np113py36_10  
numexpr                   2.6.2            py36hdd3393f_1  
numpy                     1.14.3           py36h28100ab_1  
numpy-base                1.14.3           py36h0ea5e3f_1  
numpydoc                  0.7.0            py36h18f165f_0  
odo                       0.5.1            py36h90ed295_0  
olefile                   0.44             py36h79f9f78_0  
openblas                  0.2.20                        7    conda-forge
openpyxl                  2.4.8            py36h41dd2a8_1  
openssl                   1.0.2p               h470a237_0    conda-forge
packaging                 16.8             py36ha668100_1  
pandas                    0.20.3           py36h842e28d_2  
pandoc                    1.19.2.1             hea2e7c5_1  
pandocfilters             1.4.2            py36ha6701b7_1  
pango                     1.40.11              h8191d47_0  
partd                     0.3.8            py36h36fd896_0  
patchelf                  0.9                  hf79760b_2  
path.py                   10.3.1           py36he0c6f6d_0  
pathlib2                  2.3.0            py36h49efa8e_0  
patsy                     0.4.1            py36ha3be15e_0  
pcre                      8.41                 hc71a17e_0  
pep8                      1.7.0            py36h26ade29_0  
pexpect                   4.2.1            py36h3b9d41b_0  
pickleshare               0.7.4            py36h63277f8_0  
pillow                    4.2.1            py36h9119f52_0  
pip                       9.0.1            py36h8ec8b28_3  
pixman                    0.34.0               h83dc358_2  
pkginfo                   1.4.1            py36h215d178_1  
ply                       3.10             py36hed35086_0  
prompt_toolkit            1.0.15           py36h17d85b1_0  
protobuf                  3.5.1                    py36_3    conda-forge
psutil                    5.4.0            py36h84c53db_0  
ptyprocess                0.5.2            py36h69acd42_0  
py                        1.4.34           py36h0712aa3_1  
py-mxnet                  1.2.1            py36h6c82189_0  
pycodestyle               2.3.1            py36hf609f19_0  
pycosat                   0.6.3                    py36_0    conda-forge
pycparser                 2.18             py36hf9f622e_1  
pycrypto                  2.6.1            py36h6998063_1  
pycuda                    2017.1.1                  <pip>
pycurl                    7.43.0           py36h5e72054_3  
pyflakes                  1.6.0            py36h7bd6a15_0  
pygments                  2.2.0            py36h0d3125c_0  
pylint                    1.7.4            py36hb9d4533_0  
pyodbc                    4.0.17           py36h999153c_0  
pyopenssl                 17.2.0           py36h5cc804b_0  
pyparsing                 2.2.0            py36hee85983_1  
pyqt                      5.6.0            py36h0386399_5  
pysocks                   1.6.7            py36hd97a5b1_1  
pytables                  3.4.2                    py36_6    conda-forge
pytest                    3.2.1            py36h11ad3bb_1  
python                    3.6.3                hc9025b9_1  
python-dateutil           2.6.1            py36h88d3b88_1  
pytools                   2017.6                    <pip>
pytorch                   0.4.1           py36_py35_py27__9.0.176_7.1.2_2    pytorch
pytz                      2017.2           py36hc2ccc2a_1  
pywavelets                0.5.2            py36he602eb0_0  
pyyaml                    3.12             py36hafb9ca4_1  
pyzmq                     16.0.2           py36h3b0cf96_2  
qt                        5.6.2               h974d657_12  
qtawesome                 0.4.4            py36h609ed8c_0  
qtconsole                 4.3.1            py36h8f73b5b_0  
qtpy                      1.3.1            py36h3691cc8_0  
readline                  7.0                  hac23ff0_3  
requests                  2.18.4           py36he2e5f8d_1  
rope                      0.10.5           py36h1f8c17e_0  
ruamel_yaml               0.11.14          py36ha2fb22d_2  
scikit-cuda               0.5.1                     <pip>
scikit-image              0.13.0           py36had3c07a_1  
scikit-learn              0.19.1           py36h7aa7ec6_0  
scipy                     0.19.1           py36h9976243_3  
seaborn                   0.8.0            py36h197244f_0  
setuptools                36.5.0           py36he42e2e1_0  
simplegeneric             0.8.1            py36h2cb9092_0  
singledispatch            3.4.0.3          py36h7a266c3_0  
sip                       4.18.1           py36h51ed4ed_2  
six                       1.11.0           py36h372c433_1  
snowballstemmer           1.2.1            py36h6febd40_0  
sortedcollections         0.5.3            py36h3c761f9_0  
sortedcontainers          1.5.7            py36hdf89491_0  
sphinx                    1.6.3            py36he5f0bdb_0  
sphinxcontrib             1.0              py36h6d0f590_1  
sphinxcontrib-websupport  1.0.1            py36hb5cb234_1  
spyder                    3.2.4            py36hbe6152b_0  
sqlalchemy                1.1.13           py36hfb5efd7_0  
sqlite                    3.20.1               h6d8b0f3_1  
statsmodels               0.8.0            py36h8533d0b_0  
sympy                     1.1.1            py36hc6d1c1c_0  
tblib                     1.3.2            py36h34cf8b6_0  
tensorflow-gpu            1.4.1                         0  
tensorflow-gpu-base       1.4.1            py36h01caf0a_0  
tensorflow-tensorboard    0.1.5                    py36_0  
terminado                 0.6              py36ha25a19f_0  
testpath                  0.3.1            py36h8cadb63_0  
tk                        8.6.7                h5979e9b_1  
toolz                     0.8.2            py36h81f2dff_0  
torchvision               0.2.1                    py36_1    pytorch
tornado                   4.5.2            py36h1283b2a_0  
traitlets                 4.3.2            py36h674d592_0  
typing                    3.6.2            py36h7da032a_0  
udunits2                  2.2.25                        2    conda-forge
unicodecsv                0.14.1           py36ha668878_0  
unixodbc                  2.3.4                hc36303a_1  
urllib3                   1.22             py36hbe7ace6_0  
wcwidth                   0.1.7            py36hdf4376a_0  
webencodings              0.5.1            py36h800622e_1  
werkzeug                  0.12.2           py36hc703753_0  
wheel                     0.29.0           py36he7f4e38_1  
widgetsnbextension        3.0.2            py36hd01bb71_1  
wrapt                     1.10.11          py36h28b7045_0  
xlrd                      1.1.0            py36h1db9f0c_1  
xlsxwriter                1.0.2            py36h3de1aca_0  
xlwt                      1.3.0            py36h7b00a1f_0  
xz                        5.2.3                h2bcbf08_1  
yaml                      0.1.7                h96e3832_1  
zeromq                    4.2.2                hb0b69da_1  
zict                      0.1.3            py36h3a3bf81_0  
zlib                      1.2.11               h470a237_3    conda-forge

Когда я запускаю код, я получаю сообщение об ошибке:

(myenv) [sbasumallik@hpc1 CNN]$ python state_CNN.py 
Using TensorFlow backend.
Traceback (most recent call last):
  File "/home/sbasumallik/.conda/envs/myenv/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
    from tensorflow.python.pywrap_tensorflow_internal import *
  File "/home/sbasumallik/.conda/envs/myenv/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
    _pywrap_tensorflow_internal = swig_import_helper()
  File "/home/sbasumallik/.conda/envs/myenv/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
  File "/home/sbasumallik/.conda/envs/myenv/lib/python3.6/imp.py", line 243, in load_module
    return load_dynamic(name, filename, file)
  File "/home/sbasumallik/.conda/envs/myenv/lib/python3.6/imp.py", line 343, in load_dynamic
    return _load(spec)
ImportError: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /home/sbasumallik/.conda/envs/myenv/lib/python3.6/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "stateEstimator_CNN.py", line 12, in <module>
    import keras as k
  File "/home/sbasumallik/.conda/envs/myenv/lib/python3.6/site-packages/keras/__init__.py", line 3, in <module>
    from . import utils
  File "/home/sbasumallik/.conda/envs/myenv/lib/python3.6/site-packages/keras/utils/__init__.py", line 6, in <module>
    from . import conv_utils
  File "/home/sbasumallik/.conda/envs/myenv/lib/python3.6/site-packages/keras/utils/conv_utils.py", line 9, in <module>
    from .. import backend as K
  File "/home/sbasumallik/.conda/envs/myenv/lib/python3.6/site-packages/keras/backend/__init__.py", line 89, in <module>
    from .tensorflow_backend import *
  File "/home/sbasumallik/.conda/envs/myenv/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 5, in <module>
    import tensorflow as tf
  File "/home/sbasumallik/.conda/envs/myenv/lib/python3.6/site-packages/tensorflow/__init__.py", line 34, in <module>
    from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
  File "/home/sbasumallik/.conda/envs/myenv/lib/python3.6/site-packages/tensorflow/python/__init__.py", line 49, in <module>
    from tensorflow.python import pywrap_tensorflow
  File "/home/sbasumallik/.conda/envs/myenv/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow.py", line 74, in <module>
    raise ImportError(msg)
ImportError: Traceback (most recent call last):
  File "/home/sbasumallik/.conda/envs/myenv/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
    from tensorflow.python.pywrap_tensorflow_internal import *
  File "/home/sbasumallik/.conda/envs/myenv/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
    _pywrap_tensorflow_internal = swig_import_helper()
  File "/home/sbasumallik/.conda/envs/myenv/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
  File "/home/sbasumallik/.conda/envs/myenv/lib/python3.6/imp.py", line 243, in load_module
    return load_dynamic(name, filename, file)
  File "/home/sbasumallik/.conda/envs/myenv/lib/python3.6/imp.py", line 343, in load_dynamic
    return _load(spec)
ImportError: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /home/sbasumallik/.conda/envs/myenv/lib/python3.6/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)


Failed to load the native TensorFlow runtime.

See https://www.tensorflow.org/install/errors

for some common reasons and solutions.  Include the entire stack trace
above this error message when asking for help.

Я пробовал все разные версии Tensorflow, как рассказывали пользователи в соответствующих публикациях в Stackoverflow, но ни одна из них не работала.

...