Компиляция Tensorflow с ROCm приводит к ImportError с неопределенным символом - PullRequest
0 голосов
/ 13 октября 2018

Я пытаюсь скомпилировать Tensorflow из источника с поддержкой ROCm из следующего репозитория, предоставленного AMD: https://github.com/ROCmSoftwarePlatform/tensorflow-upstream

Моя система - CentOS 7.5, но без корневого доступа , так как этообщий ресурс.Я использую версию tenorflow r1.8, ROCm 1.9, Python 2.7.5, GCC 4.8.5 и bazel 0.15 (собранный из исходного кода).

Ранее у меня было много проблем, связанных с библиотеками, которые не былине загружены правильно, с базальным CROSSTOOL и с флагами оптимизации, включенными по умолчанию, которые приводят к сбою процесса сборки.

На этот раз я получаю следующую ошибку почти в конце изпроцесс компиляции:

ImportError: /homelocal/rfgillocal/.cache/bazel/_bazel_rfgillocal/ea4f3f9ea1629f214818602553d90693/execroot/org/tensorflow/bazel-aporflow/bazintoori.dllrunfiles / org_tensorflow / tensorflow / питон / _pywrap_tensorflow_internal.so: не определено символ: _ZN10tensorflow7functor13BinaryFunctorIN5Eigen9GpuDeviceENS0_13greater_equalIhEELi1ELb0EE5RightERKS3_NS2_9TensorMapINS2_6TensorIbLi1ELi1ElEELi16ENS2_11MakePointerEEENS9_INSA_IKhLi1ELi1ElEELi16ESC_EENS9_INS2_15TensorFixedSizeISE_NS2_5SizesIIEEELi1ElEELi16ESC_EEPb

1016 *

Честно говоря, понятия не имею, почему это происходит.У меня была похожая ошибка, без ROCm, ( Tensorflow "неопределенный символ" при загрузке пользовательского графического процессора op ).Эта ошибка была связана с ABI gcc.Однако тогда это было во время импорта, а теперь это происходит на этапе компиляции.

Я попробовал как новый, так и старый ABI с флагом --cxxopt = "- D_GLIBCXX_USE_CXX11_ABI" просто чтобы убедиться.Оба приводят к одной и той же проблеме.

Не думаю, что это вызвано самим кодом ROCm или Tensorlfow, а скорее с каким-то параметром компиляции, который мне не хватает.

Вот полный стекЯ получаю:

ERROR: /homelocal/rfgillocal/src/tensorflow-upstream/tensorflow/tools/api/generator/BUILD:27:1: Executing genrule //tensorflow/tools/api/generator:python_api_gen failed (Exit 1): bash failed: error executing command 
  (cd /homelocal/rfgillocal/.cache/bazel/_bazel_rfgillocal/ea4f3f9ea1629f214818602553d90693/execroot/org_tensorflow && \
  exec env - \
    LD_LIBRARY_PATH=/homelocal/rfgillocal/rocm/lib:/homelocal/rfgillocal/rocm/miopen/lib:/homelocal/rfgillocal/rocm/rocblas/lib:/homelocal/rfgillocal/rocm/rocfft/lib:/homelocal/rfgillocal/rocm/hiprand/lib:/homelocal/rfgillocal/local/lib:/usr/lib64:/usr/lib \
    PATH=/homelocal/rfgillocal/local/bin:/homelocal/rfgillocal/python_env/bin:/opt/rocm/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin \
    PYTHON_BIN_PATH=/homelocal/rfgillocal/python_env/bin/python \
    PYTHON_LIB_PATH=/homelocal/rfgillocal/python_env/lib/python2.7/site-packages \
    TF_DOWNLOAD_CLANG=0 \
    TF_NEED_CUDA=0 \
    TF_NEED_OPENCL_SYCL=0 \
    TF_NEED_ROCM=1 \
  /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; bazel-out/host/bin/tensorflow/tools/api/generator/create_python_api bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/app/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/bitwise/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/compat/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/contrib/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/contrib/stat_summarizer/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/data/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/distributions/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/distributions/bijectors/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/errors/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/estimator/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/estimator/export/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/estimator/inputs/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/feature_column/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/gfile/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/graph_util/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/image/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/initializers/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/activations/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/applications/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/applications/densenet/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/applications/inception_resnet_v2/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/applications/inception_v3/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/applications/mobilenet/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/applications/nasnet/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/applications/resnet50/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/applications/vgg16/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/applications/vgg19/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/applications/xception/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/backend/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/callbacks/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/constraints/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/datasets/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/datasets/boston_housing/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/datasets/cifar10/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/datasets/cifar100/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/datasets/fashion_mnist/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/datasets/imdb/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/datasets/mnist/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/datasets/reuters/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/estimator/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/initializers/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/layers/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/losses/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/metrics/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/models/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/optimizers/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/preprocessing/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/preprocessing/image/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/preprocessing/sequence/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/preprocessing/text/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/regularizers/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/utils/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/wrappers/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/keras/wrappers/scikit_learn/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/layers/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/linalg/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/logging/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/losses/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/manip/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/math/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/metrics/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/nn/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/nn/rnn_cell/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/profiler/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/python_io/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/resource_loader/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/saved_model/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/saved_model/builder/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/saved_model/constants/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/saved_model/loader/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/saved_model/main_op/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/saved_model/signature_constants/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/saved_model/signature_def_utils/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/saved_model/tag_constants/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/saved_model/utils/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/sets/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/spectral/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/summary/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/sysconfig/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/test/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/train/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/train/queue_runner/__init__.py bazel-out/k8-opt/genfiles/tensorflow/tools/api/generator/api/user_ops/__init__.py')
Traceback (most recent call last):
  File "/homelocal/rfgillocal/.cache/bazel/_bazel_rfgillocal/ea4f3f9ea1629f214818602553d90693/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/tools/api/generator/create_python_api.runfiles/org_tensorflow/tensorflow/tools/api/generator/create_python_api.py", line 26, in <module>
    from tensorflow.python.util import tf_decorator
  File "/homelocal/rfgillocal/.cache/bazel/_bazel_rfgillocal/ea4f3f9ea1629f214818602553d90693/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/tools/api/generator/create_python_api.runfiles/org_tensorflow/tensorflow/python/__init__.py", line 49, in <module>
    from tensorflow.python import pywrap_tensorflow
  File "/homelocal/rfgillocal/.cache/bazel/_bazel_rfgillocal/ea4f3f9ea1629f214818602553d90693/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/tools/api/generator/create_python_api.runfiles/org_tensorflow/tensorflow/python/pywrap_tensorflow.py", line 74, in <module>
    raise ImportError(msg)
ImportError: Traceback (most recent call last):    
  File "/homelocal/rfgillocal/.cache/bazel/_bazel_rfgillocal/ea4f3f9ea1629f214818602553d90693/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/tools/api/generator/create_python_api.runfiles/org_tensorflow/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
    from tensorflow.python.pywrap_tensorflow_internal import *
  File "/homelocal/rfgillocal/.cache/bazel/_bazel_rfgillocal/ea4f3f9ea1629f214818602553d90693/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/tools/api/generator/create_python_api.runfiles/org_tensorflow/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
    _pywrap_tensorflow_internal = swig_import_helper()
  File "/homelocal/rfgillocal/.cache/bazel/_bazel_rfgillocal/ea4f3f9ea1629f214818602553d90693/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/tools/api/generator/create_python_api.runfiles/org_tensorflow/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
ImportError: /homelocal/rfgillocal/.cache/bazel/_bazel_rfgillocal/ea4f3f9ea1629f214818602553d90693/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/tools/api/generator/create_python_api.runfiles/org_tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so: undefined symbol: _ZN10tensorflow7functor13BinaryFunctorIN5Eigen9GpuDeviceENS0_13greater_equalIhEELi1ELb0EE5RightERKS3_NS2_9TensorMapINS2_6TensorIbLi1ELi1ElEELi16ENS2_11MakePointerEEENS9_INSA_IKhLi1ELi1ElEELi16ESC_EENS9_INS2_15TensorFixedSizeISE_NS2_5SizesIIEEELi1ElEELi16ESC_EEPb


Failed to load the native TensorFlow runtime.

See https://www.tensorflow.org/install/install_sources#common_installation_problems

for some common reasons and solutions.  Include the entire stack trace
above this error message when asking for help.
Target //tensorflow/tools/pip_package:build_pip_package failed to build

Любая помощь будет принята с благодарностью.

Заранее спасибо!

...