Компиляция C ++ правила '// tensorflow / core / kernels: sparse_reduce_op' не удалась (выход 1) - PullRequest
2 голосов
/ 08 мая 2020

Я пытаюсь построить тензорный поток из исходного кода, он продолжает работать несколько часов, затем сборка завершается ошибкой, и появляется следующая ошибка: .................... .................................................. .................................................. .................................................. ..........................

ERROR: /home/emadboctor/tensorflow/tensorflow/core/kernels/BUILD:5398:1: C++ compilation of rule '//tensorflow/core/kernels:sparse_reduce_op' failed (Exit 1)
In file included from external/eigen_archive/unsupported/Eigen/CXX11/Tensor:104:0,
                 from ./third_party/eigen3/unsupported/Eigen/CXX11/Tensor:1,
                 from ./tensorflow/core/framework/numeric_types.h:20,
                 from ./tensorflow/core/framework/allocator.h:26,
                 from ./tensorflow/core/framework/op_kernel.h:24,
                 from tensorflow/core/kernels/sparse_reduce_op.cc:20:
external/eigen_archive/unsupported/Eigen/CXX11/src/Tensor/TensorReduction.h: In static member function 'static void std::_Function_handler<void(_ArgTypes ...), _Functor>::_M_invoke(const std::_Any_data&, _ArgTypes&& ...) [with _Functor = Eigen::internal::TensorExecutor<Expression, Eigen::ThreadPoolDevice, Vectorizable, Tiling>::run(const Expression&, const Eigen::ThreadPoolDevice&) [with Expression = const Eigen::TensorAssignOp<Eigen::TensorMap<Eigen::TensorFixedSize<std::complex<float>, Eigen::Sizes<>, 1, long int>, 16, Eigen::MakePointer>, const Eigen::TensorReductionOp<Eigen::internal::SumReducer<std::complex<float> >, const Eigen::DimensionList<long int, 1ul>, const Eigen::TensorMap<Eigen::Tensor<std::complex<float>, 1, 1, long int>, 0, Eigen::MakePointer>, Eigen::MakePointer> >; bool Vectorizable = true; Eigen::internal::TiledEvaluation Tiling = (Eigen::internal::TiledEvaluation)0u]::<lambda(Eigen::internal::TensorExecutor<const Eigen::TensorAssignOp<Eigen::TensorMap<Eigen::TensorFixedSize<std::complex<float>, Eigen::Sizes<>, 1, long int>, 16, Eigen::MakePointer>, const Eigen::TensorReductionOp<Eigen::internal::SumReducer<std::complex<float> >, const Eigen::DimensionList<long int, 1ul>, const Eigen::TensorMap<Eigen::Tensor<std::complex<float>, 1, 1, long int>, 0, Eigen::MakePointer>, Eigen::MakePointer> >, Eigen::ThreadPoolDevice, true, (Eigen::internal::TiledEvaluation)0u>::StorageIndex, Eigen::internal::TensorExecutor<const Eigen::TensorAssignOp<Eigen::TensorMap<Eigen::TensorFixedSize<std::complex<float>, Eigen::Sizes<>, 1, long int>, 16, Eigen::MakePointer>, const Eigen::TensorReductionOp<Eigen::internal::SumReducer<std::complex<float> >, const Eigen::DimensionList<long int, 1ul>, const Eigen::TensorMap<Eigen::Tensor<std::complex<float>, 1, 1, long int>, 0, Eigen::MakePointer>, Eigen::MakePointer> >, Eigen::ThreadPoolDevice, true, (Eigen::internal::TiledEvaluation)0u>::StorageIndex)>; _ArgTypes = {long int, long int}]':
external/eigen_archive/unsupported/Eigen/CXX11/src/Tensor/TensorReduction.h:806:9: internal compiler error: in emit_move_insn, at expr.c:3547
         values[i] = internal::InnerMostDimReducer<Self, Op>::reduce(*this, firstIndex + i * num_values_to_reduce,
         ^~~~~~
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-6/README.Bugs> for instructions.
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: /home/emadboctor/tensorflow/tensorflow/tools/pip_package/BUILD:65:1 C++ compilation of rule '//tensorflow/core/kernels:sparse_reduce_op' failed (Exit 1)
INFO: Elapsed time: 8153.055s, Critical Path: 926.03s
INFO: 8566 processes: 8566 local.
FAILED: Build did NOT complete successfully

Информация о системе (экземпляр GCP):

Distributor ID: Debian
Description:    Debian GNU/Linux 9.12 (stretch)
Release:        9.12
Codename:       stretch
gcc version: gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516
g++ version: g++ (Debian 6.3.0-18+deb9u1) 6.3.0 20170516
tensorflow version: 2.1
bazel version: 3.1.0

Действия, которые привели к проблеме:

apt update && apt install -y \
    build-essential \
    libc-ares-dev \
    libjpeg-dev \
    openjdk-8-jdk \
    gcc \
    g++ \
    python3-pip \

pip3 install six numpy wheel setuptools mock && \
    pip3 install keras_applications --no-deps && \
    pip3 install keras_preprocessing --no-deps

sudo apt-get update
sudo apt-get -y upgrade
wget https://dl.google.com/go/go1.13.3.linux-amd64.tar.gz
sudo tar -xvf go1.13.3.linux-amd64.tar.gz
sudo mv go /usr/local
export GOROOT=/usr/local/go
export GOPATH=$HOME/Projects/Proj1
export PATH=$GOPATH/bin:$GOROOT/bin:$PATH
go get github.com/bazelbuild/bazelisk
mkdir -p ~/bin
ln -s $(go env GOPATH)/bin/bazelisk ~/bin/bazel
export PATH=$HOME/bin:$PATH
git clone https://github.com/tensorflow/tensorflow
cd tensorflow
./configure

Конфигурация:

emadboctor@reg-build-vm:~/tensorflow$ ./configure 
You have bazel 3.0.0 installed.
Please specify the location of python. [Default is /usr/bin/python3]: /usr/bin/python3


Found possible Python library paths:
  /usr/local/lib/python3.5/dist-packages
  /usr/lib/python3/dist-packages
Please input the desired Python library path to use.  Default is [/usr/local/lib/python3.5/dist-packages]
/usr/local/lib/python3.5/dist-packages
Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: n
No OpenCL SYCL support will be enabled for TensorFlow.

Do you wish to build TensorFlow with ROCm support? [y/N]: n
No ROCm support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]: n
No CUDA support will be enabled for TensorFlow.

Do you wish to download a fresh release of clang? (Experimental) [y/N]: y
Clang will be downloaded and used to compile tensorflow.

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]: -config=mkl


Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: n
Not configuring the WORKSPACE for Android builds.

Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
    --config=mkl            # Build with MKL support.
    --config=monolithic     # Config for mostly static monolithic build.
    --config=ngraph         # Build with Intel nGraph support.
    --config=numa           # Build with NUMA support.
    --config=dynamic_kernels    # (Experimental) Build kernels into separate shared objects.
    --config=v2             # Build TensorFlow 2.x instead of 1.x.
Preconfigured Bazel build configs to DISABLE default on features:
    --config=noaws          # Disable AWS S3 filesystem support.
    --config=nogcp          # Disable GCP support.
    --config=nohdfs         # Disable HDFS support.
    --config=nonccl         # Disable NVIDIA NCCL support.
Configuration finished
emadboctor@reg-build-vm:~/tensorflow$

Затем

bazel build -c opt \
            --define=grpc_no_ares=true  \
            --linkopt="-lrt" \
            --linkopt="-lm" \
            --host_linkopt="-lrt" \
            --host_linkopt="-lm" \
            --action_env="LD_LIBRARY_PATH=${LD_LIBRARY_PATH}" \
            --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mfpmath=both \
            --copt=-w \
            --jobs=26 \
            //tensorflow/tools/pip_package:build_pip_package
...