Попытка запустить пример распределенного тензорного потока на ЦП из:
https://github.com/tmulc18/Distributed-TensorFlow-Guide/blob/master/Distributed-Setup/dist_setup.py
Команды для запуска примера можно найти по адресу:
https://github.com/tmulc18/Distributed-TensorFlow-Guide/blob/master/Distributed-Setup/run.sh
Он отлично работает, когда я запускаю его на одной платформе (ПК-ПК или ноутбук-ноутбук или RP (Raspberry pi3) -RP) или на нескольких платформах с одинаковой архитектурой (ПК-ноутбук, оба x86 или RP-RP, обаarm64).Но сочетание arm64 и x86 со стороны arm64 дает сбой со следующей ошибкой:
2019-06-15 01:20:35.179745: F tensorflow/core/common_runtime/renamed_device.cc:27] Check failed: DeviceNameUtils::ParseFullName(new_base, &parsed_name)
Команда для ПК: Обратите внимание, что в вашем коде IP-адреса должны быть установлены соответствующим образом.
python dist_setup.py --job_name "worker" --task_index 0
Вывод:
2019-06-14 18:20:35.040413: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-06-14 18:20:35.070714: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3593265000 Hz
2019-06-14 18:20:35.071281: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x4c9ce60 executing computations on platform Host. Devices:
2019-06-14 18:20:35.071303: I tensorflow/compiler/xla/service/service.cc:158] StreamExecutor device (0): <undefined>, <undefined>
2019-06-14 18:20:35.072829: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:252] Initialize GrpcChannelCache for job ps -> {0 -> 10.1.1.2:2222}
2019-06-14 18:20:35.072861: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:252] Initialize GrpcChannelCache for job worker -> {0 -> localhost:2223}
2019-06-14 18:20:35.074703: I tensorflow/core/distributed_runtime/rpc/grpc_server_lib.cc:391] Started server with target: grpc://localhost:2223
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
2019-06-14 18:20:35.178858: I tensorflow/core/distributed_runtime/master_session.cc:1192] Start master session 3634afcffbd6cc2d with config:
2019-06-14 18:20:45.214939: W tensorflow/core/distributed_runtime/master_session.cc:1363] Timeout for closing worker session
2019-06-14 18:20:55.218267: I tensorflow/core/distributed_runtime/master.cc:267] CreateSession still waiting for response from worker: /job:ps/replica:0/task:0
2019-06-14 18:21:05.218392: I tensorflow/core/distributed_runtime/master.cc:267] CreateSession still waiting for response from worker: /job:ps/replica:0/task:0
2019-06-14 18:21:15.218519: I tensorflow/core/distributed_runtime/master.cc:267] CreateSession still waiting for response from worker: /job:ps/replica:0/task:0
Команда для RP:
python dist_setup.py --job_name "ps" --task_index 0
Вывод:
/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_util.py:33: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
from tensorflow.python.framework import fast_tensor_util
2019-06-15 01:19:54.226102: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:215] Initialize GrpcChannelCache for job ps -> {0 -> localhost:2222}
2019-06-15 01:19:54.226278: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:215] Initialize GrpcChannelCache for job worker -> {0 -> 10.1.1.1:2223}
2019-06-15 01:19:54.227740: I tensorflow/core/distributed_runtime/rpc/grpc_server_lib.cc:324] Started server with target: grpc://localhost:2222
2019-06-15 01:20:35.179745: F tensorflow/core/common_runtime/renamed_device.cc:27] Check failed: DeviceNameUtils::ParseFullName(new_base, &parsed_name)
Aborted
Любая идея, почему я получаю эту ошибкуиз ниоткуда.Похоже, ошибка выдается сразу после подключения серверов.