Цапля не смогла установить план упаковки для топологии «WordCountTopology»

Question

Когда я отправил WordCountTopology в кластер Heron, развернутый с помощью планировщика Aurora и Zookeeper, эта ошибка произошла следующим образом:

yitian@heron01:~$ heron submit aurora/yitian/devel --config-path ~/.heron/conf ~/.heron/examples/heron-api-examples.jar com.twitter.heron.examples.api.WordCountTopology WordCountTopology --deploy-deactivated
[2018-06-04 00:55:54 +0000] [INFO]: Using cluster definition in /home/yitian/.heron/conf/aurora
[2018-06-04 00:55:54 +0000] [INFO]: Launching topology: 'WordCountTopology'
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/yitian/.heron/lib/uploader/heron-dlog-uploader.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/yitian/.heron/lib/statemgr/heron-zookeeper-statemgr.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
[2018-06-04 00:55:55 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Starting Curator client connecting to: heron01:2181  
[2018-06-04 00:55:55 -0700] [INFO] org.apache.curator.framework.imps.CuratorFrameworkImpl: Starting  
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT  
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:host.name=heron01  
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:java.version=1.8.0_151  
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation  
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:java.home=/usr/java/jdk1.8.0_151/jre  
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:java.class.path=:/home/yitian/.heron/lib/scheduler/heron-scheduler.jar:/home/yitian/.heron/lib/scheduler/heron-binpacking-packing.jar:/home/yitian/.heron/lib/scheduler/heron-yarn-scheduler.jar:/home/yitian/.heron/lib/scheduler/heron-slurm-scheduler.jar:/home/yitian/.heron/lib/scheduler/heron-mesos-scheduler.jar:/home/yitian/.heron/lib/scheduler/heron-marathon-scheduler.jar:/home/yitian/.heron/lib/scheduler/heron-kubernetes-scheduler.jar:/home/yitian/.heron/lib/scheduler/heron-roundrobin-packing.jar:/home/yitian/.heron/lib/scheduler/heron-local-scheduler.jar:/home/yitian/.heron/lib/scheduler/heron-aurora-scheduler.jar:/home/yitian/.heron/lib/uploader/heron-localfs-uploader.jar:/home/yitian/.heron/lib/uploader/heron-hdfs-uploader.jar:/home/yitian/.heron/lib/uploader/heron-null-uploader.jar:/home/yitian/.heron/lib/uploader/heron-scp-uploader.jar:/home/yitian/.heron/lib/uploader/heron-s3-uploader.jar:/home/yitian/.heron/lib/uploader/heron-gcs-uploader.jar:/home/yitian/.heron/lib/uploader/heron-dlog-uploader.jar:/home/yitian/.heron/lib/statemgr/heron-localfs-statemgr.jar:/home/yitian/.heron/lib/statemgr/heron-zookeeper-statemgr.jar:/home/yitian/.heron/lib/packing/heron-binpacking-packing.jar:/home/yitian/.heron/lib/packing/heron-roundrobin-packing.jar  
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib  
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp  
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=<NA>  
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux  
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:os.arch=amd64  
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:os.version=4.10.0-28-generic  
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:user.name=yitian  
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:user.home=/home/yitian  
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/home/yitian  
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=heron01:2181 sessionTimeout=30000 watcher=org.apache.curator.ConnectionState@2a17b7b6  
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ClientCnxn: Opening socket connection to server heron01/218.195.241.174:2181. Will not attempt to authenticate using SASL (unknown error)  
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ClientCnxn: Socket connection established to heron01/218.195.241.174:2181, initiating session  
[2018-06-04 00:55:55 -0700] [INFO] org.apache.zookeeper.ClientCnxn: Session establishment complete on server heron01/218.195.241.174:2181, sessionid = 0x163c9cb80580003, negotiated timeout = 30000  
[2018-06-04 00:55:55 -0700] [INFO] org.apache.curator.framework.state.ConnectionStateManager: State change: CONNECTED  
[2018-06-04 00:55:55 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Directory tree initialized.  
[2018-06-04 00:55:55 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Checking existence of path: /heron/topologies/WordCountTopology  
[2018-06-04 00:55:58 -0700] [INFO] com.twitter.heron.uploader.hdfs.HdfsUploader: Target topology file already exists at '/heron/topologies/aurora/WordCountTopology-yitian-tag-0--1244338252177277003.tar.gz'. Overwriting it now  
[2018-06-04 00:55:58 -0700] [INFO] com.twitter.heron.uploader.hdfs.HdfsUploader: Uploading topology package at '/tmp/tmpXlYukj/topology.tar.gz' to target HDFS at '/heron/topologies/aurora/WordCountTopology-yitian-tag-0--1244338252177277003.tar.gz'  
[2018-06-04 00:56:02 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Created node for path: /heron/topologies/WordCountTopology  
[2018-06-04 00:56:02 -0700] [WARNING] com.twitter.heron.spi.statemgr.SchedulerStateManagerAdaptor: Exception processing future: java.lang.RuntimeException: Could not createNode:  
[2018-06-04 00:56:02 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Deleted node for path: /heron/topologies/WordCountTopology  
18/06/04 00:56:05 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 0 minutes, Emptier interval = 0 minutes.
Deleted /heron/topologies/aurora/WordCountTopology-yitian-tag-0--1244338252177277003.tar.gz
[2018-06-04 00:56:05 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Closing the CuratorClient to: heron01:2181  
[2018-06-04 00:56:05 -0700] [INFO] org.apache.zookeeper.ZooKeeper: Session: 0x163c9cb80580003 closed  
[2018-06-04 00:56:05 -0700] [INFO] org.apache.zookeeper.ClientCnxn: EventThread shut down  
[2018-06-04 00:56:05 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Closing the tunnel processes  
[2018-06-04 00:56:05 +0000] [ERROR]: Failed to set packing plan for topology 'WordCountTopology'
[2018-06-04 00:56:05 +0000] [ERROR]: Failed to launch topology 'WordCountTopology' 

Что с этим не так?Спасибо за вашу помощь.

Answer 1

Из журналов:

[2018-06-04 00:56:02 -0700] [INFO] com.twitter.heron.statemgr.zookeeper.curator.CuratorStateManager: Создан узел дляпуть: / heron / topologies / WordCountTopology
[2018-06-04 00:56:02 -0700] [ПРЕДУПРЕЖДЕНИЕ] com.twitter.heron.spi.statemgr.SchedulerStateManagerAdaptor: будущее обработки исключений: java.lang.RuntimeException:Не удалось создать узел:

Heron может подключиться к ZK и успешно проверить узел, поэтому кластер ZK, кажется, работает и доступен для чтения.Однако кажется, что по какой-то причине куратору не удалось создать узел в ZK: / heron / topologies / WordCountTopology

Топология Heron хранит свои текущие данные в ZK, поэтому она не может запуститься, если не удалось создать ключевые узлы.Вам необходимо выяснить причину сбоя ZK (разрешение?) И устранить ее.