Zookeeper for Apache-Kafka: проблемы с портом 2181 в Ubuntu 18.04.01 Server - PullRequest
Новая
       10

Zookeeper for Apache-Kafka: проблемы с портом 2181 в Ubuntu 18.04.01 Server

0 голосов
/ 04 сентября 2018

Я пытаюсь выяснить, как успешно дойти до конца этого урока, для установки и первого запуска Apache-Kafka: Как установить Apache Kafka в Ubuntu 18.04 LTS

После убийства ранее запущенного экземпляра zookeeper я снова попытался !, запустить сервер zookeeper снова, но когда процесс достигает фазы привязки порта, он останавливается навсегда ..: KafkaActivationFailed 

marco@PC:/usr/share/zookeeper$ sudo lsof -i :2181
COMMAND  PID      USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
java    4545 zookeeper   32u  IPv6  43042      0t0  TCP *:2181 (LISTEN)
marco@PC:/usr/share/zookeeper$ sudo kill 4545


marco@PC:/usr/local/kafka$ bin/zookeeper-server-start.sh   
config/zookeeper.properties
[2018-09-04 12:18:14,608] INFO Reading configuration from:  
config/zookeeper.properties  
(org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2018-09-04 12:18:14,609] INFO autopurge.snapRetainCount set to 3  
(org.apache.zookeeper.server.DatadirCleanupManager)
[2018-09-04 12:18:14,609] INFO autopurge.purgeInterval set to 0  
(org.apache.zookeeper.server.DatadirCleanupManager)
[2018-09-04 12:18:14,609] INFO Purge task is not scheduled. 
(org.apache.zookeeper.server.DatadirCleanupManager)
[2018-09-04 12:18:14,610] WARN Either no config or no quorum defined in  
config, running  in standalone mode    
(org.apache.zookeeper.server.quorum.QuorumPeerMain)
[2018-09-04 12:18:14,620] INFO Reading configuration from:  
config/zookeeper.properties   
(org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2018-09-04 12:18:14,620] INFO Starting server       
(org.apache.zookeeper.server.ZooKeeperServerMain)
[2018-09-04 12:18:14,624] INFO Server  
environment:zookeeper.version=3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03 
built on 06/29/2018 00:39 GMT (org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,624] INFO Server environment:host.name=PC.station  
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:java.version=1.8.0_181 
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:java.vendor=Oracle 
Corporation (org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:java.home=/usr/lib/jvm
/java-8-oracle/jre (org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:java.class.path=/usr/local
/kafka/bin/../libs/activation-1.1.1.jar:/usr/local/kafka/bin/..
/libs/aopalliance-repackaged-2.5.0-b42.jar:/usr/local/kafka/bin/..
/libs/argparse4j-0.7.0.jar:/usr/local/kafka/bin/../libs/audience-annotations-
0.5.0.jar:/usr/local/kafka/bin/../libs/commons-lang3-3.5.jar:/usr/local/kafka
/bin/../libs/connect-api-2.0.0.jar:/usr/local/kafka/bin/../libs/connect-
basic-auth-extension-2.0.0.jar:/usr/local/kafka/bin/../libs/connect-
file-2.0.0.jar:/usr/local/kafka/bin/../libs/connect-json-2.0.0.jar:/usr/local
/kafka/bin/../libs/connect-runtime-2.0.0.jar:/usr/local/kafka/bin/..
/libs/connect-transforms-2.0.0.jar:/usr/local/kafka/bin/../libs/guava-
20.0.jar:/usr/local/kafka/bin/../libs/hk2-api-2.5.0-b42.jar:/usr/local/kafka
/bin/../libs/hk2-locator-2.5.0-b42.jar:/usr/local/kafka/bin/../libs/hk2-
utils-2.5.0-b42.jar:/usr/local/kafka/bin/../libs/jackson-annotations-
2.9.6.jar:/usr/local/kafka/bin/../libs/jackson-core-2.9.6.jar:/usr/local
/kafka/bin/../libs/jackson-databind-2.9.6.jar:/usr/local/kafka/bin/..
/libs/jackson-jaxrs-base-2.9.6.jar:/usr/local/kafka/bin/../libs/jackson-
jaxrs-json-provider-2.9.6.jar:/usr/local/kafka/bin/../libs/jackson-module-
jaxb-annotations-2.9.6.jar:/usr/local/kafka/bin/../libs/javassist-3.22.0-
CR2.jar:/usr/local/kafka/bin/../libs/javax.annotation-api-1.2.jar:/usr/local
/kafka/bin/../libs/javax.inject-1.jar:/usr/local/kafka/bin/.. 
/libs/javax.inject-2.5.0-b42.jar:/usr/local/kafka/bin/../libs/javax.servlet-
api-3.1.0.jar:/usr/local/kafka/bin/../libs/javax.ws.rs-api-2.1.jar:/usr/local
/kafka/bin/../libs/jaxb-api-2.3.0.jar:/usr/local/kafka/bin/../libs/jersey-
client-2.27.jar:/usr/local/kafka/bin/../libs/jersey-common-2.27.jar:
/usr/local/kafka/bin/../libs/jersey-container-servlet-2.27.jar:/usr/local
/kafka/bin/../libs/jersey-container-servlet-core-2.27.jar:/usr/local/kafka
/bin/../libs/jersey-hk2-2.27.jar:/usr/local/kafka/bin/../libs/jersey-media-
jaxb-2.27.jar:/usr/local/kafka/bin/../libs/jersey-server-2.27.jar:/usr/local
/kafka/bin/../libs/jetty-client-9.4.11.v20180605.jar:/usr/local/kafka/bin/..
/libs/jetty-continuation-9.4.11.v20180605.jar:/usr/local/kafka/bin/..
/libs/jetty-http-9.4.11.v20180605.jar:/usr/local/kafka/bin/../libs/jetty-
io-9.4.11.v20180605.jar:/usr/local/kafka/bin/../libs/jetty-security-
9.4.11.v20180605.jar:/usr/local/kafka/bin/../libs/jetty-server-
9.4.11.v20180605.jar:/usr/local/kafka/bin/../libs/jetty-servlet-
9.4.11.v20180605.jar:/usr/local/kafka/bin/../libs/jetty-servlets-
9.4.11.v20180605.jar:/usr/local/kafka/bin/../libs/jetty-
util-9.4.11.v20180605.jar:/usr/local/kafka/bin/../libs/jopt-simple-5.0.4.jar:
/usr/local/kafka/bin/../libs/kafka_2.11-2.0.0.jar:/usr/local/kafka/bin/..
/libs/kafka_2.11-2.0.0-sources.jar:/usr/local/kafka/bin/../libs/kafka-
clients-2.0.0.jar:/usr/local/kafka/bin/../libs/kafka-log4j-appender-
2.0.0.jar:/usr/local/kafka/bin/../libs/kafka-streams-2.0.0.jar:/usr/local
/kafka/bin/../libs/kafka-streams-examples-2.0.0.jar:/usr/local/kafka/bin/..
/libs/kafka-streams-scala_2.11-2.0.0.jar:/usr/local/kafka/bin/../libs/kafka-
streams-test-utils-2.0.0.jar:/usr/local/kafka/bin/../libs/kafka-tools-
2.0.0.jar:/usr/local/kafka/bin/../libs/log4j-1.2.17.jar:/usr/local/kafka
/bin/../libs/lz4-java-1.4.1.jar:/usr/local/kafka/bin/../libs/maven-artifact-
3.5.3.jar:/usr/local/kafka/bin/../libs/metrics-core-2.2.0.jar:/usr/local
/kafka/bin/../libs/osgi-resource-locator-1.0.1.jar:/usr/local/kafka/bin/..
/libs/plexus-utils-3.1.0.jar:/usr/local/kafka/bin/../libs/reflections-
0.9.11.jar:/usr/local/kafka/bin/../libs/rocksdbjni-5.7.3.jar:/usr/local/kafka
/bin/../libs/scala-library-2.11.12.jar:/usr/local/kafka/bin/../libs/scala-
logging_2.11-3.9.0.jar:/usr/local/kafka/bin/../libs/scala-reflect-
2.11.12.jar:/usr/local/kafka/bin/../libs/slf4j-api-1.7.25.jar:/usr/local
/kafka/bin/../libs/slf4j-log4j12-1.7.25.jar:/usr/local/kafka/bin/..
/libs/snappy-java-1.1.7.1.jar:/usr/local/kafka/bin/../libs/validation-
api-1.1.0.Final.jar:/usr/local/kafka/bin/../libs/zkclient-0.10.jar:/usr/local
/kafka/bin/../libs/zookeeper-3.4.13.jar 
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:java.library.path=/usr
/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib   
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:java.io.tmpdir=/tmp
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:java.compiler=<NA>  
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:os.name=Linux  
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:os.arch=amd64 
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:os.version=4.15.0-33-
generic (org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:user.name=marco 
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:user.home=/home/marco 
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:user.dir=/usr/local/kafka 
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,630] INFO tickTime set to 3000 
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,630] INFO minSessionTimeout set to -1 
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,630] INFO maxSessionTimeout set to -1 
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,634] INFO Using  
org.apache.zookeeper.server.NIOServerCnxn Factory as server connection  
factory (org.apache.zookeeper.server.ServerCnxnFactory)
[2018-09-04 12:18:14,636] INFO binding to port 0.0.0.0/0.0.0.0:2181   
(org.apache.zookeeper.server.NIOServerCnxnFactory)

Проблема в том, что перезагрузка zookeeper активна: 

marco@PC:~$ netstat -ant | grep :2181
tcp6       0      0 :::2181                 :::*                      
LISTEN
marco@PC:~$ systemctl status zookeeper
● zookeeper.service - LSB: centralized coordination service
   Loaded: loaded (/etc/init.d/zookeeper; generated)
   Active: active (running) since Tue 2018-09-04 15:51:24 CEST; 2min 19s
   ago
     Docs: man:systemd-sysv-generator(8)
  Process: 1919 ExecStart=/etc/init.d/zookeeper start (code=exited, 
status=0/SUCCESS)
    Tasks: 25 (limit: 4915)
   CGroup: /system.slice/zookeeper.service
           └─1997 /usr/bin/java -cp /etc/zookeeper/conf:/usr/share
/java/jline.jar:/usr/share/java/log4j-1.2.jar:/usr/share   
/java/xercesImpl.jar:/usr/share/java/xmlParserAPIs.jar:/usr/share
/java/netty.jar:/usr/share/java/slf4j-api.jar:/usr

Sep 04 15:51:24 PC systemd[1]: Starting LSB: centralized coordination 
service...
Sep 04 15:51:24 PC systemd[1]: Started LSB: centralized coordination 
service.

Но при попытке запустить кафку ...: 

marco@PC:~$ cd /usr/local/kafka
marco@PC:/usr/local/kafka$ bin/zookeeper-server-start.sh   
config/zookeeper.properties
[2018-09-04 15:54:29,981] INFO Reading configuration from:  
config/zookeeper.properties  
(org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2018-09-04 15:54:29,982] INFO autopurge.snapRetainCount set to 3  
(org.apache.zookeeper.server.DatadirCleanupManager)
[2018-09-04 15:54:29,983] INFO autopurge.purgeInterval set to 0 
(org.apache.zookeeper.server.DatadirCleanupManager)
...
[2018-09-04 15:54:30,012] INFO binding to port 0.0.0.0/0.0.0.0:2181  
(org.apache.zookeeper.server.NIOServerCnxnFactory)
[2018-09-04 15:54:30,013] ERROR Unexpected exception, exiting abnormally 
(org.apache.zookeeper.server.ZooKeeperServerMain)
java.net.BindException: Address already in use
        at sun.nio.ch.Net.bind0(Native Method)
        at sun.nio.ch.Net.bind(Net.java:433)
        at sun.nio.ch.Net.bind(Net.java:425)
        at 

sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
    at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
    at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)

Ответы [ 2 ]

0 голосов
/ 05 сентября 2018

Может быть, я "решил" проблему:

Я оставил открытым окно «в режиме ожидания» ( Первое окно ), я открыл новое окно и запустил сервер kafka: Шаг 2.b из ApachaKafka-QuickStart : / usr / local / kafka / bin / kafka-server-start.sh config / server.properties ( Второе окно ). Затем я открыл другое окно, чтобы создать тему и отправить несколько сообщений: 

marco@PC:/usr/local/kafka$ bin/kafka-topics.sh --create --zookeeper localhost:2181   
--replication-factor 1 --partitions 1 --topic test
Created topic "test".
marco@PC:/usr/local/kafka$ bin/kafka-topics.sh --list --zookeeper localhost:2181
test
marco@PC:/usr/local/kafka$ bin/kafka-console-producer.sh --broker-list localhost:9092   
--topic test
>This is a message
>This is another message
>

( ThirdWindow ). Наконец, в другом новом окне я запустил потребителя: 

marco@PC:/usr/local/kafka$ bin/kafka-console-consumer.sh --bootstrap-server  
localhost:9092 --topic test --from-beginning
This is a message
This is another message

: ( Четвертое окно )

Что я на самом деле сейчас не понимаю, так это почему я должен открыть все эти четыре сессии.

Marco

0 голосов
/ 04 сентября 2018 
$ bin/zookeeper-server-start.sh  config/zookeeper.properties

Все началось хорошо, если вы посмотрите журналы

[2018-09-04 12: 18: 14,636] Привязка INFO к порту 0.0.0.0/0.0.0.0:2181
(Org.apache.zookeeper.server.NIOServerCnxnFactory)

И это сообщение будет храниться до тех пор, пока клиент Zookeeper, такой как Kafka, не подключится к Zookeeper.

Когда вы говорите, что «пытаетесь запустить kafka», вы снова запускаете zookeeper-server-start, а не kafka-server-start, поэтому Zookeeper уже связан с портом 2181 и не запускает второй.

...