Не могу запустить Кафку на Mac - PullRequest
0 голосов
/ 25 февраля 2020

Я очень новичок в использовании Microservices и у меня возникают проблемы с запуском Kafka после того, как я запустил zookeeper.

Zookeeper запускается нормально, но при попытке запустить сервер Kafka выдает ошибку.

Я искал в Google, чтобы попытаться решить мою проблему, но это было довольно сложно, так как я не уверен, что означают / делают все эти различные конфигурационные файлы.

Я попытался включить слушателей = PLAINTEXT: //: 9092 в настройках сервера, но это не работает.

Я также попытался удалить и переустановить Kafka и ZooKeeper, но я все еще получаю ту же ошибку.

Antonias-MacBook-Pro:kafka antonia$ kafka-server-start /usr/local/etc/kafka/server.properties
[2020-02-25 10:26:55,944] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2020-02-25 10:26:56,514] INFO Registered signal handlers for TERM, INT, HUP (org.apache.kafka.common.utils.LoggingSignalHandler)
[2020-02-25 10:26:56,514] INFO starting (kafka.server.KafkaServer)
[2020-02-25 10:26:56,515] INFO Connecting to zookeeper on localhost:2181 (kafka.server.KafkaServer)
[2020-02-25 10:26:56,541] INFO [ZooKeeperClient Kafka server] Initializing a new session to localhost:2181. (kafka.zookeeper.ZooKeeperClient)
[2020-02-25 10:26:56,571] INFO Client environment:zookeeper.version=3.5.6-c11b7e26bc554b8523dc929761dd28808913f091, built on 10/08/2019 20:18 GMT (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,571] INFO Client environment:host.name=10.111.180.36 (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,571] INFO Client environment:java.version=1.8.0_242 (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,571] INFO Client environment:java.vendor=AdoptOpenJDK (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,571] INFO Client environment:java.home=/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/jre (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,571] INFO Client environment:java.class.path=/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/activation-1.1.1.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/aopalliance-repackaged-2.5.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/argparse4j-0.7.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/audience-annotations-0.5.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/commons-cli-1.4.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/commons-lang3-3.8.1.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/connect-api-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/connect-basic-auth-extension-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/connect-file-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/connect-json-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/connect-mirror-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/connect-mirror-client-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/connect-runtime-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/connect-transforms-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/guava-20.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/hk2-api-2.5.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/hk2-locator-2.5.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/hk2-utils-2.5.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jackson-annotations-2.10.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jackson-core-2.10.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jackson-databind-2.10.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jackson-dataformat-csv-2.10.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jackson-datatype-jdk8-2.10.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jackson-jaxrs-base-2.10.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jackson-jaxrs-json-provider-2.10.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jackson-module-jaxb-annotations-2.10.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jackson-module-paranamer-2.10.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jackson-module-scala_2.12-2.10.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jakarta.activation-api-1.2.1.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jakarta.annotation-api-1.3.4.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jakarta.inject-2.5.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jakarta.ws.rs-api-2.1.5.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jakarta.xml.bind-api-2.3.2.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/javassist-3.22.0-CR2.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/javax.servlet-api-3.1.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/javax.ws.rs-api-2.1.1.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jaxb-api-2.3.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jersey-client-2.28.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jersey-common-2.28.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jersey-container-servlet-2.28.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jersey-container-servlet-core-2.28.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jersey-hk2-2.28.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jersey-media-jaxb-2.28.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jersey-server-2.28.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jetty-client-9.4.20.v20190813.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jetty-continuation-9.4.20.v20190813.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jetty-http-9.4.20.v20190813.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jetty-io-9.4.20.v20190813.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jetty-security-9.4.20.v20190813.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jetty-server-9.4.20.v20190813.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jetty-servlet-9.4.20.v20190813.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jetty-servlets-9.4.20.v20190813.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jetty-util-9.4.20.v20190813.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/jopt-simple-5.0.4.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/kafka-clients-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/kafka-log4j-appender-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/kafka-streams-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/kafka-streams-examples-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/kafka-streams-scala_2.12-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/kafka-streams-test-utils-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/kafka-tools-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/kafka_2.12-2.4.0-sources.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/kafka_2.12-2.4.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/log4j-1.2.17.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/lz4-java-1.6.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/maven-artifact-3.6.1.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/metrics-core-2.2.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/netty-buffer-4.1.42.Final.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/netty-codec-4.1.42.Final.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/netty-common-4.1.42.Final.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/netty-handler-4.1.42.Final.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/netty-resolver-4.1.42.Final.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/netty-transport-4.1.42.Final.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/netty-transport-native-epoll-4.1.42.Final.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/netty-transport-native-unix-common-4.1.42.Final.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/osgi-resource-locator-1.0.1.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/paranamer-2.8.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/plexus-utils-3.2.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/reflections-0.9.11.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/rocksdbjni-5.18.3.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/scala-collection-compat_2.12-2.1.2.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/scala-java8-compat_2.12-0.9.0.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/scala-library-2.12.10.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/scala-logging_2.12-3.9.2.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/scala-reflect-2.12.10.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/slf4j-api-1.7.28.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/slf4j-log4j12-1.7.28.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/snappy-java-1.1.7.3.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/validation-api-2.0.1.Final.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/zookeeper-3.5.6.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/zookeeper-jute-3.5.6.jar:/usr/local/Cellar/kafka/2.4.0/libexec/bin/../libs/zstd-jni-1.4.3-1.jar (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,572] INFO Client environment:java.library.path=/Users/antonia/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:. (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,572] INFO Client environment:java.io.tmpdir=/var/folders/13/gvhn5dqj4179ynmpm6pkp9kh0000gn/T/ (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,572] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,572] INFO Client environment:os.name=Mac OS X (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,572] INFO Client environment:os.arch=x86_64 (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,573] INFO Client environment:os.version=10.14.5 (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,573] INFO Client environment:user.name=antonia (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,573] INFO Client environment:user.home=/Users/antonia (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,573] INFO Client environment:user.dir=/usr/local/etc/kafka (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,573] INFO Client environment:os.memory.free=977MB (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,573] INFO Client environment:os.memory.max=1024MB (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,573] INFO Client environment:os.memory.total=1024MB (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,575] INFO Initiating client connection, connectString=localhost:2181 sessionTimeout=6000 watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@28eaa59a (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:56,582] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2020-02-25 10:26:56,592] INFO jute.maxbuffer value is 4194304 Bytes (org.apache.zookeeper.ClientCnxnSocket)
[2020-02-25 10:26:56,599] INFO zookeeper.request.timeout value is 0. feature enabled= (org.apache.zookeeper.ClientCnxn)
[2020-02-25 10:26:56,601] INFO [ZooKeeperClient Kafka server] Waiting until connected. (kafka.zookeeper.ZooKeeperClient)
[2020-02-25 10:26:56,605] INFO Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2020-02-25 10:26:56,625] INFO Socket connection established, initiating session, client: /0:0:0:0:0:0:0:1:63152, server: localhost/0:0:0:0:0:0:0:1:2181 (org.apache.zookeeper.ClientCnxn)
[2020-02-25 10:26:56,641] INFO Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x10004969600000a, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
[2020-02-25 10:26:56,645] INFO [ZooKeeperClient Kafka server] Connected. (kafka.zookeeper.ZooKeeperClient)
[2020-02-25 10:26:56,979] INFO Cluster ID = S4SZ31nVRTCQ4uwRJ9_7mg (kafka.server.KafkaServer)
[2020-02-25 10:26:56,992] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.common.InconsistentClusterIdException: The Cluster ID S4SZ31nVRTCQ4uwRJ9_7mg doesn't match stored clusterId Some(Y_mQi4q4TSuhlWdx4DHiaQ) in meta.properties. The broker is trying to join the wrong cluster. Configured zookeeper.connect may be wrong.
	at kafka.server.KafkaServer.startup(KafkaServer.scala:220)
	at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44)
	at kafka.Kafka$.main(Kafka.scala:84)
	at kafka.Kafka.main(Kafka.scala)
[2020-02-25 10:26:56,995] INFO shutting down (kafka.server.KafkaServer)
[2020-02-25 10:26:56,997] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient)
[2020-02-25 10:26:57,109] INFO Session: 0x10004969600000a closed (org.apache.zookeeper.ZooKeeper)
[2020-02-25 10:26:57,109] INFO EventThread shut down for session: 0x10004969600000a (org.apache.zookeeper.ClientCnxn)
[2020-02-25 10:26:57,111] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient)
[2020-02-25 10:26:57,118] INFO shut down completed (kafka.server.KafkaServer)
[2020-02-25 10:26:57,119] ERROR Exiting Kafka. (kafka.server.KafkaServerStartable)
[2020-02-25 10:26:57,120] INFO shutting down (kafka.server.KafkaServer)

1 Ответ

1 голос
/ 25 февраля 2020

Причина проблемы показана в этом сообщении:

kafka.common.InconsistentClusterIdException:

Идентификатор кластера S4SZ31nVRTCQ4uwRJ9_7mg

не соответствует сохраненному clusterId Some (Y_mQi4q4TSuhlWdx4DHiaQ)

в meta.properties.

Брокер пытается присоединиться к неправильному кластеру.
Возможно, неправильно настроен zookeeper.connect.

Данная проблема возникает при запуске нового экземпляра Kafka. на хранилище данных, созданное другим сервером kafka. Кафка хранит свои сообщения в лог-файлах.

Как решить проблему?

Проблема может быть решена с помощью следующих шагов:

  1. Выключение Kafka и Zookeeper
  2. При необходимости создайте резервную копию существующих журналов Kafka и Zookeeper
  3. Удалите каталоги журналов Kafka и Zookeeper
  4. Перезапустите Zookeeper и Kafka
...