Дебезиум: Невозможно настроить наблюдателя на контейнере Kafka-Connect Docker. Ошибка при получении метаданных с идентификатором корреляции 2 ... LEADER_NOT_AVAILABLE - PullRequest
Новая
       83

Дебезиум: Невозможно настроить наблюдателя на контейнере Kafka-Connect Docker. Ошибка при получении метаданных с идентификатором корреляции 2 ... LEADER_NOT_AVAILABLE

3 голосов
/ 08 апреля 2020

Вопрос:

Как настроить watcher для наблюдения за потоком Kafka-Connect в режиме реального времени при внесении изменений в базу данных? Я получаю Error while fetching metadata with correlation id 2 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE}. В чем причина проблемы и как ее исправить?

Контекст:

Этот вопрос относится к: Дебезиум Как правильно зарегистрировать соединитель SqlServer с Kafka Connect - соединение отклонено и https://github.com/debezium/debezium-examples/blob/master/tutorial/README.md#using - sql -сервер

В Windows 10 я настроил Zookeeper, Kafka и Kafka-Connector все в их собственные отдельные Docker контейнеры с помощью следующих команд: 

docker run -it --rm --name zookeeper -p 2181:2181 -p 2888:2888 -p 3888:3888 debezium/zookeeper:1.1
docker run -it --rm --name kafka -p 9092:9092 -e KAFKA_ADVERTISED_HOST_NAME: kafka --link zookeeper:zookeeper debezium/kafka:1.1
docker run -it --rm --name connect -p 8083:8083 -e GROUP_ID=1 -e CONFIG_STORAGE_TOPIC=my_connect_configs -e OFFSET_STORAGE_TOPIC=my_connect_offsets -e STATUS_STORAGE_TOPIC=my_connect_statuses --link zookeeper:zookeeper --link kafka:kafka debezium/connect:1.1

Затем я настраиваю Kafka Connector на экземпляр SQL Сервера, который работает на локальном хосте, вне контейнера Docker. Журналы показывают, что соединитель может подключиться к базе данных и получить данные CD C. До этого момента я был в порядке.

Однако, когда я пытаюсь создать наблюдателя, используя следующую команду: 

docker run -it --rm --name watcher --link zookeeper:zookeeper --link kafka:kafka debezium/kafka:1.1

Я получаю следующую ошибку. 

watch-topic -a -k dbhistory.Test.Posts
WARNING: Using default BROKER_ID=1, which is valid only for non-clustered installations.
Using ZOOKEEPER_CONNECT=172.17.0.2:2181
Using KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://172.17.0.5:9092
Using KAFKA_BROKER=172.17.0.3:9092
Contents of topic dbhistory.Test.Posts:
[2020-04-08 15:18:39,373] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 2 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:20,501] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:20,605] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:20,807] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:21,262] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:22,120] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:23,330] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:24,189] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:59,826] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:04,051] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:07,121] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:08,024] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,354] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11570 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,458] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11572 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,571] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11574 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,677] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11576 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,786] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11578 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,895] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11580 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,002] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11582 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,112] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11584 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,220] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11586 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,326] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11588 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,435] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11590 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,537] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11592 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,648] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11594 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,751] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11596 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,854] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11598 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 17:08:48,268] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 17:09:55,602] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)

1 Ответ

0 голосов
/ 13 апреля 2020

Можете ли вы попробовать один раз добавить --network = host в docker run.

Добро пожаловать на сайт PullRequest, где вы можете задавать вопросы и получать ответы от других членов сообщества.

Похожие темы

...