У меня есть приложение весенней загрузки, и у нас есть 3 брокера, которые не находятся под моим управлением, и я не могу подключиться. Когда я пробую свое приложение с моей локальной версией кафки с одним брокером 1.0.1, оно подключается, и я могу производить или потреблять, но.
Их кафка - версия 1.0.1. Моя версия клиента 2.2.0. Я даже не знаю, что такое приведенный ниже журнал.
Когда мое приложение запускается, оно использует KafkaAdmin и что-то регистрирует, пытается получить темы kafka и т. Д.
Когда я изучаю журналы, я вижуlot error_code = 35, это означает, что версия API не поддерживается. (https://kafka.apache.org/protocol#protocol_error_codes), но, как я уже сказал, я могу подключиться и создавать сообщения.
Я знаю, что версия Кафки 1.0.1 поддерживает клиентскую версию Кафки 2.2.0. У меня есть springframework.kafka версии 1.3.6RELEASE.
Я не уверен, что это проблема с версией или нет, их машина - Windows, но сервер находится на машине linux и dockerized kafka с использованием docker swarm. Возможно ли, что не удается подключиться? Но нижелоги показывают, что подключены, но не правильно. Здесь (https://github.com/wurstmeister/kafka-docker/issues/456). Может быть, их слушатели неправильно настроены, я точно не знаю, и у меня нет возможности узнать это.
Есть ли вероятность, чтоони обновили свою версию kafka и не установили конфигурацию inter.broker.protocol.version, чтобы их брокер не мог общаться ?? Еще одна вещь, я использую керберизованную kafka, которая взаимодействует через SASL_PLAINTEXT. Однако клиент не может подключить kafka с или безKerberos, так что я пропускаю эту точку. Здесь ( Обновление Kafka: необходимо указать inter.broker.protocol.versИон при обновлении? )
2019-10-17 08:20:26.796 DEBUG [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Node 2 disconnected.
2019-10-17 08:20:26.796 TRACE [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Removing node kafka3:9092 (id: 2 rack: null) from least loaded node selection: is-blacked-out: true, in-flight-requests: 0
2019-10-17 08:20:26.796 TRACE [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Removing node kafka1:9092 (id: 0 rack: null) from least loaded node selection: is-blacked-out: false, in-flight-requests: 0
2019-10-17 08:20:26.796 TRACE [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Found least loaded node kafka2:9092 (id: 1 rack: null)
2019-10-17 08:20:26.797 DEBUG [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Initialize connection to node kafka2:9092 (id: 1 rack: null) for sending metadata request
2019-10-17 08:20:26.798 DEBUG [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Initiating connection to node kafka2:9092 (id: 1 rack: null) using address kafka2/10.194.63.156
2019-10-17 08:20:26.799 DEBUG [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] o.apache.kafka.common.network.Selector : [Producer clientId=producer-1] Created socket with SO_RCVBUF = 32768, SO_SNDBUF = 131072, SO_TIMEOUT = 0 to node 1
2019-10-17 08:20:26.799 DEBUG [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Completed connection to node 1. Fetching API versions.
2019-10-17 08:20:26.800 DEBUG [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Initiating API versions fetch from node 1.
2019-10-17 08:20:26.800 TRACE [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] No version information found when sending API_VERSIONS with correlation id 51 to node 1. Assuming version 2.
2019-10-17 08:20:26.800 TRACE [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Sending API_VERSIONS {} with correlation id 51 to node 1
2019-10-17 08:20:26.800 TRACE [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Removing node kafka3:9092 (id: 2 rack: null) from least loaded node selection: is-blacked-out: true, in-flight-requests: 0
2019-10-17 08:20:26.800 TRACE [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Removing node kafka2:9092 (id: 1 rack: null) from least loaded node selection: is-blacked-out: false, in-flight-requests: 1
2019-10-17 08:20:26.800 TRACE [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Found least loaded node kafka1:9092 (id: 0 rack: null)
2019-10-17 08:20:26.800 DEBUG [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Sending metadata request (type=MetadataRequest, topics=) to node kafka1:9092 (id: 0 rack: null)
2019-10-17 08:20:26.800 DEBUG [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Using older server API v5 to send METADATA {topics=[],allow_auto_topic_creation=true} with correlation id 52 to node 0
2019-10-17 08:20:26.803 TRACE [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Completed receive from node 1 for API_VERSIONS with correlation id 51, received {error_code=35,api_versions=[]}
2019-10-17 08:20:26.803 DEBUG [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Initiating API versions fetch from node 1.
2019-10-17 08:20:26.803 TRACE [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] No version information found when sending API_VERSIONS with correlation id 53 to node 1. Assuming version 0.
2019-10-17 08:20:26.803 DEBUG [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Using older server API v0 to send API_VERSIONS {} with correlation id 53 to node 1
2019-10-17 08:20:26.805 TRACE [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Completed receive from node 1 for API_VERSIONS with correlation id 53, received {error_code=0,api_versions=[{api_key=0,min_version=0,max_version=5},{api_key=1,min_version=0,max_version=6},{api_key=2,min_version=0,max_version=2},{api_key=3,min_version=0,max_version=5},{api_key=4,min_version=0,max_version=1},{api_key=5,min_version=0,max_version=0},{api_key=6,min_version=0,max_version=4},{api_key=7,min_version=0,max_version=1},{api_key=8,min_version=0,max_version=3},{api_key=9,min_version=0,max_version=3},{api_key=10,min_version=0,max_version=1},{api_key=11,min_version=0,max_version=2},{api_key=12,min_version=0,max_version=1},{api_key=13,min_version=0,max_version=1},{api_key=14,min_version=0,max_version=1},{api_key=15,min_version=0,max_version=1},{api_key=16,min_version=0,max_version=1},{api_key=17,min_version=0,max_version=1},{api_key=18,min_version=0,max_version=1},{api_key=19,min_version=0,max_version=2},{api_key=20,min_version=0,max_version=1},{api_key=21,min_version=0,max_version=0},{api_key=22,min_version=0,max_version=0},{api_key=23,min_version=0,max_version=0},{api_key=24,min_version=0,max_version=0},{api_key=25,min_version=0,max_version=0},{api_key=26,min_version=0,max_version=0},{api_key=27,min_version=0,max_version=0},{api_key=28,min_version=0,max_version=0},{api_key=29,min_version=0,max_version=0},{api_key=30,min_version=0,max_version=0},{api_key=31,min_version=0,max_version=0},{api_key=32,min_version=0,max_version=0},{api_key=33,min_version=0,max_version=0},{api_key=34,min_version=0,max_version=0},{api_key=35,min_version=0,max_version=0},{api_key=36,min_version=0,max_version=0},{api_key=37,min_version=0,max_version=0}]}
2019-10-17 08:20:26.805 DEBUG [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Recorded API versions for node 1: (Produce(0): 0 to 5 [usable: 5], Fetch(1): 0 to 6 [usable: 6], ListOffsets(2): 0 to 2 [usable: 2], Metadata(3): 0 to 5 [usable: 5], LeaderAndIsr(4): 0 to 1 [usable: 1], StopReplica(5): 0 [usable: 0], UpdateMetadata(6): 0 to 4 [usable: 4], ControlledShutdown(7): 0 to 1 [usable: 1], OffsetCommit(8): 0 to 3 [usable: 3], OffsetFetch(9): 0 to 3 [usable: 3], FindCoordinator(10): 0 to 1 [usable: 1], JoinGroup(11): 0 to 2 [usable: 2], Heartbeat(12): 0 to 1 [usable: 1], LeaveGroup(13): 0 to 1 [usable: 1], SyncGroup(14): 0 to 1 [usable: 1], DescribeGroups(15): 0 to 1 [usable: 1], ListGroups(16): 0 to 1 [usable: 1], SaslHandshake(17): 0 to 1 [usable: 1], ApiVersions(18): 0 to 1 [usable: 1], CreateTopics(19): 0 to 2 [usable: 2], DeleteTopics(20): 0 to 1 [usable: 1], DeleteRecords(21): 0 [usable: 0], InitProducerId(22): 0 [usable: 0], OffsetForLeaderEpoch(23): 0 [usable: 0], AddPartitionsToTxn(24): 0 [usable: 0], AddOffsetsToTxn(25): 0 [usable: 0], EndTxn(26): 0 [usable: 0], WriteTxnMarkers(27): 0 [usable: 0], TxnOffsetCommit(28): 0 [usable: 0], DescribeAcls(29): 0 [usable: 0], CreateAcls(30): 0 [usable: 0], DeleteAcls(31): 0 [usable: 0], DescribeConfigs(32): 0 [usable: 0], AlterConfigs(33): 0 [usable: 0], AlterReplicaLogDirs(34): 0 [usable: 0], DescribeLogDirs(35): 0 [usable: 0], SaslAuthenticate(36): 0 [usable: 0], CreatePartitions(37): 0 [usable: 0], CreateDelegationToken(38): UNSUPPORTED, RenewDelegationToken(39): UNSUPPORTED, ExpireDelegationToken(40): UNSUPPORTED, DescribeDelegationToken(41): UNSUPPORTED, DeleteGroups(42): UNSUPPORTED, ElectPreferredLeaders(43): UNSUPPORTED)
2019-10-17 08:20:26.806 TRACE [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Completed receive from node 0 for METADATA with correlation id 52, received {throttle_time_ms=0,brokers=[{node_id=2,host=kafka3,port=9092,rack=null},{node_id=1,host=2,port=9092,rack=null},{node_id=0,host=kafka1,port=9092,rack=null}],cluster_id=8Tcl7QjQS16ldE0OXEZwGw,controller_id=0,topic_metadata=[]}
2019-10-17 08:20:26.806 DEBUG [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.Metadata : Updated cluster metadata version 23 to MetadataCache{cluster=Cluster(id = 8Tcl7QjQS16ldE0OXEZwGw, nodes = kafka1:9092 (id: 0 rack: null), kafka2:9092 (id: 1 rack: null), kafka3:9092 (id: 2 rack: null)], partitions = [], controller = kafka1:9092 (id: 0 rack: null))}
2019-10-17 08:25:26.834 TRACE [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Removing node kafka2:9092 (id: 1 rack: null) from least loaded node selection: is-blacked-out: false, in-flight-requests: 0
2019-10-17 08:25:26.834 TRACE [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Removing node kafka3:9092 (id: 2 rack: null) from least loaded node selection: is-blacked-out: false, in-flight-requests: 0
2019-10-17 08:25:26.834 TRACE [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Found least loaded node kafka1:9092 (id: 0 rack: null)
2019-10-17 08:25:26.834 DEBUG [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Sending metadata request (type=MetadataRequest, topics=) to node kafka1:9092 (id: 0 rack: null)
2019-10-17 08:25:26.834 DEBUG [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Using older server API v5 to send METADATA {topics=[],allow_auto_topic_creation=true} with correlation id 54 to node 0
2019-10-17 08:25:26.836 TRACE [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Completed receive from node 0 for METADATA with correlation id 54, received {throttle_time_ms=0,brokers=[{node_id=2,host=kafka3,port=9092,rack=null},{node_id=1,host=kafka2,port=9092,rack=null},{node_id=0,host=kafka1,port=9092,rack=null}],cluster_id=8Tcl7QjQS16ldE0OXEZwGw,controller_id=0,topic_metadata=[]}
2019-10-17 08:25:26.836 DEBUG [bootstrap,,,] 1 --- [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.Metadata : Updated cluster metadata version 24 to MetadataCache{cluster=Cluster(id = 8Tcl7QjQS16ldE0OXEZwGw, nodes = [kafka1:9092 (id: 0 rack: null), kafka2:9092 (id: 1 rack: null), kafka3:9092 (id: 2 rack: null)], partitions = [], controller = kafka1:9092 (id: 0 rack: null))}
журналы постоянно одинаковы в течение 1 минуты. Я очень застрял на этом этапе.
Спасибо!