Я интегрирую Spring и Kafka, но не могу установить sh соединений с Kafka Broker. у меня есть две машины, я установил kafka на машине A и хочу подключиться с машины B, но у меня возникли некоторые проблемы с подключением, в трассировке я получил эти повторяющиеся строки
- Попытка выбрать узлы для [Call (callName = descriptionTopics, deadlineMs = 1588452756281)] в 1588452636314
- Метаданные не готовы: мы еще не получили метаданные из узлов bootstrap.
- Невозможно назначить Call (callName = descriptionTopics, deadlineMs = 1588452756281) на узел.
- Клиент не готов к отправке на 192.168.1.2:9092 (id: -1 rack: null). Время задержки 9223372036854775807 мс
- Вход в опрос KafkaClient # (время ожидания = 100)
- Опрос KafkaClient # получен 0 ответ (ов)
Файл журнала
2020-05-03 02:20:36.133 INFO 22603 --- [ main] o.s.s.c.ThreadPoolTaskScheduler : Initializing ExecutorService 'taskScheduler'
2020-05-03 02:20:36.222 INFO 22603 --- [ main] o.a.k.clients.admin.AdminClientConfig : AdminClientConfig values:
bootstrap.servers = [192.168.1.2:9092]
client.dns.lookup = default
client.id =
connections.max.idle.ms = 300000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
receive.buffer.bytes = 65536
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 120000
retries = 5
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = https
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
2020-05-03 02:20:36.230 DEBUG 22603 --- [ main] o.a.k.c.a.i.AdminMetadataManager : [AdminClient clientId=adminclient-1] Setting bootstrap cluster metadata Cluster(id = null, nodes = [192.168.1.2:9092 (id: -1 rack: null)], partitions = [], controller = null).
2020-05-03 02:20:36.235 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=count, group=kafka-metrics-count, description=total number of registered metrics, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.246 DEBUG 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Added sensor with name connections-closed:
2020-05-03 02:20:36.249 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=connection-close-total, group=admin-client-metrics, description=The total number of connections closed, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.250 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=connection-close-rate, group=admin-client-metrics, description=The number of connections closed per second, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.250 DEBUG 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Added sensor with name connections-created:
2020-05-03 02:20:36.251 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=connection-creation-total, group=admin-client-metrics, description=The total number of new connections established, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.252 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=connection-creation-rate, group=admin-client-metrics, description=The number of new connections established per second, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.252 DEBUG 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Added sensor with name successful-authentication:
2020-05-03 02:20:36.252 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=successful-authentication-total, group=admin-client-metrics, description=The total number of connections with successful authentication, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.252 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=successful-authentication-rate, group=admin-client-metrics, description=The number of connections with successful authentication per second, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.252 DEBUG 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Added sensor with name successful-reauthentication:
2020-05-03 02:20:36.253 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=successful-reauthentication-total, group=admin-client-metrics, description=The total number of successful re-authentication of connections, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.253 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=successful-reauthentication-rate, group=admin-client-metrics, description=The number of successful re-authentication of connections per second, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.253 DEBUG 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Added sensor with name successful-authentication-no-reauth:
2020-05-03 02:20:36.253 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=successful-authentication-no-reauth-total, group=admin-client-metrics, description=The total number of connections with successful authentication where the client does not support re-authentication, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.254 DEBUG 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Added sensor with name failed-authentication:
2020-05-03 02:20:36.255 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=failed-authentication-total, group=admin-client-metrics, description=The total number of connections with failed authentication, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.256 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=failed-authentication-rate, group=admin-client-metrics, description=The number of connections with failed authentication per second, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.256 DEBUG 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Added sensor with name failed-reauthentication:
2020-05-03 02:20:36.256 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=failed-reauthentication-total, group=admin-client-metrics, description=The total number of failed re-authentication of connections, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.257 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=failed-reauthentication-rate, group=admin-client-metrics, description=The number of failed re-authentication of connections per second, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.257 DEBUG 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Added sensor with name reauthentication-latency:
2020-05-03 02:20:36.257 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=reauthentication-latency-max, group=admin-client-metrics, description=The max latency observed due to re-authentication, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.258 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=reauthentication-latency-avg, group=admin-client-metrics, description=The average latency observed due to re-authentication, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.258 DEBUG 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Added sensor with name bytes-sent-received:
2020-05-03 02:20:36.259 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=network-io-total, group=admin-client-metrics, description=The total number of network operations (reads or writes) on all connections, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.260 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=network-io-rate, group=admin-client-metrics, description=The number of network operations (reads or writes) on all connections per second, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.260 DEBUG 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Added sensor with name bytes-sent:
2020-05-03 02:20:36.260 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=outgoing-byte-total, group=admin-client-metrics, description=The total number of outgoing bytes sent to all servers, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.261 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=outgoing-byte-rate, group=admin-client-metrics, description=The number of outgoing bytes sent to all servers per second, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.261 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=request-total, group=admin-client-metrics, description=The total number of requests sent, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.261 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=request-rate, group=admin-client-metrics, description=The number of requests sent per second, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.262 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=request-size-avg, group=admin-client-metrics, description=The average size of requests sent., tags={client-id=adminclient-1}]
2020-05-03 02:20:36.262 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=request-size-max, group=admin-client-metrics, description=The maximum size of any request sent., tags={client-id=adminclient-1}]
2020-05-03 02:20:36.262 DEBUG 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Added sensor with name bytes-received:
2020-05-03 02:20:36.263 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=incoming-byte-total, group=admin-client-metrics, description=The total number of bytes read off all sockets, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.263 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=incoming-byte-rate, group=admin-client-metrics, description=The number of bytes read off all sockets per second, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.264 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=response-total, group=admin-client-metrics, description=The total number of responses received, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.264 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=response-rate, group=admin-client-metrics, description=The number of responses received per second, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.264 DEBUG 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Added sensor with name select-time:
2020-05-03 02:20:36.264 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=select-total, group=admin-client-metrics, description=The total number of times the I/O layer checked for new I/O to perform, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.265 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=select-rate, group=admin-client-metrics, description=The number of times the I/O layer checked for new I/O to perform per second, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.265 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=io-wait-time-ns-avg, group=admin-client-metrics, description=The average length of time the I/O thread spent waiting for a socket ready for reads or writes in nanoseconds., tags={client-id=adminclient-1}]
2020-05-03 02:20:36.266 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=io-waittime-total, group=admin-client-metrics, description=The total time the I/O thread spent waiting, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.266 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=io-wait-ratio, group=admin-client-metrics, description=The fraction of time the I/O thread spent waiting, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.267 DEBUG 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Added sensor with name io-time:
2020-05-03 02:20:36.267 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=io-time-ns-avg, group=admin-client-metrics, description=The average length of time for I/O per select call in nanoseconds., tags={client-id=adminclient-1}]
2020-05-03 02:20:36.267 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=iotime-total, group=admin-client-metrics, description=The total time the I/O thread spent doing I/O, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.267 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=io-ratio, group=admin-client-metrics, description=The fraction of time the I/O thread spent doing I/O, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.268 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=connection-count, group=admin-client-metrics, description=The current number of active connections., tags={client-id=adminclient-1}]
2020-05-03 02:20:36.276 INFO 22603 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka version: 2.3.1
2020-05-03 02:20:36.276 INFO 22603 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka commitId: 18a913733fb71c01
2020-05-03 02:20:36.276 INFO 22603 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka startTimeMs: 1588452636274
2020-05-03 02:20:36.279 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=version, group=app-info, description=Metric indicating version, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.279 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=commit-id, group=app-info, description=Metric indicating commit-id, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.279 TRACE 22603 --- [ main] org.apache.kafka.common.metrics.Metrics : Registered metric named MetricName [name=start-time-ms, group=app-info, description=Metric indicating start-time-ms, tags={client-id=adminclient-1}]
2020-05-03 02:20:36.280 DEBUG 22603 --- [ main] o.a.k.clients.admin.KafkaAdminClient : [AdminClient clientId=adminclient-1] Kafka admin client initialized
2020-05-03 02:20:36.280 TRACE 22603 --- [| adminclient-1] o.a.k.clients.admin.KafkaAdminClient : [AdminClient clientId=adminclient-1] Thread starting
2020-05-03 02:20:36.281 TRACE 22603 --- [| adminclient-1] o.a.k.clients.admin.KafkaAdminClient : [AdminClient clientId=adminclient-1] Trying to choose nodes for [] at 1588452636280
2020-05-03 02:20:36.283 TRACE 22603 --- [| adminclient-1] org.apache.kafka.clients.NetworkClient : [AdminClient clientId=adminclient-1] Found least loaded node 192.168.1.2:9092 (id: -1 rack: null) with no active connection
2020-05-03 02:20:36.283 DEBUG 22603 --- [ main] o.a.k.clients.admin.KafkaAdminClient : [AdminClient clientId=adminclient-1] Queueing Call(callName=describeTopics, deadlineMs=1588452756281) with a timeout 120000 ms from now.
2020-05-03 02:20:36.283 TRACE 22603 --- [| adminclient-1] o.a.k.clients.admin.KafkaAdminClient : [AdminClient clientId=adminclient-1] Assigned Call(callName=fetchMetadata, deadlineMs=1588452756280) to node 192.168.1.2:9092 (id: -1 rack: null)
2020-05-03 02:20:36.285 DEBUG 22603 --- [| adminclient-1] org.apache.kafka.clients.NetworkClient : [AdminClient clientId=adminclient-1] Initiating connection to node 192.168.1.2:9092 (id: -1 rack: null) using address /192.168.1.2
2020-05-03 02:20:36.311 TRACE 22603 --- [| adminclient-1] o.a.k.clients.admin.KafkaAdminClient : [AdminClient clientId=adminclient-1] Client is not ready to send to 192.168.1.2:9092 (id: -1 rack: null). Must delay 9223372036854775807 ms
2020-05-03 02:20:36.311 TRACE 22603 --- [| adminclient-1] o.a.k.clients.admin.KafkaAdminClient : [AdminClient clientId=adminclient-1] Entering KafkaClient#poll(timeout=1200000)
2020-05-03 02:20:36.314 TRACE 22603 --- [| adminclient-1] o.a.k.clients.admin.KafkaAdminClient : [AdminClient clientId=adminclient-1] KafkaClient#poll retrieved 0 response(s)
2020-05-03 02:20:36.314 TRACE 22603 --- [| adminclient-1] o.a.k.clients.admin.KafkaAdminClient : [AdminClient clientId=adminclient-1] Trying to choose nodes for [Call(callName=describeTopics, deadlineMs=1588452756281)] at 1588452636314
2020-05-03 02:20:36.315 TRACE 22603 --- [| adminclient-1] o.a.k.c.a.i.AdminMetadataManager : [AdminClient clientId=adminclient-1] Metadata is not ready: we have not fetched metadata from the bootstrap nodes yet.
2020-05-03 02:20:36.315 TRACE 22603 --- [| adminclient-1] o.a.k.clients.admin.KafkaAdminClient : [AdminClient clientId=adminclient-1] Unable to assign Call(callName=describeTopics, deadlineMs=1588452756281) to a node.
2020-05-03 02:20:36.315 TRACE 22603 --- [| adminclient-1] o.a.k.clients.admin.KafkaAdminClient : [AdminClient clientId=adminclient-1] Client is not ready to send to 192.168.1.2:9092 (id: -1 rack: null). Must delay 9223372036854775807 ms
2020-05-03 02:20:36.315 TRACE 22603 --- [| adminclient-1] o.a.k.clients.admin.KafkaAdminClient : [AdminClient clientId=adminclient-1] Entering KafkaClient#poll(timeout=100)
Конфигурации Spring
spring:
kafka:
consumer:
bootstrap-servers: 192.168.1.2:9092
group-id: group_id
auto-offset-reset: earliest
key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
value-deserializer: org.apache.kafka.common.serialization.StringDeserializer
producer:
bootstrap-servers: 192.168.1.2:9092
key-serializer: org.apache.kafka.common.serialization.StringSerializer
value-serializer: org.apache.kafka.common.serialization.StringSerializer
Файл сервера Kafka
broker.id=0
advertised.listeners=PLAINTEXT://192.168.1.2.com:9092
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/tmp/kafka-logs
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=localhost:2181
zookeeper.connection.timeout.ms=18000
group.initial.rebalance.delay.ms=0