Python Производитель Kafka не может написать в тему - PullRequest
0 голосов
/ 14 марта 2020

Я использую python производитель kafka для записи в топи c. Сервер Kafka является внешним, и я подключился с учетными данными, используя Kerberos. Когда я запускаю сценарий производителя, создается впечатление, что соединение с сервером устанавливается, но по какой-то причине запрос метаданных завершается с ошибкой «Запрос 1: MetadataRequest_v1 (themes = NULL)».

Вот код производителя:

from kafka import KafkaProducer

broker = ['mybroker.com:9092']

p = KafkaProducer(bootstrap_servers=broker,security_protocol='SSL',ssl_cafile="truststore.pem",ssl_keyfile='truststore.jks',sasl_mechanism='GSSAPI',api_version=(0, 10, 1))
topic = 'test-topic'
# Write hello world to test topic
p.send(topic, bytes("Hello World", 'utf-8'))
p.flush()

Вот ошибка, которую я сейчас получаю:

EBUG:kafka.producer.kafka:Starting the Kafka producer
DEBUG:kafka.metrics.metrics:Added sensor with name connections-closed
DEBUG:kafka.metrics.metrics:Added sensor with name connections-created
DEBUG:kafka.metrics.metrics:Added sensor with name select-time
DEBUG:kafka.metrics.metrics:Added sensor with name io-time
INFO:kafka.client:Bootstrapping cluster metadata from [('fakehost.lef.ceg.com', 9092, <AddressFamily.AF_UNSPEC: 0>)]
DEBUG:kafka.client:Attempting to bootstrap via node at mybroker.com:9092
DEBUG:kafka.metrics.metrics:Added sensor with name bytes-sent-received
DEBUG:kafka.metrics.metrics:Added sensor with name bytes-sent
DEBUG:kafka.metrics.metrics:Added sensor with name bytes-received
DEBUG:kafka.metrics.metrics:Added sensor with name request-latency
DEBUG:kafka.metrics.metrics:Added sensor with name node-bootstrap.bytes-sent
DEBUG:kafka.metrics.metrics:Added sensor with name node-bootstrap.bytes-received
DEBUG:kafka.metrics.metrics:Added sensor with name node-bootstrap.latency
DEBUG:kafka.conn:<BrokerConnection node_id=bootstrap host=:)>: creating new socket
DEBUG:kafka.conn:<BrokerConnection node_id=bootstrap host=:)=9092>: setting socket option (6, 1, 1)
DEBUG:kafka.conn:<BrokerConnection node_id=bootstrap host=:)/iphere port=9092>: configuring default SSL Context
INFO:kafka.conn:<BrokerConnection node_id=bootstrap host=:)/iphere port=9092>: Loading SSL CA from /truststore.pem
DEBUG:kafka.conn:<BrokerConnection node_id=bootstrap host=:)/iphere port=9092>: wrapping socket in ssl context
INFO:kafka.conn:<BrokerConnection node_id=bootstrap host=:)/iphere port=9092>: connecting to iphere
DEBUG:kafka.conn:<BrokerConnection node_id=bootstrap host=:)/iphere port=9092>: established TCP connection
DEBUG:kafka.conn:<BrokerConnection node_id=bootstrap host=:)/iphere port=9092>: initiating SSL handshake
DEBUG:kafka.conn:<BrokerConnection node_id=bootstrap host=:)/iphere port=9092>: completed SSL handshake.
DEBUG:kafka.conn:<BrokerConnection node_id=bootstrap host=:)/iphere port=9092>: Connection complete.
DEBUG:kafka.client:Node bootstrap connected
DEBUG:kafka.conn:<BrokerConnection node_id=bootstrap host=:)/iphere port=9092> Request 1: MetadataRequest_v1(topics=NULL)
ERROR:kafka.conn:<BrokerConnection node_id=bootstrap host=:)/iphere port=9092>: socket disconnected
INFO:kafka.conn:<BrokerConnection node_id=bootstrap host=:)/iphere port=9092>: Closing connection. ConnectionError: socket disconnected
DEBUG:kafka.conn:<BrokerConnection node_id=bootstrap host=:)/iphere port=9092>: reconnect backoff 0.04487732169774206 after 1 failures
ERROR:kafka.client:Unable to bootstrap from [('mybroker.com', 9092, <AddressFamily.AF_UNSPEC: 0>)]
DEBUG:kafka.metrics.metrics:Added sensor with name bufferpool-wait-time
DEBUG:kafka.metrics.metrics:Added sensor with name batch-size
DEBUG:kafka.metrics.metrics:Added sensor with name compression-rate
DEBUG:kafka.metrics.metrics:Added sensor with name queue-time
DEBUG:kafka.metrics.metrics:Added sensor with name produce-throttle-time
DEBUG:kafka.metrics.metrics:Added sensor with name records-per-request
DEBUG:kafka.metrics.metrics:Added sensor with name bytes
DEBUG:kafka.metrics.metrics:Added sensor with name record-retries
DEBUG:kafka.metrics.metrics:Added sensor with name errors
DEBUG:kafka.metrics.metrics:Added sensor with name record-size-max
DEBUG:kafka.producer.sender:Starting Kafka producer I/O thread.
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.producer.kafka:Kafka producer started
DEBUG:kafka.producer.kafka:Requesting metadata update for topic test-topic
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available
DEBUG:kafka.client:Give up sending metadata request since no node is available

Что это на самом деле означает и как мне go отладить это?

...