Мы используем kafka 1.11, мы выдаем задержку почти 3 секунды в наших сообщениях после того, как мы развернем приложения наших потребителей
Мы обнаружили через журналы, что, когда мы добавляем новых потребителей или удаляем их (все, что вызывает изменение баланса kafka) стратегия) мы видим эти ошибки (мы находимся в режиме отладки):
2020-01-29 14:33:14,530 INFO [pool-27-thread-1] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_DefaultRdsDlqConsumer_Thread[pool-27-thread-1,5,main], groupId=global_placebet_dlq-contribution-consumer-group] Error sending fetch request (sessionId=2021010829, epoch=1245) to node 1: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:14,530 INFO [kafka-coordinator-heartbeat-thread | global_placebet_dlq-contribution-consumer-group] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_DefaultRdsDlqConsumer_Thread[pool-27-thread-1,5,main], groupId=global_placebet_dlq-contribution-consumer-group] Error sending fetch request (sessionId=1466537553, epoch=973) to node 2: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:14,988 INFO [pool-8-thread-1] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_OperatorProfileChangeEventConsumerVerticle_Thread[pool-8-thread-1,5,main], groupId=user-operator-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=1455035308, epoch=1240) to node 2: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:15,046 INFO [pool-6-thread-1] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_GameEventConsumerVerticle_Thread[pool-6-thread-1,5,main], groupId=game-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=947862232, epoch=1240) to node 2: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:15,218 INFO [pool-14-thread-2] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_PlaceBetEventConsumerVerticle_Thread[pool-14-thread-2,5,main], groupId=contribution-pbevent-consumer-group] Error sending fetch request (sessionId=673240750, epoch=1961) to node 2: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:15,227 INFO [pool-14-thread-1] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_PlaceBetEventConsumerVerticle_Thread[pool-14-thread-1,5,main], groupId=contribution-pbevent-consumer-group] Error sending fetch request (sessionId=1271635584, epoch=2359) to node 3: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:15,310 INFO [pool-8-thread-3] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_OperatorProfileChangeEventConsumerVerticle_Thread[pool-8-thread-3,5,main], groupId=user-operator-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=710763051, epoch=1239) to node 1: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:15,310 INFO [pool-8-thread-3] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_OperatorProfileChangeEventConsumerVerticle_Thread[pool-8-thread-3,5,main], groupId=user-operator-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=863755310, epoch=1241) to node 2: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:15,343 INFO [pool-8-thread-2] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_OperatorProfileChangeEventConsumerVerticle_Thread[pool-8-thread-2,5,main], groupId=user-operator-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=1033075261, epoch=1239) to node 1: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:15,344 INFO [pool-12-thread-1] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_OperatorProfileChangeEventConsumerVerticle_Thread[pool-12-thread-1,5,main], groupId=user-operator-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=1393213630, epoch=1238) to node 1: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:15,344 INFO [kafka-coordinator-heartbeat-thread | user-operator-consumer-group_73129f731eba/172.17.0.2] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_OperatorProfileChangeEventConsumerVerticle_Thread[pool-12-thread-1,5,main], groupId=user-operator-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=1601907582, epoch=1239) to node 3: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:15,344 INFO [kafka-coordinator-heartbeat-thread | user-operator-consumer-group_73129f731eba/172.17.0.2] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_OperatorProfileChangeEventConsumerVerticle_Thread[pool-8-thread-2,5,main], groupId=user-operator-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=550867345, epoch=1239) to node 3: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:15,390 INFO [pool-12-thread-2] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_OperatorProfileChangeEventConsumerVerticle_Thread[pool-12-thread-2,5,main], groupId=user-operator-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=1195097805, epoch=1238) to node 1: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:15,391 INFO [kafka-coordinator-heartbeat-thread | user-operator-consumer-group_73129f731eba/172.17.0.2] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_OperatorProfileChangeEventConsumerVerticle_Thread[pool-12-thread-2,5,main], groupId=user-operator-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=1022911425, epoch=1241) to node 2: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:15,431 INFO [pool-8-thread-1] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_OperatorProfileChangeEventConsumerVerticle_Thread[pool-8-thread-1,5,main], groupId=user-operator-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=1077958524, epoch=1239) to node 3: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:15,444 INFO [pool-6-thread-3] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_GameEventConsumerVerticle_Thread[pool-6-thread-3,5,main], groupId=game-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=1202424597, epoch=1238) to node 1: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:15,444 INFO [kafka-coordinator-heartbeat-thread | game-consumer-group_73129f731eba/172.17.0.2] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_GameEventConsumerVerticle_Thread[pool-6-thread-3,5,main], groupId=game-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=1292624284, epoch=1239) to node 3: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:15,477 INFO [pool-6-thread-1] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_GameEventConsumerVerticle_Thread[pool-6-thread-1,5,main], groupId=game-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=431511463, epoch=1238) to node 1: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:15,548 INFO [pool-6-thread-2] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_GameEventConsumerVerticle_Thread[pool-6-thread-2,5,main], groupId=game-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=803094402, epoch=1241) to node 2: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:15,549 INFO [pool-12-thread-3] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_OperatorProfileChangeEventConsumerVerticle_Thread[pool-12-thread-3,5,main], groupId=user-operator-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=836156481, epoch=1241) to node 2: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:15,549 INFO [pool-6-thread-2] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_GameEventConsumerVerticle_Thread[pool-6-thread-2,5,main], groupId=game-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=570345720, epoch=1239) to node 3: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:15,549 INFO [kafka-coordinator-heartbeat-thread | user-operator-consumer-group_73129f731eba/172.17.0.2] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_OperatorProfileChangeEventConsumerVerticle_Thread[pool-12-thread-3,5,main], groupId=user-operator-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=956900985, epoch=1239) to node 3: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:16,480 INFO [pool-14-thread-3] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_PlaceBetEventConsumerVerticle_Thread[pool-14-thread-3,5,main], groupId=contribution-pbevent-consumer-group] Error sending fetch request (sessionId=1501125789, epoch=2443) to node 1: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:16,673 INFO [kafka-coordinator-heartbeat-thread | player-opt-consumer-group_73129f731eba/172.17.0.2] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_PlayerEventConsumerVerticle_Thread[pool-24-thread-3,5,main], groupId=player-opt-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=1193286497, epoch=1236) to node 3: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:16,673 INFO [pool-24-thread-3] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_PlayerEventConsumerVerticle_Thread[pool-24-thread-3,5,main], groupId=player-opt-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=1606122808, epoch=1235) to node 1: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:16,673 INFO [pool-24-thread-2] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_PlayerEventConsumerVerticle_Thread[pool-24-thread-2,5,main], groupId=player-opt-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=908588100, epoch=1238) to node 2: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:16,674 INFO [kafka-coordinator-heartbeat-thread | player-opt-consumer-group_73129f731eba/172.17.0.2] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_PlayerEventConsumerVerticle_Thread[pool-24-thread-2,5,main], groupId=player-opt-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=602069977, epoch=1236) to node 3: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:16,676 INFO [pool-24-thread-1] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_PlayerEventConsumerVerticle_Thread[pool-24-thread-1,5,main], groupId=player-opt-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=297221147, epoch=1235) to node 1: org.apache.kafka.common.errors.DisconnectException.
2020-01-29 14:33:16,677 INFO [kafka-coordinator-heartbeat-thread | player-opt-consumer-group_73129f731eba/172.17.0.2] org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=172.17.0.2_PlayerEventConsumerVerticle_Thread[pool-24-thread-1,5,main], groupId=player-opt-consumer-group_73129f731eba/172.17.0.2] Error sending fetch request (sessionId=1745857354, epoch=1239) to node 2: org.apache.kafka.common.errors.DisconnectException.
журналы выше происходят ровно через 10 минут после того, как потребитель работает (мы выяснили, что эти 10 минут могут быть определены в брокере с помощью этот параметр по умолчанию: connections.max.idle.ms = 600000)
-> это происходит только один раз через 10 минут после того, как потребитель подключен и кафка возвращается в нормальное состояние
в конечном итоге через 10 минут мы испытываем задержку в 3-4 секунды для всех наших сообщений в брокере, что в нашем случае очень плохо, поскольку мы очень чувствительны к проблемам с задержкой
любые отведения, которые могут вызвать эту проблему