Искробезопасный абонент Dstream для брокера - PullRequest
Новая
       43

Искробезопасный абонент Dstream для брокера

0 голосов
/ 03 сентября 2018

Я написал приложение для потокового воспроизведения, которое использует данные, хранящиеся в теме pubsub (pull subscriber). во время выполнения искровой работы я получаю следующую информацию журнала: 

   18/09/02 18:57:19 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1886] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913504000 ms
18/09/02 18:57:32 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1895] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913516000 ms
18/09/02 18:57:50 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1898] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913528000 ms
18/09/02 18:58:07 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1913] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913540000 ms
18/09/02 18:58:10 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1916] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913552000 ms
18/09/02 18:58:22 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1925] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913564000 ms
18/09/02 18:58:39 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1928] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913576000 ms
18/09/02 18:58:51 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1937] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913588000 ms
18/09/02 18:59:03 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1946] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913600000 ms
18/09/02 18:59:21 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1955] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913612000 ms
18/09/02 18:59:39 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1960] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913624000 ms
18/09/02 18:59:52 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1967] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913636000 ms
18/09/02 19:00:26 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1976] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 1535913648000 ms
18/09/02 19:00:45 INFO com.renat.det.pubsub.connector.PubsubInputDStream: Removing blocks of RDD WriteAheadLogBackedBlockRDD[1979] at ReceiverInputDStream at PubsubInputDStream.scala:27 of time 153

Задание успешно выполнено, и я получаю ожидаемый результат. Я просто хочу понять значение этой информации журнала.

...