Твиттер искра потоковой передачи: слишком много попыток входа в систему - PullRequest
0 голосов
/ 01 сентября 2018

Я использую этот пример, предоставленный apache для потоковой передачи твитов. Однако я не могу получить какие-либо данные, потому что кажется, что потоковый API постоянно пытается подключиться к Twitter, вызывая таким образом:

2816 [Twitter Stream consumer /  [1][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
2927 [Twitter Stream consumer /  [2][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
2989 [Twitter Stream consumer /  [3][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3063 [Twitter Stream consumer /  [4][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3113 [Twitter Stream consumer /  [5][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3153 [Twitter Stream consumer /  [6][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3197 [Twitter Stream consumer /  [7][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3245 [Twitter Stream consumer /  [8][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3288 [Twitter Stream consumer /  [9][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3333 [Twitter Stream consumer /  [10][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3374 [Twitter Stream consumer /  [11][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.

Popular topics in last 60 seconds (0 total):
3428 [Twitter Stream consumer /  [12][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3510 [Twitter Stream consumer /  [13][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3563 [Twitter Stream consumer /  [14][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.

Popular topics in last 10 seconds (0 total):
3595 [Twitter Stream consumer /  [15][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3626 [Twitter Stream consumer /  [16][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3672 [Twitter Stream consumer /  [17][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3723 [Twitter Stream consumer /  [18][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3762 [Twitter Stream consumer /  [19][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3806 [Twitter Stream consumer /  [20][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3849 [Twitter Stream consumer /  [21][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3865 [Twitter Stream consumer /  [7][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Exceeded connection limit for user

3868 [Twitter Stream consumer /  [7][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - Waiting for 10000 milliseconds
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
        at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
3878 [Twitter Stream consumer /  [1][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Exceeded connection limit for user

3879 [Twitter Stream consumer /  [1][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - Waiting for 10000 milliseconds
3880 [Twitter Stream consumer /  [5][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Exceeded connection limit for user

3880 [Twitter Stream consumer /  [5][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - Waiting for 10000 milliseconds
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
        at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
        at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
3898 [Twitter Stream consumer /  [22][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3911 [Twitter Stream consumer /  [3][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Exceeded connection limit for user

3911 [Twitter Stream consumer /  [3][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - Waiting for 10000 milliseconds
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
        at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
3944 [Twitter Stream consumer /  [23][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
3985 [Twitter Stream consumer /  [24][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4036 [Twitter Stream consumer /  [25][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4086 [Twitter Stream consumer /  [26][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4127 [Twitter Stream consumer /  [27][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4168 [Twitter Stream consumer /  [28][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4208 [Twitter Stream consumer /  [29][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4256 [Twitter Stream consumer /  [30][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4347 [Twitter Stream consumer /  [31][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4377 [Twitter Stream consumer /  [4][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Easy there, Turbo. Too many requests recently. Enhance your calm.

4377 [Twitter Stream consumer /  [4][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - Waiting for 10000 milliseconds
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
        at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
4402 [Twitter Stream consumer /  [32][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4455 [Twitter Stream consumer /  [33][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4504 [Twitter Stream consumer /  [34][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4551 [Twitter Stream consumer /  [35][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4584 [Twitter Stream consumer /  [36][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.

Popular topics in last 60 seconds (0 total):
4621 [Twitter Stream consumer /  [37][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4664 [Twitter Stream consumer /  [38][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.

Popular topics in last 10 seconds (0 total):
4697 [Twitter Stream consumer /  [39][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4728 [Twitter Stream consumer /  [40][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4755 [Twitter Stream consumer /  [41][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4786 [Twitter Stream consumer /  [42][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4818 [Twitter Stream consumer /  [43][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4854 [Twitter Stream consumer /  [44][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4877 [Twitter Stream consumer /  [8][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Easy there, Turbo. Too many requests recently. Enhance your calm.

4877 [Twitter Stream consumer /  [8][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - Waiting for 10000 milliseconds
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
        at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
4891 [Twitter Stream consumer /  [45][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4924 [Twitter Stream consumer /  [46][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4955 [Twitter Stream consumer /  [47][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
4990 [Twitter Stream consumer /  [48][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5025 [Twitter Stream consumer /  [49][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5067 [Twitter Stream consumer /  [50][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5103 [Twitter Stream consumer /  [51][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5137 [Twitter Stream consumer /  [52][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5171 [Twitter Stream consumer /  [53][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5203 [Twitter Stream consumer /  [54][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5235 [Twitter Stream consumer /  [55][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5271 [Twitter Stream consumer /  [56][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5302 [Twitter Stream consumer /  [57][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5334 [Twitter Stream consumer /  [58][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5364 [Twitter Stream consumer /  [59][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5397 [Twitter Stream consumer /  [60][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5411 [Twitter Stream consumer /  [9][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Easy there, Turbo. Too many requests recently. Enhance your calm.

5411 [Twitter Stream consumer /  [9][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - Waiting for 10000 milliseconds
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
        at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
5429 [Twitter Stream consumer /  [61][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5461 [Twitter Stream consumer /  [62][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5492 [Twitter Stream consumer /  [63][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5521 [Twitter Stream consumer /  [64][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5553 [Twitter Stream consumer /  [65][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5582 [Twitter Stream consumer /  [66][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5616 [Twitter Stream consumer /  [67][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
5650 [Twitter Stream consumer /  [68][initializing]]INFO  twitter4j.TwitterStreamImpl  - Establishing connection.
^CException in thread "dispatcher-event-loop-6" java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
        at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.stop(ReceiverSupervisor.scala:137)
        at org.apache.spark.streaming.receiver.ReceiverSupervisorImpl$$anon$1$$anonfun$receive$1.applyOrElse(ReceiverSupervisorImpl.scala:82)
        at org.apache.spark.rpc.netty.Inbox$$anonfun$process$1.apply$mcV$sp(Inbox.scala:117)
        at org.apache.spark.rpc.netty.Inbox.safelyCall(Inbox.scala:205)
        at org.apache.spark.rpc.netty.Inbox.process(Inbox.scala:101)
        at org.apache.spark.rpc.netty.Dispatcher$MessageLoop.run(Dispatcher.scala:221)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
5907 [Twitter Stream consumer /  [10][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Easy there, Turbo. Too many requests recently. Enhance your calm.

5907 [Twitter Stream consumer /  [10][Establishing connection]]INFO  twitter4j.TwitterStreamImpl  - Waiting for 10000 milliseconds
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
        at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
        at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
        at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
        at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
        at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
        at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Кто-нибудь знает, почему он пытается установить несколько соединений? Это, кажется, коренная причина проблемы.

1 Ответ

0 голосов
/ 04 сентября 2018

Исключение вызвано несовместимостью между программными зависимостями. Похоже, что Apache Bahir прекратил разработку с тех пор, как их последний коммит был в 2017 году. Поэтому кажется, что единственный способ запустить Twitter в потоковом режиме - это либо понизить все до совместимых версий, либо использовать twitter4j для создания потока, а не искры. потоковые конструкции.

...