Я перешел с версии 1.9.1 на 1.10.0 и теперь моя программа не запускается. Я использую простую компьютерную топологию, представленную на рисунке ниже.
Я попытался настроить зависимости в моей программе и в папке flink dist, но безрезультатно. Я перенес oop зависимостей в plugins/s3
, но похоже, что Флинк не может найти банку. JobManager сообщает:
2020-02-14 18:52:02,376 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - --------------------------------------------------------------------------------
2020-02-14 18:52:02,376 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Starting TaskManager (Version: 1.10.0, Rev:aa4eb8f, Date:07.02.2020 @ 19:18:19 CET)
2020-02-14 18:52:02,376 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - OS current user: ubuntu
2020-02-14 18:52:02,376 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Current Hadoop/Kerberos user: <no hadoop dependency found>
2020-02-14 18:52:02,377 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - JVM: OpenJDK 64-Bit Server VM - Private Build - 1.8/25.242-b08
2020-02-14 18:52:02,377 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Maximum heap size: 4432 MiBytes
2020-02-14 18:52:02,377 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
2020-02-14 18:52:02,377 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - No Hadoop Dependency available
2020-02-14 18:52:02,377 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - JVM Options:
2020-02-14 18:52:02,377 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - -XX:+UseG1GC
2020-02-14 18:52:02,377 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - -Xmx4647288761
2020-02-14 18:52:02,377 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - -Xms4647288761
2020-02-14 18:52:02,377 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - -XX:MaxDirectMemorySize=1090519054
2020-02-14 18:52:02,377 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - -XX:MaxMetaspaceSize=100663296
2020-02-14 18:52:02,377 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - -Dlog.file=/home/ubuntu/flink-1.10.0/log/flink-ubuntu-taskexecutor-0-ip-10-0-1-68.log
2020-02-14 18:52:02,377 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - -Dlog4j.configuration=file:/home/ubuntu/flink-1.10.0/conf/log4j.properties
2020-02-14 18:52:02,378 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - -Dlogback.configurationFile=file:/home/ubuntu/flink-1.10.0/conf/logback.xml
2020-02-14 18:52:02,378 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Program Arguments:
2020-02-14 18:52:02,378 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - --configDir
2020-02-14 18:52:02,378 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - /home/ubuntu/flink-1.10.0/conf
2020-02-14 18:52:02,378 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - -D
2020-02-14 18:52:02,378 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - taskmanager.memory.framework.off-heap.size=134217728b
2020-02-14 18:52:02,378 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - -D
2020-02-14 18:52:02,378 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - taskmanager.memory.network.max=956301326b
2020-02-14 18:52:02,378 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - -D
2020-02-14 18:52:02,378 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - taskmanager.memory.network.min=956301326b
2020-02-14 18:52:02,378 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - -D
2020-02-14 18:52:02,378 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - taskmanager.memory.framework.heap.size=134217728b
2020-02-14 18:52:02,378 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - -D
2020-02-14 18:52:02,378 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - taskmanager.memory.managed.size=3825205305b
2020-02-14 18:52:02,379 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - -D
2020-02-14 18:52:02,379 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - taskmanager.cpu.cores=8.0
2020-02-14 18:52:02,379 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - -D
2020-02-14 18:52:02,379 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - taskmanager.memory.task.heap.size=4513071033b
2020-02-14 18:52:02,379 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - -D
2020-02-14 18:52:02,379 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - taskmanager.memory.task.off-heap.size=0b
2020-02-14 18:52:02,379 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Classpath: /home/ubuntu/flink-1.10.0/lib/flink-metrics-datadog-1.10.0.jar:/home/ubuntu/flink-1.10.0/lib/flink-metrics-statsd-1.10.0.jar:/home/ubuntu/flink-1.10.0/lib/flink-table-blink_2.11-1.10.0.jar:/home/ubuntu/flink-1.10.0/lib/flink-table_2.11-1.10.0.jar:/home/ubuntu/flink-1.10.0/lib/log4j-1.2.17.jar:/home/ubuntu/flink-1.10.0/lib/slf4j-log4j12-1.7.15.jar:/home/ubuntu/flink-1.10.0/lib/flink-dist_2.11-1.10.0.jar::/home/ubuntu/app/conf:
2020-02-14 18:52:02,379 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - --------------------------------------------------------------------------------
Я использую StreamingFileSink. Я получаю следующее исключение примерно через минуту после начала работы, когда начинаю работу:
2020-02-14 20:06:24
org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy
at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:110)
at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:76)
at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:192)
at org.apache.flink.runtime.scheduler.DefaultScheduler.maybeHandleTaskFailure(DefaultScheduler.java:186)
at org.apache.flink.runtime.scheduler.DefaultScheduler.updateTaskExecutionStateInternal(DefaultScheduler.java:180)
at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:484)
at org.apache.flink.runtime.scheduler.UpdateSchedulerNgOnInternalFailuresListener.notifyTaskFailure(UpdateSchedulerNgOnInternalFailuresListener.java:49)
at org.apache.flink.runtime.executiongraph.ExecutionGraph.notifySchedulerNgAboutInternalTaskFailure(ExecutionGraph.java:1703)
at org.apache.flink.runtime.executiongraph.Execution.processFail(Execution.java:1252)
at org.apache.flink.runtime.executiongraph.Execution.processFail(Execution.java:1220)
at org.apache.flink.runtime.executiongraph.Execution.fail(Execution.java:955)
at org.apache.flink.runtime.jobmaster.slotpool.SingleLogicalSlot.signalPayloadRelease(SingleLogicalSlot.java:173)
at org.apache.flink.runtime.jobmaster.slotpool.SingleLogicalSlot.release(SingleLogicalSlot.java:165)
at org.apache.flink.runtime.jobmaster.slotpool.SlotSharingManager$SingleTaskSlot.release(SlotSharingManager.java:732)
at org.apache.flink.runtime.jobmaster.slotpool.SlotSharingManager$MultiTaskSlot.release(SlotSharingManager.java:537)
at org.apache.flink.runtime.jobmaster.slotpool.AllocatedSlot.releasePayload(AllocatedSlot.java:149)
at org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl.releaseTaskManagerInternal(SlotPoolImpl.java:815)
at org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl.releaseTaskManager(SlotPoolImpl.java:774)
at org.apache.flink.runtime.jobmaster.JobMaster.disconnectTaskManager(JobMaster.java:429)
at org.apache.flink.runtime.jobmaster.JobMaster$TaskManagerHeartbeatListener.notifyHeartbeatTimeout(JobMaster.java:1147)
at org.apache.flink.runtime.heartbeat.HeartbeatMonitorImpl.run(HeartbeatMonitorImpl.java:109)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:397)
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:190)
at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74)
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:152)
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26)
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21)
at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
at akka.actor.Actor$class.aroundReceive(Actor.scala:517)
at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592)
at akka.actor.ActorCell.invoke(ActorCell.scala:561)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258)
at akka.dispatch.Mailbox.run(Mailbox.scala:225)
at akka.dispatch.Mailbox.exec(Mailbox.scala:235)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.util.concurrent.TimeoutException: Heartbeat of TaskManager with id 56a63a7493bc527e7af383dc008a4800 timed out.
... 26 more