Я пытаюсь создать интерактивный сеанс Spark с livy. И мне нужно добавить библиотеку, например банку, которую я использую в hdfs (см. Мой код). но сеанс мертв, а журнал ниже.
код:
client = LivyClient('http://sandbox.c4e.kyomei.fr:10500')
session = client.create_session(SessionKind.SPARK , jars = ['hdfs://sandbox-hdp.hortonworks.com:8020/tmp/tsa2-assembly-0.1.jar'] )
журнал:
> 20/05/09 01:43:48 INFO LineBufferedStream: Exception in thread "main"
> scala.reflect.internal.FatalError: object Predef does not have a
> member classOf 20/05/09 01:43:48 WARN RSCClient:
> Error stopping RPC.
> io.netty.util.concurrent.BlockingOperationException:
> DefaultChannelPromise@786d2cd8(uncancellable)
>
> at io.netty.util.concurrent.DefaultPromise.checkDeadLock(DefaultPromise.java:394)
>
> at io.netty.channel.DefaultChannelPromise.checkDeadLock(DefaultChannelPromise.java:157)
>
> at io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:230)
> ....
> Exception in thread "Thread-32" java.io.IOException: Stream closed
>
> at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:170)
>
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:283)
>
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> 20/05/09 01:43:48 WARN ContextLauncher: Child process exited with code
20/05/09 01:43:48 ERROR SparkProcApp: job was killed by user