Идея в Windows может успешно записывать данные в куду, но когда она будет упакована и запущена на сервере, она сообщит об ошибке. Информация об ошибке следующая:
ERROR Executor: Exception in task 0.0 in stage 1.0 (TID 3)
java.lang.IllegalArgumentException: cannot import authentication data from a different user: old='', new='hadoop'
at org.apache.kudu.client.SecurityContext.checkUserMatches(SecurityContext.java:293)
at org.apache.kudu.client.SecurityContext.importAuthenticationCredentials(SecurityContext.java:310)
at org.apache.kudu.client.AsyncKuduClient.importAuthenticationCredentials(AsyncKuduClient.java:827)
at org.apache.kudu.spark.kudu.KuduContext.asyncClient$lzycompute(KuduContext.scala:104)
at org.apache.kudu.spark.kudu.KuduContext.asyncClient(KuduContext.scala:101)
at org.apache.kudu.spark.kudu.KuduContext.syncClient$lzycompute(KuduContext.scala:99)
at org.apache.kudu.spark.kudu.KuduContext.syncClient(KuduContext.scala:99)
at org.apache.kudu.spark.kudu.KuduContext.org$apache$kudu$spark$kudu$KuduContext$$writePartitionRows(KuduContext.scala:308)
at org.apache.kudu.spark.kudu.KuduContext$$anonfun$writeRows$1.apply(KuduContext.scala:290)
at org.apache.kudu.spark.kudu.KuduContext$$anonfun$writeRows$1.apply(KuduContext.scala:289)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:935)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:935)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2101)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2101)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:121)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/05/25 08:24:20 INFO TaskSetManager: Starting task 2.0 in stage 1.0 (TID 4, localhost, executor driver, partition 2, PROCESS_LOCAL, 7767 bytes)
20/05/25 08:24:20 INFO Executor: Running task 2.0 in stage 1.0 (TID 4)
20/05/25 08:24:20 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks including 0 local blocks and 0 remote blocks
20/05/25 08:24:20 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
20/05/25 08:24:20 WARN TaskSetManager: Lost task 0.0 in stage 1.0 (TID 3, localhost, executor driver): java.lang.IllegalArgumentException: cannot import authentication data from a different user: old='', new='hadoop'
at org.apache.kudu.client.SecurityContext.checkUserMatches(SecurityContext.java:293)
at org.apache.kudu.client.SecurityContext.importAuthenticationCredentials(SecurityContext.java:310)
at org.apache.kudu.client.AsyncKuduClient.importAuthenticationCredentials(AsyncKuduClient.java:827)
at org.apache.kudu.spark.kudu.KuduContext.asyncClient$lzycompute(KuduContext.scala:104)
at org.apache.kudu.spark.kudu.KuduContext.asyncClient(KuduContext.scala:101)
at org.apache.kudu.spark.kudu.KuduContext.syncClient$lzycompute(KuduContext.scala:99)
at org.apache.kudu.spark.kudu.KuduContext.syncClient(KuduContext.scala:99)
at org.apache.kudu.spark.kudu.KuduContext.org$apache$kudu$spark$kudu$KuduContext$$writePartitionRows(KuduContext.scala:308)
at org.apache.kudu.spark.kudu.KuduContext$$anonfun$writeRows$1.apply(KuduContext.scala:290)
at org.apache.kudu.spark.kudu.KuduContext$$anonfun$writeRows$1.apply(KuduContext.scala:289)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:935)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:935)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2101)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2101)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:121)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/05/25 08:24:20 ERROR TaskSetManager: Task 0 in stage 1.0 failed 1 times; aborting job
20/05/25 08:24:20 INFO TaskSchedulerImpl: Cancelling stage 1
20/05/25 08:24:20 INFO TaskSchedulerImpl: Killing all running tasks in stage 1: Stage cancelled
20/05/25 08:24:20 INFO TaskSchedulerImpl: Stage 1 was cancelled
20/05/25 08:24:20 ERROR Executor: Exception in task 2.0 in stage 1.0 (TID 4)
Информация о файле конфигурации kudu следующая:
master.gflagfile
# Do not modify these two lines. If you wish to change these variables,
# modify them in /etc/default/kudu-master.
--fromenv=rpc_bind_addresses
--fromenv=log_dir
--fs_wal_dir=/data/kudu/kudu_master_data
--fs_data_dirs=/data/kudu/kudu_master_data
--rpc_encryption=disabled
--rpc_authentication=disabled
--trusted_subnets=0.0.0.0/0
tserver.gflagfile
# Do not modify these two lines. If you wish to change these variables,
# modify them in /etc/default/kudu-tserver.
--fromenv=rpc_bind_addresses
--fromenv=log_dir
--fs_wal_dir=/data/kudu/kudu_tserver_data
--fs_data_dirs=/data/kudu/kudu_tserver_data
--tserver_master_addrs=hadoop000:7051
--rpc_encryption=disabled
--rpc_authentication=disabled
--trusted_subnets=0.0.0.0/0
--unlock_unsafe_flags=true
--allow_unsafe_replication_factor=true
--default_num_replicas=1
--rpc_negotiation_timeout_ms=9000
Я kudu newb ie, этот вопрос застрял у меня на весь день, мне нужна ваша помощь, спасибо.