Ниже приведена простая вставка stmt, которую я использую в версии aws emr 5.28.1 с ульем 2.3.6-amzn-0 и oop 2.8.5. Я пробовал это изменение на сайте пряжи. xml auxService: mapreduce_shuffle не существует в улье по-прежнему возникает та же ошибка.
[hadoop@ip-172-31-23-115 conf]$ hive
which: no hbase in (/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/aws/bin)
Logging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
hive> insert into table t1 values ('001','MN');
WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Query ID = hadoop_20200616102723_2e6debad-42cb-4faf-b095-630772da6557
Total jobs = 3
Launching Job 1 out of 3
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_1592293006206_0003, Tracking URL = http://ip-172-31-23-115.us-east-2.compute.internal:20888/proxy/application_1592293006206_0003/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill job_1592293006206_0003
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2020-06-16 10:27:31,811 Stage-1 map = 0%, reduce = 0%
2020-06-16 10:27:40,086 Stage-1 map = 100%, reduce = 0%
Ended Job = job_1592293006206_0003 with errors
Error during job, obtaining debugging information...
Examining task ID: task_1592293006206_0003_m_000000 (and more) from job job_1592293006206_0003
Task with the most failures(4):
-----
Task ID:
task_1592293006206_0003_m_000000
URL:
http://ip-172-31-23-115.us-east-2.compute.internal:8088/taskdetails.jsp?jobid=job_1592293006206_0003&tipid=task_1592293006206_0003_m_000000
-----
Diagnostic Messages for this Task:
Container launch failed for container_1592293006206_0003_01_000005 : org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateExceptionImpl(SerializedExceptionPBImpl.java:171)
at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:182)
at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:162)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:408)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Stage-Stage-1: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
hive>