У меня проблемы с запуском задания mapreduce с моим oop и пряжей, ниже приведен журнал выполнения.
Журнал выполнения этой команды
hadoop jar /opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.2.1.jar grep /user/hadoop output2 'dfs[a-z.]+'
2020-04-19 01:37:36,960 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
2020-04-19 01:37:37,864 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/hadoop/.staging/job_1587269102661_0005
2020-04-19 01:37:38,136 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2020-04-19 01:37:38,299 INFO input.FileInputFormat: Total input files to process : 10
2020-04-19 01:37:38,368 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2020-04-19 01:37:38,431 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2020-04-19 01:37:38,446 INFO mapreduce.JobSubmitter: number of splits:10
2020-04-19 01:37:38,635 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2020-04-19 01:37:38,656 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1587269102661_0005
2020-04-19 01:37:38,657 INFO mapreduce.JobSubmitter: Executing with tokens: []
2020-04-19 01:37:38,920 INFO conf.Configuration: resource-types.xml not found
2020-04-19 01:37:38,920 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
2020-04-19 01:37:39,001 INFO impl.YarnClientImpl: Submitted application application_1587269102661_0005
2020-04-19 01:37:39,064 INFO mapreduce.Job: The url to track the job: http://lucas-Inspiron-N4050:8088/proxy/application_1587269102661_0005/
2020-04-19 01:37:39,065 INFO mapreduce.Job: Running job: job_1587269102661_0005
2020-04-19 01:37:42,105 INFO mapreduce.Job: Job job_1587269102661_0005 running in uber mode : false
2020-04-19 01:37:42,106 INFO mapreduce.Job: map 0% reduce 0%
2020-04-19 01:37:42,124 INFO mapreduce.Job: Job job_1587269102661_0005 failed with state FAILED due to: Application application_1587269102661_0005 failed 2 times due to AM Container for appattempt_1587269102661_0005_000002 exited with exitCode: 127
Failing this attempt.Diagnostics: [2020-04-19 01:37:41.103]Exception from container-launch.
Container id: container_1587269102661_0005_02_000001
Exit code: 127
[2020-04-19 01:37:41.106]Container exited with a non-zero exit code 127. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
/bin/bash: /bin/java: No such file or directory
[2020-04-19 01:37:41.106]Container exited with a non-zero exit code 127. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
/bin/bash: /bin/java: No such file or directory
For more detailed output, check the application tracking page: http://lucas-Inspiron-N4050:8088/cluster/app/application_1587269102661_0005 Then click on links to logs of each attempt.
. Failing the application.
2020-04-19 01:37:42,150 INFO mapreduce.Job: Counters: 0
2020-04-19 01:37:42,184 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
2020-04-19 01:37:42,229 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/hadoop/.staging/job_1587269102661_0006
2020-04-19 01:37:42,286 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2020-04-19 01:37:42,336 INFO mapreduce.JobSubmitter: Cleaning up the staging area /tmp/hadoop-yarn/staging/hadoop/.staging/job_1587269102661_0006
org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://localhost:9000/user/hadoop/grep-temp-280692674
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:332)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:274)
at org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:59)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:396)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:310)
at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:327)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:200)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1570)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1567)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1567)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1588)
at org.apache.hadoop.examples.Grep.run(Grep.java:94)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.examples.Grep.main(Grep.java:103)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
my переменные окружения
export JAVA_HOME=/opt/jdkhadoop
export PATH=$PATH:$JAVA_HOME/bin
export HADOOP_HOME=/opt/hadoop
export HADOOP_INSTALL=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
Я использую имел oop 3.2.1, кто-то может мне помочь?