У проблемной установки были oop -2.9.1 и jdk1.8.0.111 - PullRequest
Новая
       1

У проблемной установки были oop -2.9.1 и jdk1.8.0.111

0 голосов
/ 04 февраля 2020

У меня установлено oop -2.9.1 и java 8 

 INFO:  map 0% reduce 0%  .. then Failed 2 times due to AM Container for appattempt_1580795821532_0006_000002 exited with  exitCode: 1 .... Exception message: '/tmp/hadoop-Zahedur' is not recognized as an internal or external command,
operable program or batch file.

C:\BigData\hadoop-2.9.1\sbin>hadoop jar C:\Users\sere\Desktop\zahedcon.jar zahedcon.WordCount /input_dir output_dir
Feb 04, 2020 2:03:39 PM org.apache.hadoop.yarn.client.RMProxy newProxyInstance
INFO: Connecting to ResourceManager at /0.0.0.0:8032
Feb 04, 2020 2:03:40 PM org.apache.hadoop.mapreduce.JobResourceUploader uploadResourcesInternal
WARNING: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
Feb 04, 2020 2:03:40 PM org.apache.hadoop.mapreduce.lib.input.FileInputFormat listStatus
INFO: Total input files to process : 1
Feb 04, 2020 2:03:41 PM org.apache.hadoop.mapreduce.JobSubmitter submitJobInternal
INFO: number of splits:1
20/02/04 14:03:41 INFO Configuration.deprecation: yarn.resourcemanager.system-metrics-publisher.enabled is deprecated. Instead, use yarn.system-metrics-publisher.enabled
Feb 04, 2020 2:03:41 PM org.apache.hadoop.mapreduce.JobSubmitter printTokens
INFO: Submitting tokens for job: job_1580795821532_0006
Feb 04, 2020 2:03:41 PM org.apache.hadoop.yarn.client.api.impl.YarnClientImpl submitApplication
INFO: Submitted application application_1580795821532_0006
Feb 04, 2020 2:03:41 PM org.apache.hadoop.mapreduce.Job submit
INFO: The url to track the job: http://ZAHED_ICE_RU:8088/proxy/application_1580795821532_0006/
Feb 04, 2020 2:03:41 PM org.apache.hadoop.mapreduce.Job monitorAndPrintJob
INFO: Running job: job_1580795821532_0006
Feb 04, 2020 2:03:50 PM org.apache.hadoop.mapreduce.Job monitorAndPrintJob
INFO: Job job_1580795821532_0006 running in uber mode : false
Feb 04, 2020 2:03:50 PM org.apache.hadoop.mapreduce.Job monitorAndPrintJob


INFO:  map 0% reduce 0%
Feb 04, 2020 2:03:51 PM org.apache.hadoop.mapreduce.Job monitorAndPrintJob
``    NFO: Job job_1580795821532_0006 failed with state FAILED due to: Application   ``application_1580795821532_0006 failed 2 times due to AM Container for appattempt_1580795821532_0006_000002 exited with  exitCode: 1
Failing this attempt.Diagnostics: [2020-02-04 14:03:48.712]Exception from container-launch.
Container id: container_1580795821532_0006_02_000001
Exit code: 1
Exception message: '/tmp/hadoop-Zahedur' is not recognized as an internal or external command,
operable program or batch file.


[2020-02-04 14:03:48.713]Container exited with a non-zero exit code 1.
[2020-02-04 14:03:48.713]Container exited with a non-zero exit code 1.
For more detailed output, check the application tracking page: http://ZAHED_ICE_RU:8088/cluster/app/application_1580795821532_0006 Then click on links to logs of each attempt.
. Failing the application.
Feb 04, 2020 2:03:51 PM org.apache.hadoop.mapreduce.Job monitorAndPrintJob
INFO: Counters: 0

Пожалуйста, помогите мне ... вот мои другие файлы ниже .. 

Mapred-site.xml 
      <configuration>
           <property>
                <name>mapreduce.framework.name</name>
       <value>yarn</value>
</property>

<property>
<name>mapreduce.jobhistory.address </name>
<value>localhost:10020</value>
</property>

<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>localhost:19888</value>
   </property>

</configuration>

yarn-site.xml
 <configuration>

<property>
         <name>yarn.log.server.url</name>
         <value>http://localhost:19888/jobhistory/logs</value>
</property>

<property>
    <name>yarn.server.resourcemanager.address</name>
    <value>localhost:8020</value>
  </property>

  <property>
    <name>yarn.server.resourcemanager.application.expiry.interval</name>
    <value>60000</value>
  </property>

  <property>
    <name>yarn.server.nodemanager.address</name>
    <value>0.0.0.0:45454</value>
  </property>

  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>

  <property>
    <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
  </property>

  <property>
    <name>yarn.server.nodemanager.remote-app-log-dir</name>
    <value>/app-logs</value>
  </property>

  <property>
    <name>yarn.nodemanager.log-dirs</name>
    <value>/C:/BigData/hadoop-2.9.1/logs/userlogs</value>
  </property>

  <property>
    <name>yarn.server.mapreduce-appmanager.attempt-listener.bindAddress</name>
    <value>0.0.0.0</value>
  </property>

  <property>
    <name>yarn.server.mapreduce-appmanager.client-service.bindAddress</name>
    <value>0.0.0.0</value>
  </property>

  <property>
    <name>yarn.log-aggregation-enable</name>
    <value>true</value>
  </property>

  <property>
    <name>yarn.log-aggregation.retain-seconds</name>
    <value>-1</value>
  </property>

  <property>
    <name>yarn.application.classpath</name>
    <value>%HADOOP_CONF_DIR%,%HADOOP_COMMON_HOME%/share/hadoop/common/*,%HADOOP_COMMON_HOME%/share/hadoop/common/lib/*,%HADOOP_HDFS_HOME%/share/hadoop/hdfs/*,%HADOOP_HDFS_HOME%/share/hadoop/hdfs/lib/*,%HADOOP_MAPRED_HOME%/share/hadoop/mapreduce/*,%HADOOP_MAPRED_HOME%/share/hadoop/mapreduce/lib/*,%HADOOP_YARN_HOME%/share/hadoop/yarn/*,%HADOOP_YARN_HOME%/share/hadoop/yarn/lib/*</value>
  </property>

</configuration>

core-site.xml

<configuration>

 <property>
     <name>fs.defaultFS</name>
     <value>hdfs://localhost:19000</value>
   </property> 
</configuration>`

hdfs-site.xml
`<property>
      <name>dfs.replication</name>
      <value>1</value>
   </property>
   <property>
      <name>dfs.namenode.name.dir</name>
      <value>/C:/BigData/hadoop-2.9.1/data/namenode</value>
   </property>
   <property>
      <name>dfs.datanode.data.dir</name>
      <value>/C:/BigData/hadoop-2.9.1/data/datanode</value>
   </property>

</configuration>
...