Жираф это успех, но нет выхода - PullRequest
0 голосов
/ 26 марта 2020

Привет, я пытаюсь использовать giraph.

Я использую Имел oop -1.2.1 & Giraph-1.1.0

Пример * wordCount * oop хорошо работает. Но когда я пытаюсь использовать пример giraph "SimpleShortestPathsComputaion", это кажется успешным, но я не могу найти выходной файл (используя hadoop dfs -ls /output)

причина, по которой я думал, что это кажется успех, когда я проверяю http://master01: 50030 (jobTracker) enter image description here

команда здесь

giraph

$GIRAPH_HOME/giraph-examples-1.1.0.jar org.apache.giraph.examples.SimpleShortestPathsComputation

-vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat

-vip /input/tiny_graph.txt

-vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat

-op /output

-w 2

Вот журналы

No HADOOP_CONF_DIR set, using /opt/apps/hadoop/hadoop-1.2.1/conf Warning: $HADOOP_HOME is deprecated.

20/03/26 21:14:38 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one.

20/03/26 21:14:38 INFO utils.ConfigurationUtils: No vertex output format specified. Ensure your OutputFormat does not require one.

20/03/26 21:14:38 INFO utils.ConfigurationUtils: No edge output format specified. Ensure your OutputFormat does not require one.

20/03/26 21:14:39 INFO job.GiraphJob: run: Since checkpointing is disabled (default), do not allow any task retries (setting mapred.map.max.attempts = 0, old value = 4)

20/03/26 21:14:41 WARN bsp.BspOutputFormat: checkOutputSpecs: ImmutableOutputCommiter will not check anything

20/03/26 21:14:41 INFO job.GiraphJob: Tracking URL: http://master01:50030/jobdetails.jsp?jobid=job_202003262056_0005

20/03/26 21:14:41 INFO job.GiraphJob: Waiting for resources... Job will start only when it gets all 2 mappers

20/03/26 21:15:12 INFO job.HaltApplicationUtils$DefaultHaltInstructionsWriter: writeHaltInstructions: To halt after next superstep execute: 'bin/halt-application --zkServer master01:22181 --zkNode /_hadoopBsp/job_202003262056_0005/_haltComputation'

20/03/26 21:15:12 INFO mapred.JobClient: Running job: job_202003262056_0005


20/03/26 21:15:13 INFO mapred.JobClient:  map 100% reduce 0%

20/03/26 21:15:18 INFO mapred.JobClient: Job complete: job_202003262056_0005

20/03/26 21:15:18 INFO mapred.JobClient: Counters: 40

20/03/26 21:15:18 INFO mapred.JobClient:   Map-Reduce Framework

20/03/26 21:15:18 INFO mapred.JobClient:     Spilled Records=0

20/03/26 21:15:18 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=3940524032

20/03/26 21:15:18 INFO mapred.JobClient:     Map input records=2

20/03/26 21:15:18 INFO mapred.JobClient:     SPLIT_RAW_BYTES=88

20/03/26 21:15:18 INFO mapred.JobClient:     Map output records=0

20/03/26 21:15:18 INFO mapred.JobClient:     Physical memory (bytes) snapshot=215613440

20/03/26 21:15:18 INFO mapred.JobClient:     CPU time spent (ms)=5050

20/03/26 21:15:18 INFO mapred.JobClient:     Total committed heap usage (bytes)=227147776

20/03/26 21:15:18 INFO mapred.JobClient:   Zookeeper halt node

20/03/26 21:15:18 INFO mapred.JobClient:     /_hadoopBsp/job_202003262056_0005/_haltComputation=0

20/03/26 21:15:18 INFO mapred.JobClient:   Zookeeper server:port

20/03/26 21:15:18 INFO mapred.JobClient:     master01:22181=0

20/03/26 21:15:18 INFO mapred.JobClient:   Giraph Timers

20/03/26 21:15:18 INFO mapred.JobClient:     Superstep 1 SimpleShortestPathsComputation (ms)=94

20/03/26 21:15:18 INFO mapred.JobClient:     Initialize (ms)=1223

20/03/26 21:15:18 INFO mapred.JobClient:     Superstep 0 SimpleShortestPathsComputation (ms)=90

20/03/26 21:15:18 INFO mapred.JobClient:     Shutdown (ms)=8741

20/03/26 21:15:18 INFO mapred.JobClient:     Setup (ms)=71

20/03/26 21:15:18 INFO mapred.JobClient:     Superstep 2 SimpleShortestPathsComputation (ms)=44

20/03/26 21:15:18 INFO mapred.JobClient:     Input superstep (ms)=369

20/03/26 21:15:18 INFO mapred.JobClient:     Total (ms)=9487

20/03/26 21:15:18 INFO mapred.JobClient:     Superstep 3 SimpleShortestPathsComputation (ms)=78

20/03/26 21:15:18 INFO mapred.JobClient:   File Input Format Counters

20/03/26 21:15:18 INFO mapred.JobClient:     Bytes Read=0

20/03/26 21:15:18 INFO mapred.JobClient:   Zookeeper base path

20/03/26 21:15:18 INFO mapred.JobClient:     /_hadoopBsp/job_202003262056_0005=0

20/03/26 21:15:18 INFO mapred.JobClient:   Giraph Stats

20/03/26 21:15:18 INFO mapred.JobClient:     Aggregate finished vertices=5

20/03/26 21:15:18 INFO mapred.JobClient:     Aggregate edges=12

20/03/26 21:15:18 INFO mapred.JobClient:     Sent messages=0

20/03/26 21:15:18 INFO mapred.JobClient:     Aggregate sent message message bytes=267

20/03/26 21:15:18 INFO mapred.JobClient:     Sent message bytes=0

20/03/26 21:15:18 INFO mapred.JobClient:     Current workers=1

20/03/26 21:15:18 INFO mapred.JobClient:     Last checkpointed superstep=0

20/03/26 21:15:18 INFO mapred.JobClient:     Current master task partition=0

20/03/26 21:15:18 INFO mapred.JobClient:     Superstep=4

20/03/26 21:15:18 INFO mapred.JobClient:     Aggregate sent messages=12

20/03/26 21:15:18 INFO mapred.JobClient:     Aggregate vertices=5

20/03/26 21:15:18 INFO mapred.JobClient:   FileSystemCounters

20/03/26 21:15:18 INFO mapred.JobClient:     HDFS_BYTES_READ=203

20/03/26 21:15:18 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=255698

20/03/26 21:15:18 INFO mapred.JobClient:   Job Counters

20/03/26 21:15:18 INFO mapred.JobClient:     Launched map tasks=2

20/03/26 21:15:18 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0

20/03/26 21:15:18 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0

20/03/26 21:15:18 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=47486

20/03/26 21:15:18 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0

20/03/26 21:15:18 INFO mapred.JobClient:   File Output Format Counters

20/03/26 21:15:18 INFO mapred.JobClient:     Bytes Written=0

openjdk версия "1.8.0_242"

...