Привет, я пытаюсь использовать giraph.
Я использую Имел oop -1.2.1 & Giraph-1.1.0
Пример * wordCount * oop хорошо работает. Но когда я пытаюсь использовать пример giraph "SimpleShortestPathsComputaion
", это кажется успешным, но я не могу найти выходной файл (используя hadoop dfs -ls /output
)
причина, по которой я думал, что это кажется успех, когда я проверяю http://master01: 50030 (jobTracker) ![enter image description here](https://i.stack.imgur.com/c9uml.png)
команда здесь
giraph
$GIRAPH_HOME/giraph-examples-1.1.0.jar org.apache.giraph.examples.SimpleShortestPathsComputation
-vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
-vip /input/tiny_graph.txt
-vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat
-op /output
-w 2
Вот журналы
No HADOOP_CONF_DIR set, using /opt/apps/hadoop/hadoop-1.2.1/conf Warning: $HADOOP_HOME is deprecated.
20/03/26 21:14:38 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one.
20/03/26 21:14:38 INFO utils.ConfigurationUtils: No vertex output format specified. Ensure your OutputFormat does not require one.
20/03/26 21:14:38 INFO utils.ConfigurationUtils: No edge output format specified. Ensure your OutputFormat does not require one.
20/03/26 21:14:39 INFO job.GiraphJob: run: Since checkpointing is disabled (default), do not allow any task retries (setting mapred.map.max.attempts = 0, old value = 4)
20/03/26 21:14:41 WARN bsp.BspOutputFormat: checkOutputSpecs: ImmutableOutputCommiter will not check anything
20/03/26 21:14:41 INFO job.GiraphJob: Tracking URL: http://master01:50030/jobdetails.jsp?jobid=job_202003262056_0005
20/03/26 21:14:41 INFO job.GiraphJob: Waiting for resources... Job will start only when it gets all 2 mappers
20/03/26 21:15:12 INFO job.HaltApplicationUtils$DefaultHaltInstructionsWriter: writeHaltInstructions: To halt after next superstep execute: 'bin/halt-application --zkServer master01:22181 --zkNode /_hadoopBsp/job_202003262056_0005/_haltComputation'
20/03/26 21:15:12 INFO mapred.JobClient: Running job: job_202003262056_0005
20/03/26 21:15:13 INFO mapred.JobClient: map 100% reduce 0%
20/03/26 21:15:18 INFO mapred.JobClient: Job complete: job_202003262056_0005
20/03/26 21:15:18 INFO mapred.JobClient: Counters: 40
20/03/26 21:15:18 INFO mapred.JobClient: Map-Reduce Framework
20/03/26 21:15:18 INFO mapred.JobClient: Spilled Records=0
20/03/26 21:15:18 INFO mapred.JobClient: Virtual memory (bytes) snapshot=3940524032
20/03/26 21:15:18 INFO mapred.JobClient: Map input records=2
20/03/26 21:15:18 INFO mapred.JobClient: SPLIT_RAW_BYTES=88
20/03/26 21:15:18 INFO mapred.JobClient: Map output records=0
20/03/26 21:15:18 INFO mapred.JobClient: Physical memory (bytes) snapshot=215613440
20/03/26 21:15:18 INFO mapred.JobClient: CPU time spent (ms)=5050
20/03/26 21:15:18 INFO mapred.JobClient: Total committed heap usage (bytes)=227147776
20/03/26 21:15:18 INFO mapred.JobClient: Zookeeper halt node
20/03/26 21:15:18 INFO mapred.JobClient: /_hadoopBsp/job_202003262056_0005/_haltComputation=0
20/03/26 21:15:18 INFO mapred.JobClient: Zookeeper server:port
20/03/26 21:15:18 INFO mapred.JobClient: master01:22181=0
20/03/26 21:15:18 INFO mapred.JobClient: Giraph Timers
20/03/26 21:15:18 INFO mapred.JobClient: Superstep 1 SimpleShortestPathsComputation (ms)=94
20/03/26 21:15:18 INFO mapred.JobClient: Initialize (ms)=1223
20/03/26 21:15:18 INFO mapred.JobClient: Superstep 0 SimpleShortestPathsComputation (ms)=90
20/03/26 21:15:18 INFO mapred.JobClient: Shutdown (ms)=8741
20/03/26 21:15:18 INFO mapred.JobClient: Setup (ms)=71
20/03/26 21:15:18 INFO mapred.JobClient: Superstep 2 SimpleShortestPathsComputation (ms)=44
20/03/26 21:15:18 INFO mapred.JobClient: Input superstep (ms)=369
20/03/26 21:15:18 INFO mapred.JobClient: Total (ms)=9487
20/03/26 21:15:18 INFO mapred.JobClient: Superstep 3 SimpleShortestPathsComputation (ms)=78
20/03/26 21:15:18 INFO mapred.JobClient: File Input Format Counters
20/03/26 21:15:18 INFO mapred.JobClient: Bytes Read=0
20/03/26 21:15:18 INFO mapred.JobClient: Zookeeper base path
20/03/26 21:15:18 INFO mapred.JobClient: /_hadoopBsp/job_202003262056_0005=0
20/03/26 21:15:18 INFO mapred.JobClient: Giraph Stats
20/03/26 21:15:18 INFO mapred.JobClient: Aggregate finished vertices=5
20/03/26 21:15:18 INFO mapred.JobClient: Aggregate edges=12
20/03/26 21:15:18 INFO mapred.JobClient: Sent messages=0
20/03/26 21:15:18 INFO mapred.JobClient: Aggregate sent message message bytes=267
20/03/26 21:15:18 INFO mapred.JobClient: Sent message bytes=0
20/03/26 21:15:18 INFO mapred.JobClient: Current workers=1
20/03/26 21:15:18 INFO mapred.JobClient: Last checkpointed superstep=0
20/03/26 21:15:18 INFO mapred.JobClient: Current master task partition=0
20/03/26 21:15:18 INFO mapred.JobClient: Superstep=4
20/03/26 21:15:18 INFO mapred.JobClient: Aggregate sent messages=12
20/03/26 21:15:18 INFO mapred.JobClient: Aggregate vertices=5
20/03/26 21:15:18 INFO mapred.JobClient: FileSystemCounters
20/03/26 21:15:18 INFO mapred.JobClient: HDFS_BYTES_READ=203
20/03/26 21:15:18 INFO mapred.JobClient: FILE_BYTES_WRITTEN=255698
20/03/26 21:15:18 INFO mapred.JobClient: Job Counters
20/03/26 21:15:18 INFO mapred.JobClient: Launched map tasks=2
20/03/26 21:15:18 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
20/03/26 21:15:18 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
20/03/26 21:15:18 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=47486
20/03/26 21:15:18 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
20/03/26 21:15:18 INFO mapred.JobClient: File Output Format Counters
20/03/26 21:15:18 INFO mapred.JobClient: Bytes Written=0
openjdk версия "1.8.0_242"