Apache Giraph - 无法在拆分主/工作模式下运行,因为一次只能执行1个任务
蕊蕊宝宝妈妈_534 发布于 2022-12-18 15:38我在这里使用PageRank Benchmark示例运行了带有hadoop 2.2.0的Giraph 1.0.0 .
突然间我得到了这个错误结果:
线程"main"中的异常java.lang.IllegalArgumentException:checkLocalJobRunnerConfiguration:使用LocalJobRunner时,必须只有一个worker,因为一次只能有一个任务!在org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration(GiraphJob.java:151)在org.apache.giraph.job.GiraphJob.run(GiraphJob.java:225)在org.apache.giraph.benchmark.GiraphBenchmark.run (GiraphBenchmark.java:90)在org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)在org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)在org.apache. giraph.benchmark.PageRankBenchmark.main(PageRankBenchmark.java:71)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)at sun.reflect.DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43)在java.lang.reflect.Method.invoke(Method.java:606)在org.apache.hadoop.util.RunJar.main(RunJar.java:212)
当我将工人数量改为1时,我得到了:
线程"main"中的异常java.lang.IllegalArgumentException:checkLocalJobRunnerConfiguration:使用LocalJobRunner时,由于一次只有1个任务,因此无法在拆分主/工作模式下运行!在org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration(GiraphJob.java:157)在org.apache.giraph.job.GiraphJob.run(GiraphJob.java:225)在org.apache.giraph.benchmark.GiraphBenchmark.run (GiraphBenchmark.java:90)在org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)在org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)在org.apache. giraph.benchmark.PageRankBenchmark.main(PageRankBenchmark.java:71)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)at sun.reflect.DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43)在java.lang.reflect.Method.invoke(Method.java:606)在org.apache.hadoop.util.RunJar.main(RunJar.java:212)
有解决方案吗
-
嗨,我假设您没有在群集上运行？如果我在我们的演示VM中运行,我会得到同样的错误.
您可以在giraph-site.xml中禁用拆分主工作线程行为
giraph.SplitMasterWorker=false
如果您只想在一次性执行期间禁用此功能,您还可以将其作为命令行参数传递给您的程序.
-ca giraph.SplitMasterWorker=false
例如,我为我的大数据讲座运行了一个演示,如下所示:
#!/bin/bash yarn jar /root/giraph-0.0.1-SNAPSHOT-jar-with-dependencies.jar org.apache.giraph.GiraphRunner at.jku.tk.steinbauer.bigdata.giraph.MaxInDegreeComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/hue/graph/tinygraph.txt -of org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/hue/graph/degree -w 1 -ca giraph.SplitMasterWorker=false
2022-12-18 15:40 回答
书友31443126_163
京公网安备 11010802041100号