我使用Java编写Map reduce Job.设置配置
Configuration configuration = new Configuration(); configuration.set("fs.defaultFS", "hdfs://127.0.0.1:9000"); configuration.set("mapreduce.job.tracker", "localhost:54311"); configuration.set("mapreduce.framework.name", "yarn"); configuration.set("yarn.resourcemanager.address", "localhost:8032");
使用不同的案例运行
案例1:"使用Hadoop和Yarn命令":成功完成工作
案例2:"使用Eclipse":成功做好工作
案例3:"删除所有configuration.set()后使用Java -jar":
Configuration configuration = new Configuration(); Run successful but not display Job status on Yarn (default port number 8088)
案例4:"使用Java -jar":错误
Find stack trace:Exception in thread "main" java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses. at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120) at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:82) at org.apache.hadoop.mapreduce.Cluster. (Cluster.java:75) at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1255) at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1251) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556) at org.apache.hadoop.mapreduce.Job.connect(Job.java:1250) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1279) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1303) at com.my.cache.run.MyTool.run(MyTool.java:38) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at com.my.main.Main.main(Main.java:45)
我请你告诉我如何使用"Java -jar"命令运行map-reduce作业,还能检查状态并登录Yarn(默认端口8088).
为什么需要:想要创建Web服务并提交map-reduce作业.(不使用Java运行时库来执行Yarn或Hadoop命令).