在前面搭建伪分布式hadoop运行环境的基础上&＃xff0c;搭建hadoop集群&＃xff0c;配置如下&＃xff1a;
| 机器名 | 机器ip | 用途 |
| hadoop0&＃xff08;主节点&＃xff09; | 192.168.255.128 | NameNode//JobTracker |
| hadoop1&＃xff08;从节点&＃xff09; | 192.168.255.130 | DataNode/JobTracker/SeconderyNameNode |
| hadoop2&＃xff08;从节点&＃xff09; | 192.168.255.131 | DataNode/JobTracker |
1、拷贝虚拟机&＃xff1a;
将前面的伪分布的虚拟机拷贝三份&＃xff1b;
2、修改网络配置&＃xff1a;
&＃xff08;1&＃xff09;分别修改每台机器的 /etc/sysconfig/network 文件&＃xff0c;机器名分别改为hadoop0&＃xff0c;hadoop1&＃xff0c;hadoop2&＃xff1b;
&＃xff08;2&＃xff09;修改每台机器的 /etc/hosts 文件&＃xff0c;填写如下内容&＃xff1a;
&＃xff08;3&＃xff09;检测防火墙是否关闭&＃xff0c;命令&＃xff1a;service iptables status
&＃xff08;4&＃xff09;分别修改每台机器的hadoop配置文件中的core-site.xml和mapred-site.xml文件&＃xff0c;确保主节点名正确&＃xff1a;
3、设置三台机器之间SSH免密码登录&＃xff1a;
&＃xff08;1&＃xff09;分别清除三台机器上面原有的SSH秘钥信息&＃xff0c;同时生成新的秘钥&＃xff0c;分别在三台机器上执行如下命令&＃xff1a;
cd root/.sshrm –rf *ssh-keygen -t dsacat id_dsa.pub >> authorized_keys
&＃xff08;2&＃xff09;此时&＃xff0c;三台机器上分别可以免密码登录自己&＃xff0c;为了使三台机器可以相互之间免密码登录&＃xff0c;先将hadoop0和hadoop1的公钥分别按照到hadoop2上
在hadoop0上执行&＃xff1a;ssh-copy-id -i /root/.ssh/id_dsa.pub hadoop2
在hadoop1上执行&＃xff1a;ssh-copy-id -i /root/.ssh/id_dsa.pub hadoop2
之后查看hadoop2的authorized_keys文件&＃xff0c;里面已有hadoop0和hadoop1的公钥&＃xff1a;
&＃xff08;3&＃xff09;将hadoop2上的authorized_keys文件复制到hadoop0和hadoop1中&＃xff0c;在hadoop2中执行如下命令&＃xff1a;
scp /root/.ssh/authorized_keys hadoop0:/root/.ssh/
scp /root/.ssh/authorized_keys hadoop1:/root/.ssh/
至此&＃xff0c;hadoop0、hadoop1、hadoop2可以通过SSH免密码相互登录&＃xff0c;测试&＃xff1a;在hadoop0上输入ssh hadoop1和ssh hadoop2&＃xff0c;另外两台机器测试也是如此&＃xff1b;
4、在hadoop0&＃xff08;主节点&＃xff09;点上修改hadoop配置文件&＃xff1a;
&＃xff08;1&＃xff09;NameNode在主节点&＃xff0c;默认SecondaryNameNode也是在主节点&＃xff0c;这里通过修改/usr/local/hadoop/conf/masters文件&＃xff0c;将SecondaryNameNode放在hadoop1上&＃xff0c;如下图&＃xff1a;
&＃xff08;2&＃xff09;修改/usr/local/hadoop/conf/slaves文件&＃xff0c;设置从节点&＃xff0c;如下图&＃xff1a;
5、启动/停止hadoop集群&＃xff1b;
&＃xff08;1&＃xff09;因为虚拟机是拷贝自原来的伪分布环境&＃xff0c;所以分别删除三台机器的hadoop目录下的logs和tmp目录&＃xff1b;
&＃xff08;2&＃xff09;在hadoop0上格式化nameNode&＃xff0c;命令&＃xff1a;hadoop namenode –format&＃xff1b;
&＃xff08;3&＃xff09;启动&＃xff0c;在hadoop0上start-all.sh;
&＃xff08;4&＃xff09;终止服务命令&＃xff0c;在hadoop0上stop-all.sh;
6、启动之后验证相应服务&＃xff1a;
hadoop0上&＃xff1a;
hadoop1上
hadoop2上&＃xff1a;
至此&＃xff0c;说明hadoop集群环境安装及配置成功。
关于前台的处理调用&＃xff0c;都没有发生变化。
来自为知笔记(Wiz)
京公网安备 11010802041100号