Linux下Hadoop集群安装详细步骤

1. 环境的需求(这里虚拟机下Centos6的安装就不说了)
          Centos6 + hadoop-0.21.0.tar

2. 服务器的配置(我在这里均为2个CPU,2G内存,100硬盘)
   在这里,服务器IP最好是固定的,也就是说,能相互之间用ping命令ping通的IP

   建议在公司开发,因为公司有这个条件,我在这里配置了三台datanode,分别为 Datanode1,Datanode2,Datanode3(这些都是创建虚机的主机名)

      服务器名称                 IP地址(自己定)

   Namenode                            192.168.16.1

   Datanode1                            192.168.16.2

   Datanode2                            192.168.16.3

   Datanode3                            192.168.16.4

   2.1 修改服务器IP的方法如下:
                            vi  /etc/sysconfig/network-scripts/ifcfg-eth0

                            DEVICE=\"eth0\"

                            #这里是你网卡的物理地址，通常检测到的网卡你就不用输入了

                            #打开后该项已经存在,无需修改

                            HWADDR=\"00:0C:29:95:1D:A5\"

                            BOOTPROTO=\"static\"

                            OnBOOT=\"yes\"

                            #这里是IP地址,不能重复,从253降序排列,被占用了的不能再次使用.

                            IPADDR=172.16.101.245

                            NETMASK=255.255.255.0

                            NETWORK=172.16.101.0

                            BROADCAST=172.16.101.255

                            GATEWAY=172.16.101.254

                            退出保存后,执行如下命令,使设置的网关马上生效:

                            shell>> ifdown eth0

                            shell>> ifup eth0

                            shell>> /etc/init.d/network restart

3. 安装JDK6(我用的是jdk-6u26-linux-x64-rpm.bin)
        3.1. 创建安装目录 mkdir /usr/java/
        3.2. 把jdk-6u26-linux-x64-rpm.bin移动到/usr/java/下然后执行
                          ./jdk-6u26-linux-x64-rpm.bin

                          运行中会要求输入，顺序yes和按回车就行.

                          执行后会看到一个文件夹名为:jdk1.6.0_26

        3.3. 设置环境变量
                            /etc/profile增加如下内容

                            #config java

                            JAVA_HOME=/usr/java/jdk1.6.0_26

                            CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar

                            PATH=$JAVA_HOME/bin:$HOME/bin:$PATH

                            export PATH JAVA_HOME CLASSPATH

        3.4. 让设置生效: source /etc/profile
4. 安装ssh服务器和客户端
        a. yum search ssh
        b. 找到要安装的server(这里拿openssh-server.x86_64)
        c. 安装server: yum install openssh-server.x86_64
        d. 安装client(这里拿openssh-clients.x86_64)
        e. 安装client: yum install openssh-clients.x86_64
5. 设置ssh进行Namenode和Datanode之间无密码访问
        a. 用 ssh-key-gen 在本地主机上创建公钥和密钥
                            [root@www.linuxidc.com ~]# ssh-keygen -t  rsa

                            Enter file in which to save the key (/home/jsmith/.ssh/id_rsa):[Enter key]

                            Enter passphrase (empty for no passphrase): [Press enter key]

                            Enter same passphrase again: [Pess enter key]

                            Your identification has been saved in /home/jsmith/.ssh/id_rsa.

                            Your public key has been saved in /home/jsmith/.ssh/id_rsa.pub.

                            The key fingerprint is: 33:b3:fe:af:95:95:18:11:31:d5:de:96:2f:f2:35:f9

                            root@www.linuxidc.com

        b. 用 ssh-copy-id 把公钥复制到远程主机上
                            [root@www.linuxidc.com ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub  root@Datanode1

                            root@Datanode1\'s password:

                            Now try logging into the machine, with ?ssh ?root@Datanode1‘‖, and check in: