集群搭建与启动
1.克隆虚拟机
目录
1.克隆虚拟机
关闭master主机，在VM软件左侧系统资源中单击master,选择“管理”列表下的克隆选项，如图1-1所示。在弹出克隆虚拟机向导界面上，点击下一步，在选中"虚拟机中当前状态"下点击下一页，选择克隆类型。完整克隆的虚拟机文件相对独立并且安全，采用此处完整克隆方式。对主机命名为slave1,并确定存放地址（例：D:\CentOS\slave1,与master主机放在同一个文件下，方便管理），进度条完成说明克隆成功。
图1-1 虚拟机克隆选项向导
开启克隆的slave1主机，登录后，会发现主机名、ip地址与master主机一致，为了避免IP冲突，需要更改slave1主机名和静态IP，可参考之前设置master主机名和静态IP地址步骤。之前映射master主机名和IP地址时，已确定slave1主机的ip地址为192.168.38.129，因此此次设置应保持一致。
[zhao@master ~]$ su - root
Password:
Last login: Wed Sep 29 19:57:28 CST 2021 from 192.168.38.1 on pts/2
[root@master ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:96:8a:db brd ff:ff:ff:ff:ff:ff
inet 192.168.38.128/24 brd 192.168.38.255 scope global ens33
valid_lft forever preferred_lft forever
inet6 fe80::268d:bf52:8b8f:b5be/64 scope link
valid_lft forever preferred_lft forever
inet6 fe80::d584:59a1:6240:d215/64 scope link tentative dadfailed
valid_lft forever preferred_lft forever
3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN qlen 1000
link/ether 52:54:00:ca:34:4d brd ff:ff:ff:ff:ff:ff
inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
valid_lft forever preferred_lft forever
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN qlen 1000
link/ether 52:54:00:ca:34:4d brd ff:ff:ff:ff:ff:ff
[root@master ~]# hostnamectl set-hostname slave1
[root@master ~]# bash
[root@slave1 ~]# vi /etc/sysconfig/network-scripts/ifcfg-ens33
[root@slave1 ~]# service network restart
Restarting network (via systemctl): [ OK ]
[root@slave1 ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:27:31:d8 brd ff:ff:ff:ff:ff:ff
inet 192.168.38.129/24 brd 192.168.38.255 scope global ens33
valid_lft forever preferred_lft forever
inet6 fe80::d584:59a1:6240:d215/64 scope link
valid_lft forever preferred_lft forever
3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN qlen 1000
link/ether 52:54:00:ca:34:4d brd ff:ff:ff:ff:ff:ff
inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
valid_lft forever preferred_lft forever
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN qlen 1000
link/ether 52:54:00:ca:34:4d brd ff:ff:ff:ff:ff:ff
[root@slave1 ~]#
若以上步骤是利用远程连接软件操作，网络服务重新启动后，连接会中断，是因为slave1主机IP已经改为192.168.38.129，而之前连接是通过192.168.38.128IP连接的。
重复以上步骤，可克隆slave2主机，该主机的静态IP为192.168.38.130。
2.设置免密登录
下面以master主机为例，让主机master免密进入master、slave1和slave2主机。确保三台主机都开启，并利用远程连接软件连接master主机。
输入ssh-keygen -t rsa命令，并按四次回车，/root目录下生成密钥文件。执行命令ssh-copy-id slave1，将生成的公钥复制到slave1，实现master主机到slave1主机的单向免密登录。
[root@master ~]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:OAaMu3EAxiiEnbuepVUHdde1tW5QV9W6RtL9B/oI+qI root@master
The key's randomart image is:
+---[RSA 2048]----+
|*+ . .. . .. oX|
|=ooo . . . o *|
|. o.o . o = |
| .o .... . B .|
| o...+.S + =.|
| .+o. . . . + o|
| ..= . . + .|
| + o . . |
| E. o. |
+----[SHA256]-----+
[root@master ~]# ssh-copy-id slave1
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host 'slave1 (192.168.38.129)' can't be established.
ECDSA key fingerprint is SHA256:CYMn0hmNUEip41Lu/4wLlpu0HI/CTfEWInMellcOKk4.
ECDSA key fingerprint is MD5:6c:fe:5c:9b:0e:bb:9f:85:dd:0f:3c:27:0a:e2:05:8b.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@slave1's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'slave1'"
and check to make sure that only the key(s) you wanted were added.
[root@master ~]# ssh slave1
Last login: Thu Sep 30 19:44:40 2021 from 192.168.38.1
[root@slave1 ~]#
实现master主机到slave2以及master主机的单向免密登录，执行命令ssh-copy-id slave2和ssh-copy-id master即可。注意以上命令都需要在mstart主机上。
[root@slave1 ~]# ssh master
[root@master ~]# ssh-copy-id slave2
[root@master ~]# ssh-copy-id master主机slave1和slave2主机实现免密登录，重复上述步骤即可。所以，要实现3台主机自由登录，需要执行ssh-copy-id 这条命令9次。
3.启动集群
如果集群是第一次启动，需要格式化NameNode。
[root@master ~]# cd /export/servers/hadoop-2.6.4/
[root@master hadoop-2.6.4]# bin/hdfs namenode -format输入对应命令后，若出现图3-1所示标志，说明NameNode格式化成功。后续若出现节点集群不能正常启动时，需要重新格式化NameNode时，一定要先停止上次启动的所有Namenode和datanode进程，然后再删除tmp和log数据。
图3-1 格式化NameNode成功界面
整体启动hdfs和yarn。NameNode和ResourceManger不在同一台主机上，应在拥有NameNode节点上的master主机上启动hdfs，在拥有ResouceManager节点的slave2主机上启动yarn。
[root@master hadoop-2.6.4]# start-dfs.sh[root@slave2 ~]# start-yarn.sh输入jps命令，查看主机节点启动情况。
[root@master ~]# jps
2806 Jps
2490 NameNode[root@slave1 ~]# jps
3973 DataNode
4071 SecondaryNameNode
4264 NodeManager
4505 Jps[root@slave2 ~]# jps
4466 ResourceManager
4167 NodeManager
4042 DataNode
4767 Jps若需关闭集群，在各主机上输入相应关闭的命令。启动集群是先hdfs后yarn，关闭集群是先yarn后hdfs。
4. web端集群查看
在浏览器输入http://192.168.38.128:50070,查看hdfs信息，如图4-1所示。
图4-1 hdfs的web端界面
在浏览器输入http://192.168.38.130:8088,查看yarn信息，如图4-2所示。
图4-2 yarn的web端界面
在浏览器输入http://192.168.38.129:50090,查看SecondaryNameNode信息，如图4-3所示。
图4-3 SecondaryNameNode的web端界面
华为开发者空间，是为全球开发者打造的专属开发空间，汇聚了华为优质开发资源及工具，致力于让每一位开发者拥有一台云主机，基于华为根生态开发、创新。
更多推荐
4
0- 0
已为社区贡献2条内容
所有评论(0)