Docker实现 LVS + Keepalived + bind 二级负载均衡主从模式集群
Docker实现 LVS + Keepalived + bind 二级负载均衡主从模式集群1. 概述本文利用docker快速搭建一个由17个小节点构成的二层负载均衡集群。第一层负载均衡DNS采用Bind实现，第二层负载均衡由LVS实现，同时主从切换用Keepalived来实现。本文主要是验证二层负载均衡+主从切换的方案，由于虚拟机占用资源较大，故采用docker来验证。关于LVS，Keepaliv
·
Docker实现 LVS + Keepalived + bind 二级负载均衡主从模式集群
1. 概述
本文利用docker快速搭建一个由17个小节点构成的二层负载均衡集群。第一层负载均衡DNS采用Bind
实现，第二层负载均衡由LVS实现，同时主从切换用Keepalived来实现。
本文主要是验证二层负载均衡+主从切换的方案，由于虚拟机占用资源较大，故采用docker来验证。
关于LVS，Keepalived和bind的相关原理和各自的搭建步骤，可以参考其他的文档。
2. Docker环境安装配置
2.1 宿主机环境安装
yum install -y ipvsadm
ipvsadm
2.2 Docker环境安装配置
2.2.1 安装docker
yum install -y docker
2.1.2 配置Docker镜像的源
打开 /etc/docker
vim /etc/docker/daemon.json
将daemon.json改成如下
{
"registry-mirrors":["https://almtd3fa.mirror.aliyuncs.com"]
}
2.2.3 启动docker
systemctl start docker.service
systemctl enable docker.service # 设置开机启动
2.2.4 登录docker
注册了docker账号后可以直接登录
docker login
3. 搭建集群
3.1 集群节点信息
本集群总共由17个节点构成，具体的地址映射关系如下所示
| 节点的名称 | DIP | VIP |
|---|---|---|
| DNS-Server | 172.18.0.2 | 172.18.0.100 |
| ---- | ---- | ---- |
| DS-1-Master | 172.18.1.2 | 172.18.1.100 |
| DS-1-Slave | 172.18.1.3 | 172.18.1.100 |
| RS-1-Master | 172.18.1.4 | 172.18.1.101 |
| RS-1-Slave | 172.18.1.5 | 172.18.1.101 |
| RS-2-Master | 172.18.1.6 | 172.18.1.102 |
| RS-2-Slave | 172.18.1.7 | 172.18.1.102 |
| RS-3-Master | 172.18.1.8 | 172.18.1.103 |
| RS-3-Slave | 172.18.1.9 | 172.18.1.103 |
| ---- | ---- | ---- |
| DS-2-Master | 172.18.2.2 | 172.18.2.100 |
| DS-2-Slave | 172.18.2.3 | 172.18.2.100 |
| RS-4-Master | 172.18.2.4 | 172.18.2.101 |
| RS-4-Slave | 172.18.2.5 | 172.18.2.101 |
| RS-5-Master | 172.18.2.6 | 172.18.2.102 |
| RS-5-Slave | 172.18.2.7 | 172.18.2.102 |
| RS-6-Master | 172.18.2.8 | 172.18.2.103 |
| RS-6-Slave | 172.18.2.9 | 172.18.2.103 |
DS-1和DS-2分别作为两个小集群的网关，每个网关掌握着3个RS节点。
外面的客户端通过DNS-Server获取DS-1和DS-2的地址，并由DS-1和DS-2进行消息的路由和转发。
除了DNS-Server节点外，所有的节点均拥有镜像节点，作为从节点Slave随时顶替主节点master。
整体架构图如下所示
3.2 制作单个节点的镜像
docker创建自定义网络，这是由于docker设置固定的ip需要用户自己设置subnet，在此例中
172.18.0.0/16是集群的子网。lvs-keepalived-bind-Cluster是集群所用子网的名称。
docker network create --subnet=172.18.0.0/16 lvs-keepalived-bind-Cluster
节点都采用centos7作为基础的镜像，在此基础上不断加入需要的组件。
# 启动DNS-Server
docker run -itd \
--name DNS-Server \
--privileged=true \
--network lvs-keepalived-bind-Cluster \
--ip 172.18.0.2 \
--hostname DNS-Server \
centos:centos7 /usr/sbin/init
进入DNS-Server
docker exec -it DNS-Server /bin/bash
在DNS-Server上进行环境安装
# 必需组件
yum install -y net-tools # 安装ifconfig
yum install -y keepalived # 安装keepalived
yum install -y httpd # 安装apache服务
systemctl restart httpd # 启动apache服务
systemctl enable httpd # 开机启动apache服务
echo "I am DNS-Server -VIP: 172.18.0.2 -DIP: 172.18.0.100 " > /var/www/html/index.html
# 个性化设置
sed -i 's/PS1=.*$/PS1="\[\\\[\\e\[32;1m\\\]\\u\\\[\\e\[37;0m\\\]@\\h \\\[\\e\[36;1m\\\]\\w\\\[\\e\[0m\\\]\]\\\\$ \"/g' /etc/bashrc
提交本地docker镜像
# -a :提交的镜像作者；
# -c :使用Dockerfile指令来创建镜像；
# -m :提交时的说明文字；
# -p :在commit时，将容器暂停。
docker commit -a "Kotori" -m "cluster node for lvs + keepalived + bind project" DNS-Server xzsww123/cluster-node:v1.0
提交远端docker镜像
# docker push 注册用户名/镜像名
docker push xzsww123/cluster-node
3.3 启动其他节点
基于DNS-SERVER的镜像，建立并启动其他节点的容器
3.3.1 节点DS-1-Master的设置
############# 节点DS-1-Master的设置 #############
docker run -itd --name DS-1-Master --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.1.2 --hostname DS-1-Master xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it DS-1-Master /bin/bash
### 设置LVS-DS的相关
echo "I am DS-1-Master -VIP: 172.18.1.100 -DIP: 172.18.1.2" > /var/www/html/index.html
systemctl restart httpd
yum -y install ipvsadm
# 创建虚拟网卡和路由规则的开机脚本
echo "
ifconfig eth0:0 172.18.1.100 broadcast 172.18.1.100 netmask 255.255.255.255 up
route add -host 172.18.1.100 dev eth0
ipvsadm -C
ipvsadm -A -t 172.18.1.100:80 -s rr
ipvsadm -a -t 172.18.1.100:80 -r 172.18.1.101 -g
ipvsadm -a -t 172.18.1.100:80 -r 172.18.1.102 -g
ipvsadm -a -t 172.18.1.100:80 -r 172.18.1.103 -g
ipvsadm-save
" > /root/lvs-script-ds.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-ds.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local
### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id vip-100
}
vrrp_instance VI_1 {
state MASTER
interface eth0
mcast_src_ip 172.18.1.2
virtual_router_id 100
priority 200
advert_int 1
authentication {
auth_type PASS
auth_pass MrUse
}
virtual_ipaddress {
172.18.1.100
}
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload #重新加载配置
systemctl start keepalived.service #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of DS-1-Master #############
3.3.2 节点DS-1-Slave的设置
############# 节点DS-1-Slave的设置 #############
docker run -itd --name DS-1-Slave --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.1.3 --hostname DS-1-Slave xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it DS-1-Slave /bin/bash
### 设置LVS-DS的相关
echo "I am DS-1-Slave -VIP: 172.18.1.100 -DIP: 172.18.1.3" > /var/www/html/index.html
systemctl restart httpd
yum -y install ipvsadm
# 创建虚拟网卡和路由规则的开机脚本
echo "
ifconfig eth0:0 172.18.1.100 broadcast 172.18.1.100 netmask 255.255.255.255 up
route add -host 172.18.1.100 dev eth0:0
ipvsadm -C
ipvsadm -A -t 172.18.1.100:80 -s rr
ipvsadm -a -t 172.18.1.100:80 -r 172.18.1.101 -g
ipvsadm -a -t 172.18.1.100:80 -r 172.18.1.102 -g
ipvsadm -a -t 172.18.1.100:80 -r 172.18.1.103 -g
ipvsadm-save
" > /root/lvs-script-ds.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-ds.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local
### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id vip-100
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
mcast_src_ip 172.18.1.3
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass MrUse
}
virtual_ipaddress {
172.18.1.100
}
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload #重新加载配置
systemctl start keepalived.service #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of DS-1-Slave #############
3.3.3 节点RS-1-Master的设置
############# 节点RS-1-Master的设置 #############
docker run -itd --name RS-1-Master --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.1.4 --hostname RS-1-Master xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-1-Master /bin/bash
### 设置LVS-RS的相关
echo "I am RS-1-Master -VIP: 172.18.1.101 -DIP: 172.18.1.4" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.1.100 broadcast 172.18.1.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local
### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id vip-101
}
vrrp_instance VI_1 {
state MASTER
interface eth0
mcast_src_ip 172.18.1.4
virtual_router_id 101
priority 200
advert_int 1
authentication {
auth_type PASS
auth_pass MrUse
}
virtual_ipaddress {
172.18.1.101
}
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload #重新加载配置
systemctl start keepalived.service #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of RS-1-Master #############
3.3.4 节点RS-1-Slave的设置
############# 节点RS-1-Slave的设置 #############
docker run -itd --name RS-1-Slave --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.1.5 --hostname RS-1-Slave xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-1-Slave /bin/bash
### 设置LVS-RS的相关
echo "I am RS-1-Slave -VIP: 172.18.1.101 -DIP: 172.18.1.5" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.1.100 broadcast 172.18.1.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" > /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local
### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id vip-101
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
mcast_src_ip 172.18.1.5
virtual_router_id 101
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass MrUse
}
virtual_ipaddress {
172.18.1.101
}
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload #重新加载配置
systemctl start keepalived.service #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of RS-1-Slave #############
3.3.5 节点RS-2-Master的设置
############# 节点RS-2-Master的设置 #############
docker run -itd --name RS-2-Master --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.1.6 --hostname RS-2-Master xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-2-Master /bin/bash
### 设置LVS-RS的相关
echo "I am RS-2-Master -VIP: 172.18.1.102 -DIP: 172.18.1.6" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.1.100 broadcast 172.18.1.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local
### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id vip-102
}
vrrp_instance VI_1 {
state MASTER
interface eth0
mcast_src_ip 172.18.1.6
virtual_router_id 102
priority 200
advert_int 1
authentication {
auth_type PASS
auth_pass MrUse
}
virtual_ipaddress {
172.18.1.102
}
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload #重新加载配置
systemctl start keepalived.service #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of RS-2-Master #############
3.3.6 节点RS-2-Slave的设置
############# 节点RS-2-Slave的设置 #############
docker run -itd --name RS-2-Slave --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.1.7 --hostname RS-2-Slave xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-2-Slave /bin/bash
### 设置LVS-RS的相关
echo "I am RS-2-Slave -VIP: 172.18.1.102 -DIP: 172.18.1.7" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.1.100 broadcast 172.18.1.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local
### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id vip-102
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
mcast_src_ip 172.18.1.7
virtual_router_id 102
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass MrUse
}
virtual_ipaddress {
172.18.1.102
}
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload #重新加载配置
systemctl start keepalived.service #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of RS-2-Slave #############
3.3.7 节点RS-3-Master的设置
############# 节点RS-3-Master的设置 #############
docker run -itd --name RS-3-Master --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.1.8 --hostname RS-3-Master xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-3-Master /bin/bash
### 设置LVS-RS的相关
echo "I am RS-3-Master -VIP: 172.18.1.103 -DIP: 172.18.1.8" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.1.100 broadcast 172.18.1.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local
### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id vip-103
}
vrrp_instance VI_1 {
state MASTER
interface eth0
mcast_src_ip 172.18.1.8
virtual_router_id 103
priority 200
advert_int 1
authentication {
auth_type PASS
auth_pass MrUse
}
virtual_ipaddress {
172.18.1.103
}
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload #重新加载配置
systemctl start keepalived.service #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of RS-3-Master #############
3.3.8 节点RS-3-Slave的设置
############# 节点RS-3-Slave的设置 #############
docker run -itd --name RS-3-Slave --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.1.9 --hostname RS-3-Slave xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-3-Slave /bin/bash
### 设置LVS-RS的相关
echo "I am RS-3-Slave -VIP: 172.18.1.103 -DIP: 172.18.1.9" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.1.100 broadcast 172.18.1.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local
### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id vip-103
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
mcast_src_ip 172.18.1.9
virtual_router_id 103
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass MrUse
}
virtual_ipaddress {
172.18.1.103
}
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload #重新加载配置
systemctl start keepalived.service #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of RS-3-Slave #############
3.3.9 节点DS-2-Master的设置
############# 节点DS-2-Master的设置 #############
docker run -itd --name DS-2-Master --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.2.2 --hostname DS-2-Master xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it DS-2-Master /bin/bash
### 设置LVS-DS的相关
echo "I am DS-2-Master -VIP: 172.18.2.100 -DIP: 172.18.2.2" > /var/www/html/index.html
systemctl restart httpd
yum -y install ipvsadm
# 创建虚拟网卡和路由规则的开机脚本
echo "
ifconfig eth0:0 172.18.2.100 broadcast 172.18.2.100 netmask 255.255.255.255 up
route add -host 172.18.2.100 dev eth0
ipvsadm -C
ipvsadm -A -t 172.18.2.100:80 -s rr
ipvsadm -a -t 172.18.2.100:80 -r 172.18.2.101 -g
ipvsadm -a -t 172.18.2.100:80 -r 172.18.2.102 -g
ipvsadm -a -t 172.18.2.100:80 -r 172.18.2.103 -g
ipvsadm-save
" > /root/lvs-script-ds.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-ds.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local
### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id vip-100
}
vrrp_instance VI_1 {
state MASTER
interface eth0
mcast_src_ip 172.18.2.2
virtual_router_id 200
priority 200
advert_int 1
authentication {
auth_type PASS
auth_pass MrUse
}
virtual_ipaddress {
172.18.2.100
}
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload #重新加载配置
systemctl start keepalived.service #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of DS-2-Master #############
3.3.10 节点DS-2-Slave的设置
############# 节点DS-2-Slave的设置 #############
docker run -itd --name DS-2-Slave --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.2.3 --hostname DS-2-Slave xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it DS-2-Slave /bin/bash
### 设置LVS-DS的相关
echo "I am DS-2-Slave -VIP: 172.18.2.100 -DIP: 172.18.2.3" > /var/www/html/index.html
systemctl restart httpd
yum -y install ipvsadm
# 创建虚拟网卡和路由规则的开机脚本
echo "
ifconfig eth0:0 172.18.2.100 broadcast 172.18.2.100 netmask 255.255.255.255 up
route add -host 172.18.2.100 dev eth0:0
ipvsadm -C
ipvsadm -A -t 172.18.2.100:80 -s rr
ipvsadm -a -t 172.18.2.100:80 -r 172.18.2.101 -g
ipvsadm -a -t 172.18.2.100:80 -r 172.18.2.102 -g
ipvsadm -a -t 172.18.2.100:80 -r 172.18.2.103 -g
ipvsadm-save
" > /root/lvs-script-ds.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-ds.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local
### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id vip-100
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
mcast_src_ip 172.18.2.3
virtual_router_id 200
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass MrUse
}
virtual_ipaddress {
172.18.2.100
}
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload #重新加载配置
systemctl start keepalived.service #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of DS-2-Slave #############
3.3.11 节点RS-4-Master的设置
############# 节点RS-4-Master的设置 #############
docker run -itd --name RS-4-Master --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.2.4 --hostname RS-4-Master xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-4-Master /bin/bash
### 设置LVS-RS的相关
echo "I am RS-4-Master -VIP: 172.18.2.101 -DIP: 172.18.2.4" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.2.100 broadcast 172.18.2.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local
### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id vip-101
}
vrrp_instance VI_1 {
state MASTER
interface eth0
mcast_src_ip 172.18.2.4
virtual_router_id 201
priority 200
advert_int 1
authentication {
auth_type PASS
auth_pass MrUse
}
virtual_ipaddress {
172.18.2.101
}
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload #重新加载配置
systemctl start keepalived.service #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of RS-4-Master #############
3.3.12 节点RS-4-Slave的设置
############# 节点RS-1-Slave的设置 #############
docker run -itd --name RS-4-Slave --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.2.5 --hostname RS-4-Slave xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-4-Slave /bin/bash
### 设置LVS-RS的相关
echo "I am RS-4-Slave -VIP: 172.18.2.101 -DIP: 172.18.2.5" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.2.100 broadcast 172.18.1.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" > /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local
### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id vip-101
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
mcast_src_ip 172.18.2.5
virtual_router_id 201
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass MrUse
}
virtual_ipaddress {
172.18.2.101
}
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload #重新加载配置
systemctl start keepalived.service #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of RS-5-Slave #############
3.3.13 节点RS-5-Master的设置
############# 节点RS-2-Master的设置 #############
docker run -itd --name RS-5-Master --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.2.6 --hostname RS-5-Master xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-5-Master /bin/bash
### 设置LVS-RS的相关
echo "I am RS-5-Master -VIP: 172.18.2.102 -DIP: 172.18.2.6" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.2.100 broadcast 172.18.2.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local
### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id vip-102
}
vrrp_instance VI_1 {
state MASTER
interface eth0
mcast_src_ip 172.18.2.6
virtual_router_id 202
priority 200
advert_int 1
authentication {
auth_type PASS
auth_pass MrUse
}
virtual_ipaddress {
172.18.2.102
}
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload #重新加载配置
systemctl start keepalived.service #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of RS-5-Master #############
3.3.14 节点RS-5-Slave的设置
############# 节点RS-2-Slave的设置 #############
docker run -itd --name RS-5-Slave --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.2.7 --hostname RS-5-Slave xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-5-Slave /bin/bash
### 设置LVS-RS的相关
echo "I am RS-5-Slave -VIP: 172.18.2.102 -DIP: 172.18.2.7" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.2.100 broadcast 172.18.2.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local
### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id vip-102
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
mcast_src_ip 172.18.2.7
virtual_router_id 202
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass MrUse
}
virtual_ipaddress {
172.18.2.102
}
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload #重新加载配置
systemctl start keepalived.service #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of RS-5-Slave #############
3.3.15 节点RS-6-Master的设置
############# 节点RS-6-Master的设置 #############
docker run -itd --name RS-6-Master --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.2.8 --hostname RS-6-Master xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-6-Master /bin/bash
### 设置LVS-RS的相关
echo "I am RS-6-Master -VIP: 172.18.2.103 -DIP: 172.18.2.8" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.2.100 broadcast 172.18.2.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local
### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id vip-103
}
vrrp_instance VI_1 {
state MASTER
interface eth0
mcast_src_ip 172.18.2.8
virtual_router_id 203
priority 200
advert_int 1
authentication {
auth_type PASS
auth_pass MrUse
}
virtual_ipaddress {
172.18.2.103
}
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload #重新加载配置
systemctl start keepalived.service #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of RS-6-Master #############
3.3.16 节点RS-6-Slave的设置
############# 节点RS-6-Slave的设置 #############
docker run -itd --name RS-6-Slave --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.2.9 --hostname RS-6-Slave xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-6-Slave /bin/bash
### 设置LVS-RS的相关
echo "I am RS-6-Slave -VIP: 172.18.2.103 -DIP: 172.18.2.9" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.2.100 broadcast 172.18.2.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local
### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id vip-103
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
mcast_src_ip 172.18.2.9
virtual_router_id 203
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass MrUse
}
virtual_ipaddress {
172.18.2.103
}
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload #重新加载配置
systemctl start keepalived.service #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of RS-6-Slave #############
4. 创建DNS服务器容器
直接用已经生成的镜像创建DNS服务器的容器，如果已存在这个容器就先删除
docker run -itd --name DNS-Server --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.0.2 --hostname DNS-Server xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it DNS-Server /bin/bash
# 安装和启动bind
yum install -y bind bind-utils bind-devel bind-libs bind-chroot
systemctl start named
systemctl enable named
systemctl status named
# 修改域主服务器设置 /etc/named.conf
vi /etc/named.conf
# 改成如下
options {
listen-on port 53 { 172.18.0.2; };
listen-on-v6 port 53 { ::1; };
rrset-order { order random; };
directory "/var/named";
dump-file "/var/named/data/cache_dump.db";
statistics-file "/var/named/data/named_stats.txt";
memstatistics-file "/var/named/data/named_mem_stats.txt";
recursing-file "/var/named/data/named.recursing";
secroots-file "/var/named/data/named.secroots";
allow-query { any; };
recursion yes;
dnssec-enable yes;
dnssec-validation yes;
bindkeys-file "/etc/named.root.key"; #大大
managed-keys-directory "/var/named/dynamic";
pid-file "/run/named/named.pid";
session-keyfile "/run/named/session.key";
};
logging {
channel default_debug {
file "data/named.run";
severity dynamic;
};
};
zone "." IN {
type hint;
file "named.ca";
};
include "/etc/named.rfc1912.zones";
include "/etc/named.root.key";
# 修改区域配置文件设置 /etc/named.rfc1912.zones
vi /etc/named.rfc1912.zones
# 添加如下代码
zone "Odyssey.com" IN {
type master;
file "Odyssey.com.zone";
};
# 创建自己的zone文件
vi /var/named/Odyssey.com.zone
# 改成如下
$TTL 600
@ IN SOA Odyssey.com. root.qq.com(
1811201955
1H
1M
1W
10M)
@ IN NS ns1
ns1 IN A 172.18.0.2
@ IN A 172.18.1.100
IN A 172.18.2.100
www IN A 172.18.1.100
www IN A 172.18.2.100
* IN A 255.255.255.255
# 重新加载服务
rndc reload
5. 整体方案测试
以上集群搭建完毕，接下来进行简单的测试
先在宿主机上安装nscd，用来刷新DNS缓存
yum install -y nscd
先在宿主机上改变dns （最好断掉外网，因为外网上也有Odyssey.com这个域名）
echo "
DNS1=172.18.0.2
" >> /etc/sysconfig/network-scripts/ifcfg-ens33
service network restart
然后执行
host -t a www.Odyssey.com 172.18.0.2
得到如下结果
Using domain server:
Name: 172.18.0.2
Address: 172.18.0.2#53
Aliases:
www.Odyssey.com has address 172.18.1.100
www.Odyssey.com has address 172.18.2.100
然后直接访问www.Odyssey.com
curl www.Odyssey.com
可以看到访问请求轮流在RS间循环，但是DNS轮询机制会受到多方面的影响，如：A记录的TTL时间长
短的影响；别的 DNS 服务器 Cache 的影响；windows 客户端也有一个DNS Cache。这些都会影响
DNS轮询的效果。因此 DNS 的轮询机制并不能做为一个 load balancing的解决方案，只能作为一
个 load distribution 方案。因此执行上面的指令未必会切换DS服务器，这是根据每台服务器上的
DNS过期时间等都有关系。
附录
docker stop $(docker ps -a | grep xzsww123 | awk '{print $1 }') #停止所有已经退出的容器
docker rm $(docker ps -a | grep xzsww123 | awk '{print $1 }') #删除所有已经退出的容器
参考资料
[1] LVS负载均衡之DR模式部署
[2] keepalived 配置文件解析
[3] Linux运维实战之DNS（bind）服务器的安装与配置
[4] linux实现DNS轮询实现负载平衡
华为开发者空间，是为全球开发者打造的专属开发空间，汇聚了华为优质开发资源及工具，致力于让每一位开发者拥有一台云主机，基于华为根生态开发、创新。
更多推荐
0
0- 0
已为社区贡献1条内容
所有评论(0)