学习来源：     51cto: https://edu.51cto.com/sd/518e5
                  腾讯课堂: https://ke.qq.com/course/2738602

K8s集群环境搭建—基础环境配置（1）：https://blog.csdn.net/qq_26900081/article/details/109291999

K8s集群环境搭建—K8s安装（2）：https://blog.csdn.net/qq_26900081/article/details/109311033

K8s集群环境搭建—高可用组件安装Keepalived和HAProxy（3）：https://blog.csdn.net/qq_26900081/article/details/109331192

K8s集群环境搭建—K8s集群初始化（4）：https://blog.csdn.net/qq_26900081/article/details/109331192

K8s集群环境搭建—安装Metrics和Dashboard（5）：https://blog.csdn.net/qq_26900081/article/details/109337475

只需要在Master节点上安装（Master1、Master2、Master3）；

如果是公有云，可以直接购买阿里云的SLB，也可以使用F5等其它高可用方案。

一、安装Keepalived和HAProxy

yum install keepalived haproxy -y

二、配置HAProxy

所有Master节点的HAProxy配置相同：

mkdir /etc/haproxy

vi /etc/haproxy/haproxy.cfg

global
  maxconn  2000
  ulimit-n  16384
  log  127.0.0.1 local0 err
  stats timeout 30s

defaults
  log global
  mode  http
  option  httplog
  timeout connect 5000
  timeout client  50000
  timeout server  50000
  timeout http-request 15s
  timeout http-keep-alive 15s

frontend monitor-in
  bind *:33305
  mode http
  option httplog
  monitor-uri /monitor

listen stats
  bind    *:8006
  mode    http
  stats   enable
  stats   hide-version
  stats   uri       /stats
  stats   refresh   30s
  stats   realm     Haproxy\ Statistics
  stats   auth      admin:admin

frontend k8s-master
  bind 0.0.0.0:16443
  bind 127.0.0.1:16443
  mode tcp
  option tcplog
  tcp-request inspect-delay 5s
  default_backend k8s-master

backend k8s-master
  mode tcp
  option tcplog
  option tcp-check
  balance roundrobin
  default-server inter 10s downinter 5s rise 2 fall 2 slowstart 60s maxconn 250 maxqueue 256 weight 100
  server localhost.master1.131	192.168.70.131:6443  check
  server localhost.master2.132	192.168.70.132:6443  check
  server localhost.master3.133	192.168.70.133:6443  check

三、配置Keepalived

vi /etc/keepalived/keepalived.conf

注意：以下配置健康检查是关闭的，集群建立完成后再开启。

网卡名称、主机地址、主机名称、虚拟IP、虚拟路由ID 根据自己的环境修改

a、Master1

! Configuration File for keepalived
global_defs {
    router_id LVS_DEVEL
}
vrrp_script chk_apiserver {
    script "/etc/keepalived/check_apiserver.sh"
    interval 2
    weight -5
    fall 3  
    rise 2
}
vrrp_instance VI_1 {
    state MASTER
    interface ens33
    mcast_src_ip 192.168.70.131
    virtual_router_id 51
    priority 100
    advert_int 2
    authentication {
        auth_type PASS
        auth_pass K8SHA_KA_AUTH
    }
    virtual_ipaddress {
        192.168.70.200
    }
#    track_script {
#       chk_apiserver
#    }
}

b、Master2

! Configuration File for keepalived
global_defs {
    router_id LVS_DEVEL
}
vrrp_script chk_apiserver {
    script "/etc/keepalived/check_apiserver.sh"
    interval 2
    weight -5
    fall 3  
    rise 2
}
vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    mcast_src_ip 192.168.70.132
    virtual_router_id 51
    priority 101
    advert_int 2
    authentication {
        auth_type PASS
        auth_pass K8SHA_KA_AUTH
    }
    virtual_ipaddress {
        192.168.70.200
    }
#    track_script {
#       chk_apiserver
#    }
}

c、Master3

! Configuration File for keepalived
global_defs {
    router_id LVS_DEVEL
}
vrrp_script chk_apiserver {
    script "/etc/keepalived/check_apiserver.sh"
    interval 2
    weight -5
    fall 3  
    rise 2
}
vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    mcast_src_ip 192.168.70.133
    virtual_router_id 51
    priority 102
    advert_int 2
    authentication {
        auth_type PASS
        auth_pass K8SHA_KA_AUTH
    }
    virtual_ipaddress {
        192.168.70.200
    }
#    track_script {
#       chk_apiserver
#    }
}

监控检查脚本文件内容(Master1、Master2、Master3)：vi /etc/keepalived/check_apiserver.sh

#!/bin/bash

err=0
for k in $(seq 1 5)
do
    check_code=$(pgrep kube-apiserver)
    if [[ $check_code == "" ]]; then
        err=$(expr $err + 1)
        sleep 5
        continue
    else
        err=0
        break
    fi
done

if [[ $err != "0" ]]; then
    echo "systemctl stop keepalived"
    /usr/bin/systemctl stop keepalived
    exit 1
else
    exit 0
fi

启动haproxy和keepalived：

systemctl enable --now haproxy
systemctl enable --now keepalived

ip a查看虚拟IP是否成功生成，按照上面的配置，应该是在Master3会生成虚拟ip。