k8s学习笔记5-部署和应用ingress-nginx-controller(v1.3.0)
Ingress公开从集群外部到集群内服务的HTTP和HTTPS路由。流量路由由Ingress资源上定义的规则控制。
·
k8s学习笔记5-部署和应用ingress-nginx-controller-1.3.0
一.介绍和原理
A.Ingress 是什么
Ingress 公开从集群外部到集群内服务的 HTTP 和 HTTPS 路由。 流量路由由 Ingress 资源上定义的规则控制。
下面是一个将所有流量都发送到同一 Service 的简单 Ingress 示例：
B.Ingress的组成
1.ingress：
ingress是一个API对象，通过yaml文件来配置，ingress对象的作用是定义请求如何转发到service的规则，可以理解为配置模板。
ingress通过http或https暴露集群内部service，给service提供外部URI、负载均衡、SSL/TLS能力以及基于域名的反向代理。ingress要依靠ingress-controiler来具体实现以上功能。
2.ingress-controller：
ingress-controller是具体实现反向代理及负载均衡的程序，对ingress定义的规则进行解析，根据配置的规则来实现请求转发。
ingress-controller并不是k8s自带的组件，实际上ingess-controller只是一个统称，用户可以选择不同的ingress-controller实现，目前，由k8s维护的ingress-controller只有google云的GCE与ingress-nginx两个，其他还有很多第三方维护的ingress-controller，具体可以参考官方文档。
但是不管哪一种ingress-controller，实现的机制都大同小异，只是在具体配置上有差异。
一般来说，ingress-controller的形式都是一个pod，里面跑着demon程序和反向代理程序。daemon负责不断监控集群的变化，根据ingress对象生成配置并应用新配置到反向代理，比如ingress-nginx就是动态生成nginx配置，动态更新upstreanm，并在需要的时候reload程序应用新配置。为了方便，后面的例子都以k8s官方维护的ingress-nginx为例。
3.Ingress的工作原理
1.ingress-controller通过和 kubernetes APIServer交互，动态的去感知集群中ingress规则变化；
2.然后读取它，按照自定义的规则，规则就是写明了哪个域名对应哪个service，生成一段nginx配置；
3.再写到nginx-ingress-controller的pod里，这个ingres-controller的pod里运行着一个Nginx服务，
控制器会把生成的nginx置写入/etc/nginx.conf文件中；
4.然后reload一下使配置生效。以此达到域名区分配置和动态更新的作用。
客户端一般会把请求发送到负载均衡器上由负载均衡器转发到ingress控制所在的node节点上目的减小Node节点的负载压力
ingress控制器会根据ingress资源对象的配置转发请求到对应的service
service会根据端点，把请求转发到关联的Pod
二.部署
A.下载部署文件和镜像
1.下载ingress-nginx的部署文件(如果出现无法下载此文件，请参考链接https://blog.csdn.net/weixin_43656190/article/details/117604612)
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.3.0/deploy/static/provider/cloud/deploy.yaml
备注：该文件在github上也可以下载，地址是：https://github.com/kubernetes/ingress-nginx/blob/main/deploy/static/provider/cloud/deploy.yaml
2.由于镜像问题，会导致yaml文件中的镜像无法下载，解决思路：先在docker hub上查找，下载下来之后，将其推送到私有的harbor镜像服务器上
需要下载如下两个镜像：
root@k8s-master1:~/ingress# cat deploy.yaml |grep imag
image: registry.k8s.io/ingress-nginx/controller:v1.3.0@sha256:d1707ca76d3b044ab8a28277a2466a02100ee9f58a86af1535a3edf9323ea1b5
imagePullPolicy: IfNotPresent
image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660
imagePullPolicy: IfNotPresent
image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660
imagePullPolicy: IfNotPresent
在docker hub上搜索
root@k8s-master1:~/ingress# docker search ingress-nginx --no-trunc
NAME DESCRIPTION STARS OFFICIAL AUTOMATED
willdockerhub/ingress-nginx-controller sync from k8s.gcr.io/ingress-nginx/controller 20
pollyduan/ingress-nginx-controller k8s.gcr.io/ingress-nginx/controller 9
liangjw/ingress-nginx-controller k8s.gcr.io/ingress-nginx/controller 7
wangshun1024/ingress-nginx-controller 3
dyrnq/ingress-nginx-controller k8s.gcr.io/ingress-nginx/controller 2
acicn/ingress-nginx-controller 2
wallarm/ingress-nginx 0
13916077927/ingress-nginx-controller sync from k8s.gcr.io/ingress-nginx/controller 0
rancher/ingress-nginx 0
v5cn/ingress-nginx-controller k8s.gcr.io/ingress-nginx/controller:v0.46.0 0
platform9/ingress-nginx 0
cangyin/ingress-nginx-controller 0
forging2012/ingress-nginx-controller 0
sundewang/ingress-nginx-controller 0
boy530/ingress-nginx-controller 0
blendlabs/ingress-nginx-catchall A blend branded nginx catchall. 0
cjk2atmb/ingress-nginx-controller 0
opsdockerimage/ingress-nginx-controller 0
acicn/ingress-nginx-kube-webhook-certgen 0
rudl/ingress-nginx Nginx ingress controller 0
opsdockerimage/ingress-nginx-nginx 0
lchdzh/ingress-nginx-controller 0
liy36/ingress-nginx-controller 0
opsdockerimage/ingress-nginx-e2e-test-runner 0
chainstack/ingress-nginx 0
root@k8s-master1:~/ingress# docker search kube-webhook-certgen --no-trunc
NAME DESCRIPTION STARS OFFICIAL AUTOMATED
jettech/kube-webhook-certgen Kubernetes webhook certificate generator and patcher 14
liangjw/kube-webhook-certgen k8s.gcr.io/ingress-nginx/kube-webhook-certgen 8
dyrnq/kube-webhook-certgen k8s.gcr.io/ingress-nginx/kube-webhook-certgen 5
wangshun1024/kube-webhook-certgen 3
wonderflow/kube-webhook-certgen 1
lianyuxue1020/kube-webhook-certgen new pull lianyuxue1020/kube-webhook-certgen:v1.1.1 1
newrelic/kube-webhook-certgen 0
catalystcloud/kube-webhook-certgen 0
spdplx2021/kube-webhook-certgen 0
congruity360/kube-webhook-certgen 0
pangser/kube-webhook-certgen 0
oamdev/kube-webhook-certgen 0
sundewang/kube-webhook-certgen 0
hzde0128/kube-webhook-certgen 0
kubespheredev/kube-webhook-certgen 0
caas4/kube-webhook-certgen 0
rpkatz/kube-webhook-certgen 0
xiaoyan1121/kube-webhook-certgen k8s.gcr.io/ingress-nginx/kube-webhook-certgen 0
serenade11/kube-webhook-certgen 0
invidian/kube-webhook-certgen 0
serialt/kube-webhook-certgen 0
zhangdiandong/kube-webhook-certgen kube-webhook-certgen:v1.0 0
fnape/kube-webhook-certgen k8s.gcr.io/ingress-nginx/kube-webhook-certgen Mirror accessible from CN 0
giantswarm/kube-webhook-certgen 0
madforg/kube-webhook-certgen k8s ingress-nginx 实验所需的gcr.io镜像 ( k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0 ) 0
root@k8s-master1:~/ingress#
3.下载，重命名，上传，替换镜像
#1.下载镜像
root@k8s-master1:~/ingress# docker pull dyrnq/ingress-nginx-controller:v1.3.0
root@k8s-master1:~/ingress# docker pull lianyuxue1020/kube-webhook-certgen:v1.1.1
#2.将镜像重命名
root@k8s-master1:~/ingress# docker tag dyrnq/ingress-nginx-controller:v1.3.0 registry.harbor.com/library/ingress-nginx-controller:v1.3.0
root@k8s-master1:~/ingress# docker tag lianyuxue1020/kube-webhook-certgen:v1.1.1 registry.harbor.com/library/kube-webhook-certgen:v1.1.1
#3.将镜像push到我们之前搭建的harbor仓库
root@k8s-master1:~/ingress# docker push registry.harbor.com/library/ingress-nginx-controller:v1.3.0
The push refers to repository [registry.harbor.com/library/ingress-nginx-controller]
11d5b7949407: Pushed
a2ab1d98af90: Pushed
5b42063defba: Pushed
3ee663ef34d0: Pushed
bf8b0761126a: Pushed
158aa99c71d8: Pushed
05856d1bb285: Pushed
f1b0a27f62c1: Pushed
5f70bf18a086: Pushed
7f366fb88640: Pushed
12ca2606c07b: Pushed
ca8e478eeb76: Pushed
b45bedc7f782: Pushed
24302eb7d908: Pushed
v1.3.0: digest: sha256:067673df26a65ec5c2d5b30f25db869bad4d7d391fc81882250134577e581ef0 size: 3256
root@k8s-master1:~/ingress# docker push registry.harbor.com/library/kube-webhook-certgen:v1.1.1
The push refers to repository [registry.harbor.com/library/kube-webhook-certgen]
ce7a3c1169b6: Pushed
c0d270ab7e0d: Pushed
v1.1.1: digest: sha256:78351fc9d9b5f835e0809921c029208faeb7fbb6dc2d3b0d1db0a6584195cfed size: 739
#4.将ingress-nginx部署文件中的镜像地址替换成harbor仓库的地址
root@k8s-master1:~/ingress# sed -i s#registry.k8s.io/ingress-nginx/controller:v1.3.0@sha256:d1707ca76d3b044ab8a28277a2466a02100ee9f58a86af1535a3edf9323e#registry.harbor.com/library/ingress-nginx-controller:v1.3.0#g deploy.yaml
root@k8s-master1:~/ingress# sed -i s#registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660#registry.harbor.com/library/kube-webhook-certgen:v1.1.1#g deploy.yaml
#5.查看镜像地址是否替换成功
root@k8s-master1:~/ingress# cat deploy.yaml |grep image
image: registry.harbor.com/library/ingress-nginx-controller:v1.3.0@sha256:d1707ca76d3b044ab8a28277a2466a02100ee9f58a86af1535a3edf9323ea1b5
imagePullPolicy: IfNotPresent
image: registry.harbor.com/library/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660
imagePullPolicy: IfNotPresent
image: registry.harbor.com/library/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660
imagePullPolicy: IfNotPresent
root@k8s-master1:~/ingress#
B.node节点打标签
可以通过nodeSelector确定将ingress-nginx-controller部署在那些机器上
root@k8s-master1:~/ingress# kubectl label nodes k8s-node1 hasIngress=true
node/k8s-node1 labeled
root@k8s-master1:~/ingress# kubectl label nodes k8s-node2 hasIngress=true
node/k8s-node2 labeled
root@k8s-master1:~/ingress#
C.修改部署文件
修改三个地方
1.HostNetwork
是为了打通Cluster和node的网络，让Cluster直接监听node的端口，一般是80和443，不用再通过随机绑定的nodePort来访问集群服务
以前访问：http://www.xwl.com:30080/web1/contextpath
现在访问：http://www.xwl.com/web1/contextpath
可以看到不再需要随机绑定的nodePort了，因为hostNetwork直接监听集群的80端口。
2.nodeSelector
主要是为了能够选择将ingress-nginx-controller部署到那个机器上
3.DaemonSet
因为Deployment可能会把多个pod调度到同一个node，那就失去高可用的意义了。而DaemonSet在一个节点上只会有一个Pod，符合我们的要求。
修改deploy.yaml
......
apiVersion: apps/v1
#kind: Deployment
kind: DaemonSet #这里把Deployment改成DaemonSet
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
minReadySeconds: 0
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
template:
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
spec:
hostNetwork: true #这里加一句
containers:
- args:
- /nginx-ingress-controller
- --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
- --election-id=ingress-controller-leader
- --controller-class=k8s.io/ingress-nginx
- --ingress-class=nginx
- --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
- --validating-webhook=:8443
- --validating-webhook-certificate=/usr/local/certificates/cert
- --validating-webhook-key=/usr/local/certificates/key
env:
......
nodeSelector:
kubernetes.io/os: linux
hasIngress: "true" # 这里个面具标签选择node
......
D.部署并查看
部署ingress-nginx,部署后，需要等待一会，大约1分钟这样，貌似部署成功，其实没有部署成功
root@k8s-master1:~/ingress# kubectl apply -f deploy.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
deployment.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
root@k8s-master1:~/ingress# kubectl get pods -n ingress-nginx -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
ingress-nginx-admission-create-vc554 0/1 Completed 0 3m18s 10.244.2.44 k8s-node2 <none> <none>
ingress-nginx-admission-patch-swbrw 0/1 Completed 0 3m18s 10.244.2.45 k8s-node2 <none> <none>
ingress-nginx-controller-c8bxg 1/1 Running 0 3m18s 192.168.100.243 k8s-node1 <none> <none>
ingress-nginx-controller-ph6vn 1/1 Running 0 3m18s 192.168.100.244 k8s-node2 <none> <none>
root@k8s-master1:~/ingress#
三.应用
1、创建tomcat-nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
namespace: dev
spec:
replicas: 3
selector:
matchLabels:
app: nginx-pod
template:
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- containerPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: tomcat-deployment
namespace: dev
spec:
replicas: 3
selector:
matchLabels:
app: tomcat-pod
template:
metadata:
labels:
app: tomcat-pod
spec:
containers:
- name: tomcat
image: tomcat:8.5-jre10-slim
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: nginx-service
namespace: dev
spec:
ports:
- port: 80
name: nginx
clusterIP: None
selector:
app: nginx-pod
---
apiVersion: v1
kind: Service
metadata:
name: tomcat-service
namespace: dev
spec:
ports:
- port: 8080
name: tomcat
clusterIP: None
selector:
app: tomcat-pod
2.创建ingress-http.yaml
kind: Ingress
metadata:
name: ingress-http
namespace: dev
spec:
ingressClassName: nginx
rules:
- host: nginx.xwl.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nginx-service
port:
number: 80
- host: tomcat.xwl.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: tomcat-service
port:
number: 8080
3.修改任意一个主机的/etc/hosts文件，添加如下4个域名解析
192.168.100.243 nginx.xwl.com
192.168.100.244 nginx.xwl.com
192.168.100.243 tomcat.xwl.com
192.168.100.244 tomcat.xwl.com
4.验证是否能够正常登录
登录nginx.xwl.com
登录tomcat.xwl.com
四.参考文档
1.ingress官方文档
https://kubernetes.io/zh-cn/docs/concepts/services-networking/ingress/
2.ingress-nginx官方文档
https://kubernetes.github.io/ingress-nginx/deploy/#quick-start
3.hostNetwork与ingress-nginx高可用
https://blog.csdn.net/weixin_41535316/article/details/121873795
4.k8s部署ingress-nginx的方法步骤
https://blog.csdn.net/a13568hki/article/details/124452407
5.K8S对外服务之Ingress （使用https的时候，可以参考这篇文档）
https://blog.51cto.com/u_15382481/5205194
6.k8s ingress原理
https://blog.csdn.net/weixin_46108954/article/details/105717399
为开发者提供学习成长、分享交流、生态实践、资源工具等服务，帮助开发者快速成长。
更多推荐
6
0- 0
已为社区贡献4条内容
所有评论(0)