最近更新时间:2021-01-22 15:22:35
本文将为您介绍基于金山云容器服务搭建 Nginx-ingress 服务的操作指南。
使用Nginx-ingress服务的前提是在集群内部署nginx-ingress controller,controller部署的nginx-install.yaml
如下:
apiVersion: v1
kind: Namespace
metadata:
name: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-configuration
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: tcp-services
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: udp-services
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: nginx-ingress-clusterrole
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- "extensions"
- "networking.k8s.io"
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- "extensions"
- "networking.k8s.io"
resources:
- ingresses/status
verbs:
- update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
name: nginx-ingress-role
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
resourceNames:
# Defaults to "<election-id>-<ingress-class>"
# Here: "<ingress-controller-leader>-<nginx>"
# This has to be adapted if you change either parameter
# when launching the nginx-ingress-controller.
- "ingress-controller-leader-nginx"
verbs:
- get
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- ""
resources:
- endpoints
verbs:
- get
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: nginx-ingress-role-nisa-binding
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: nginx-ingress-role
subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: nginx-ingress-clusterrole-nisa-binding
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: nginx-ingress-clusterrole
subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-ingress-controller
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
template:
metadata:
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
annotations:
prometheus.io/port: "10254"
prometheus.io/scrape: "true"
spec:
# wait up to five minutes for the drain of connections
terminationGracePeriodSeconds: 300
serviceAccountName: nginx-ingress-serviceaccount
nodeSelector:
kubernetes.io/os: linux
containers:
- name: nginx-ingress-controller
image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0
args:
- /nginx-ingress-controller
- --configmap=$(POD_NAMESPACE)/nginx-configuration
- --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
- --udp-services-configmap=$(POD_NAMESPACE)/udp-services
- --publish-service=$(POD_NAMESPACE)/ingress-nginx
- --annotations-prefix=nginx.ingress.kubernetes.io
securityContext:
allowPrivilegeEscalation: true
capabilities:
drop:
- ALL
add:
- NET_BIND_SERVICE
# www-data -> 101
runAsUser: 101
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
ports:
- name: http
containerPort: 80
protocol: TCP
- name: https
containerPort: 443
protocol: TCP
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10
lifecycle:
preStop:
exec:
command:
- /wait-shutdown
---
apiVersion: v1
kind: LimitRange
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
limits:
- min:
memory: 90Mi
cpu: 100m
type: Container
执行以下命令安装部署nginx-ingress controller 0.30.0版本:
[[email protected] ~]# kubectl apply -f nginx-install.yaml
#将会在集群中创建如下资源
namespace/ingress-nginx created
configmap/nginx-configuration created
configmap/tcp-services created
configmap/udp-services created
serviceaccount/nginx-ingress-serviceaccount created
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
role.rbac.authorization.k8s.io/nginx-ingress-role created
rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
deployment.apps/nginx-ingress-controller created
limitrange/ingress-nginx created
为Deployment nginx-ingress-controller创建一个类型为LoadBalancer的Service,将通过金山云的LB暴露到集群外。nginx-ingress-svc.yaml如下:
apiVersion: v1
kind: Service
metadata:
labels:
app: nginx
name: nginx-ingress
spec:
ports:
- name: "tcp-80-80"
port: 80
protocol: TCP
targetPort: 80
- name: "tcp-443-443"
port: 443
protocol: TCP
targetPort: 443
selector:
app.kubernetes.io/name: "ingress-nginx"
app.kubernetes.io/part-of: "ingress-nginx"
type: LoadBalancer
创建服务,并获取服务的IP地址。
[[email protected] ~]# kubectl apply -f nginx-ingress-svc.yaml -n ingress-nginx
service/nginx-ingress created
[[email protected] ~]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-ingress LoadBalancer 10.254.137.163 120.131.xx.xx 80:30563/TCP,443:30681/TCP 17s
通过EXTERNAL-IP(120.131.xx.xx),外部流量将访问到集群中的nginx-ingress-controller,进而实现Ingress规则中的路由转发。
[[email protected] ~]# kubectl get all -n ingress-nginx
NAME READY STATUS RESTARTS AGE
pod/nginx-ingress-controller-7fcf8df75d-pqbgd 1/1 Running 0 132m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/nginx-ingress LoadBalancer 10.254.137.163 120.131.xx.xx 80:30563/TCP,443:30681/TCP 2m52s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx-ingress-controller 1/1 1 1 149m
NAME DESIRED CURRENT READY AGE
replicaset.apps/nginx-ingress-controller-7fcf8df75d 1 1 1 149m
以下创建两个应用,用于测试。
hello-world.yaml如下:
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello-world
spec:
replicas: 1
selector:
matchLabels:
app: hello-world
template:
metadata:
labels:
app: hello-world
spec:
containers:
- name: hello-world
image: hub.kce.ksyun.com/kingsoft/hello-world:latest
---
apiVersion: v1
kind: Service
metadata:
labels:
app: hello-world
name: hello-world-svc
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: hello-world
type: ClusterIP
hello-k8s.yaml如下:
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello-k8s
spec:
replicas: 1
selector:
matchLabels:
app: hello-k8s
template:
metadata:
labels:
app: hello-k8s
spec:
containers:
- name: hello-k8s
image: hub.kce.ksyun.com/kingsoft/hello-k8s:latest
---
apiVersion: v1
kind: Service
metadata:
labels:
app: hello-k8s
name: hello-k8s-svc
spec:
ports:
- port: 8080
protocol: TCP
targetPort: 8080
selector:
app: hello-k8s
type: ClusterIP
创建对应的deploy和service:
[[email protected] hello]# kubectl create -f hello-k8s.yaml
deployment.extensions/hello-k8s created
service/hello-k8s-svc created
[[email protected] hello]# kubectl create -f hello-world.yaml
deployment.extensions/hello-world created
service/hello-world-svc created
[[email protected] hello]# kubectl get deploy
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
hello-k8s 1 1 1 1 5m2s
hello-world 1 1 1 1 4m50s
[[email protected] hello]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hello-k8s-svc ClusterIP 10.254.131.29 <none> 8080/TCP 5m31s
hello-world-svc ClusterIP 10.254.244.96 <none> 80/TCP 5m19s
kubernetes ClusterIP 10.254.0.1 <none> 443/TCP 52d
为了支持灵活的分发策略,ingress策略可以按照多种分发方式进行配置,下面对几种常见的ingress转发策略简单介绍。
这种配置常用于一个网站通过不同的路径提供不同服务的场景。
通过如下的访问配置:
ingress.yaml如下:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-test
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: my.nginx.test
http:
paths:
- path: /hello-world
backend:
serviceName: hello-world-svc
servicePort: 80
- path: /hello-k8s
backend:
serviceName: hello-k8s-svc
servicePort: 8080
创建ingress规则:
[[email protected] ~]# kubectl apply -f ingress.yaml
ingress.extensions/nginx-test created
[[email protected] ~]# kubectl get ingress
NAME HOSTS ADDRESS PORTS AGE
nginx-test my.nginx.test 80 15s
备注:
在浏览器的访问验证如下:
这种配置常用于一个网站通过不同的域名或者虚拟主机名提供不同的服务的场景。
通过如下的访问配置:
ingress2.yaml如下:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-test-2
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: nginx.hello.k8s
http:
paths:
- path: /
backend:
serviceName: hello-k8s-svc
servicePort: 8080
- host: nginx.hello.world
http:
paths:
- path: /
backend:
serviceName: hello-world-svc
servicePort: 80
创建Ingress规则:
[[email protected] ~]# kubectl apply -f ingress2.yaml
ingress.extensions/nginx-test-2 created
[[email protected] ~]# kubectl get ingress
NAME HOSTS ADDRESS PORTS AGE
nginx-test-2 nginx.hello.k8s,nginx.hello.world 80 19s
在浏览器的访问验证如下:
当Ingress配置TLS时,服务将以HTTPS协议的方式对外暴露。
这里我们作为测试用例使用自签名证书,使用如下命令快速创建。
[[email protected] ~]# openssl req -newkey rsa:2048 -nodes -keyout tls.key -x509 -days 365 -out tls.crt
Generating a 2048 bit RSA private key
...............+++
.............................+++
writing new private key to 'tls.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:Beijing
Locality Name (eg, city) [Default City]:Beijing
Organization Name (eg, company) [Default Company Ltd]:Kingsoft
Organizational Unit Name (eg, section) []:Ksyun
Common Name (eg, your name or your server's hostname) []:nginx.hello.k8s #配置需使用证书的域名
Email Address []:[email protected]
将会在当前路径下创建证书(tls.crt
)和私钥文件(tls.key
)。
根据现有证书和私钥创建Secret资源,将会创建一个类型为kubernetes.io/tls
的Secret。
[[email protected] ~]# kubectl create secret tls secret-https --key tls.key --cert tls.crt
secret/secret-https created
对上述示例中的http访问方式进行升级,实现对nginx.hello.k8s
的https访问。ingress-https.yaml
如下:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-test-2
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "true" #开启重定向功能
spec:
rules:
- host: nginx.hello.k8s
http:
paths:
- path: /
backend:
serviceName: hello-k8s-svc
servicePort: 8080
tls:
- hosts:
- nginx.hello.k8s
secretName: secret-https
在浏览器的访问验证如下:
更多Nginx ingress controller的特性,请参考NGINX Ingress Controller。