全部文档
当前文档
最近更新时间：2024-05-30 15:22:23
NodeLocal DNSCache 通过在集群节点上作为 DaemonSet 运行 DNS 缓存代理来提高集群 DNS 性能。当集群规模较大时，建议通过使用NodeLocal DNSCache来提升服务发现的稳定性和性能。
使用当前的 DNS 体系结构，如果没有本地 kube-dns/CoreDNS 实例，则具有最高 DNS QPS 的 Pod 可能必须延伸到另一个节点。 在这种场景下，拥有本地缓存将有助于改善延迟。
跳过 iptables DNAT 和连接跟踪将有助于减少conntrack竞争并避免 UDP DNS 条目填满 conntrack 表。
从本地缓存代理到 kube-dns 服务的连接可以升级为 TCP。 TCP conntrack 条目将在连接关闭时被删除，相反 UDP 条目必须超时 （默认nf_conntrack_udp_timeout 是 30 秒）。
将 DNS 查询从 UDP 升级到 TCP 将减少由于被丢弃的 UDP 包和 DNS 超时而带来的尾部等待时间； 这类延时通常长达 30 秒（3 次重试 + 10 秒超时）。 由于 nodelocal 缓存监听 UDP DNS 查询，应用不需要变更。
在节点级别对 DNS 请求的度量和可见性。
可以重新启用负缓存，从而减少对 kube-dns 服务的查询数量。
node-local-dns 部署模版：
# Copyright 2018 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
apiVersion: v1
kind: ServiceAccount
metadata:
name: node-local-dns
namespace: kube-system
labels:
kubernetes.io/cluster-service: "true"
addonmanager.kubernetes.io/mode: Reconcile
---
apiVersion: v1
kind: Service
metadata:
name: kube-dns-upstream
namespace: kube-system
labels:
k8s-app: kube-dns
kubernetes.io/cluster-service: "true"
addonmanager.kubernetes.io/mode: Reconcile
kubernetes.io/name: "KubeDNSUpstream"
spec:
ports:
- name: dns
port: 53
protocol: UDP
targetPort: 53
- name: dns-tcp
port: 53
protocol: TCP
targetPort: 53
selector:
k8s-app: coredns
---
apiVersion: v1
kind: ConfigMap
metadata:
name: node-local-dns
namespace: kube-system
labels:
addonmanager.kubernetes.io/mode: Reconcile
data:
Corefile: |
__PILLAR__DNS__DOMAIN__:53 {
errors
cache {
success 9984 30
denial 9984 5
}
reload
loop
bind __PILLAR__LOCAL__DNS__ __PILLAR__DNS__SERVER__
forward . __PILLAR__CLUSTER__DNS__ {
force_tcp
}
prometheus :9253
health __PILLAR__LOCAL__DNS__:8081
}
in-addr.arpa:53 {
errors
cache 30
reload
loop
bind __PILLAR__LOCAL__DNS__ __PILLAR__DNS__SERVER__
forward . __PILLAR__CLUSTER__DNS__ {
force_tcp
}
prometheus :9253
}
ip6.arpa:53 {
errors
cache 30
reload
loop
bind __PILLAR__LOCAL__DNS__ __PILLAR__DNS__SERVER__
forward . __PILLAR__CLUSTER__DNS__ {
force_tcp
}
prometheus :9253
}
.:53 {
errors
cache 30
reload
loop
bind __PILLAR__LOCAL__DNS__ __PILLAR__DNS__SERVER__
forward . __PILLAR__UPSTREAM__SERVERS__
prometheus :9253
}
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: node-local-dns
namespace: kube-system
labels:
k8s-app: node-local-dns
kubernetes.io/cluster-service: "true"
addonmanager.kubernetes.io/mode: Reconcile
spec:
updateStrategy:
rollingUpdate:
maxUnavailable: 10%
selector:
matchLabels:
k8s-app: node-local-dns
template:
metadata:
labels:
k8s-app: node-local-dns
annotations:
prometheus.io/port: "9253"
prometheus.io/scrape: "true"
spec:
priorityClassName: system-node-critical
serviceAccountName: node-local-dns
hostNetwork: true
dnsPolicy: Default # Don't use cluster DNS.
tolerations:
- key: "CriticalAddonsOnly"
operator: "Exists"
- effect: "NoExecute"
operator: "Exists"
- effect: "NoSchedule"
operator: "Exists"
containers:
- name: node-cache
image: hub.kce.ksyun.com/ksyun/k8s-dns-node-cache:1.22.8
resources:
requests:
cpu: 25m
memory: 5Mi
args: [ "-localip", "__PILLAR__LOCAL__DNS__,__PILLAR__DNS__SERVER__", "-conf", "/etc/Corefile", "-upstreamsvc", "kube-dns-upstream" ]
securityContext:
privileged: true
ports:
- containerPort: 53
name: dns
protocol: UDP
- containerPort: 53
name: dns-tcp
protocol: TCP
- containerPort: 9253
name: metrics
protocol: TCP
livenessProbe:
httpGet:
host: __PILLAR__LOCAL__DNS__
path: /health
port: 8081
initialDelaySeconds: 60
timeoutSeconds: 5
volumeMounts:
- mountPath: /run/xtables.lock
name: xtables-lock
readOnly: false
- name: config-volume
mountPath: /etc/coredns
- name: kube-dns-config
mountPath: /etc/kube-dns
volumes:
- name: xtables-lock
hostPath:
path: /run/xtables.lock
type: FileOrCreate
- name: kube-dns-config
configMap:
name: coredns
optional: true
- name: config-volume
configMap:
name: node-local-dns
items:
- key: Corefile
path: Corefile.base
---
# A headless service is a service with a service IP but instead of load-balancing it will return the IPs of our associated Pods.
# We use this to expose metrics to Prometheus.
apiVersion: v1
kind: Service
metadata:
annotations:
prometheus.io/port: "9253"
prometheus.io/scrape: "true"
labels:
k8s-app: node-local-dns
name: node-local-dns
namespace: kube-system
spec:
clusterIP: None
ports:
- name: metrics
port: 9253
targetPort: 9253
selector:
k8s-app: node-local-dns将上文对应k8s版本的 node-local-dns 安装模板保存为 nodelocaldns.yaml 文件
按照如下方式生成对应的环境变量：
$ kubedns=`kubectl get svc coredns -n kube-system -o jsonpath={.spec.clusterIP}`
# 表示集群域，默认为 cluster.local，如果进行过修改，需要按修改值进行填写
$ domain=cluster.local
# 表示 DNSCache 本地的 IP，默认为169.254.20.10
$ localdns=169.254.20.10（3）按照以下命令替换模版文件中对应的变量
IPVS模式下（v1.15及以上集群）
sed -i "s/__PILLAR__LOCAL__DNS__/$localdns/g; s/__PILLAR__DNS__DOMAIN__/$domain/g; s/,__PILLAR__DNS__SERVER__//g; s/__PILLAR__CLUSTER__DNS__/$kubedns/g" nodelocaldns.yamlIPTABLES 模式下（v1.15以下集群）
sed -i "s/__PILLAR__LOCAL__DNS__/$localdns/g; s/__PILLAR__DNS__DOMAIN__/$domain/g; s/__PILLAR__DNS__SERVER__/$kubedns/g" nodelocaldns.yaml（4）安装部署
kubectl apply -f nodelocaldns.yaml使用node-local-dns的解析有两种配置方式：
1. 配置Pod的dnsconfig。
2. 配置kubelet的cluster-dns。
nameservers 指定 169.254.20.10
配置searches，使得集群内域名可以解析
ndots默认设置为5，可以适当降低以加速查询
#使用此方法，针对某个pod设置，不影响其他pod和组件
apiVersion: v1
kind: Pod
metadata:
name: dnsutils
spec:
dnsPolicy: None
dnsConfig:
nameservers:
- 169.254.20.10
searches:
- default.svc.cluster.local # 注意：default为该pod所在的命名空间
- svc.cluster.local
- cluster.local
options:
- name: ndots
value: "5"
containers:
- name: dnsutils
image: hub.kce.ksyun.com/ksyun/jessie-dnsutils:1.3
command:
- sleep
- "7200"
imagePullPolicy: IfNotPresent
restartPolicy: Always使用此方法，生效的对象则是kubelet所管理的所有Pod，需要重启所有节点的kubelet。
修改kubelet配置，默认情况下配置为--cluster-dns=10.254.0.10（如果coredns service的IP有修改，按照修改为准）。将DNSCache本地的IP添加到coredns serviceIP之前，例如：--cluster-dns=10.254.0.10改成 --cluster-dns=169.254.20.10,10.254.0.10
执行systemctl restart kubelet ，重启kubelet
执行systemctl status kubelet ，确定kubelet重启成功。需要确保kubelet启动成功，如果kubelet长时间未启动会导致节点NotReady
注意：新增节点同样需要进行修改，如果新增节点需要统一配置，请提交工单
部署以下busybox的实例deployment：
apiVersion: apps/v1
kind: Deployment
metadata:
name: busybox
spec:
selector:
matchLabels:
app: busybox
replicas: 1
template:
metadata:
labels:
app: busybox
spec:
containers:
- name: busybox
image: hub.kce.ksyun.com/ksyun/busybox:latest
args: ["sleep","100000"]查看对应的pod运行状态：
kubectl get pod -l app=busybox
NAME READY STATUS RESTARTS AGE
busybox-64bdb5d64c-c**** 1/1 Running 0 2m查看对应pod的resolv.conf文件：
kubectl exec -it busybox-64bdb5d64c-c**** -- cat /etc/resolv.conf
search default.svc.cluster.local svc.cluster.local cluster.local
nameserver 169.254.20.10
nameserver 10.254.0.10
options ndots:5nameserver中出现 169.254.20.10时，说明NodeLocal DNSCache 组件部署成功。
纯净模式