Kubernetes（k8s）安装dns附件组件以及使用

别先生

发布于 2020-06-23 03:24:50

1.3K00

代码可运行

文章被收录于专栏：别先生别先生

运行总次数：0

代码可运行

1、K8s附加组件DNS服务。

答：Kubernetes中有一个很重要的特性，服务子发现。一旦一个service被创建，该service的service ip和service port等信息都可以被注入到pod中供它们使用。kubernetes主要支持两种service发现机制，第一种是环境变量，第二种是DNS。没有dns服务的时候，kubernetes会采用环境变量的形式，一个有很多service，环境变量会变得很复杂，为了解决这个问题，我们使用DNS服务。

2、Pod的环境变量如何查看呢？

 1 [root@k8s-master ~]# kubectl get all -o wide
 2 NAME       DESIRED   CURRENT   READY     AGE       CONTAINER(S)   IMAGE(S)                             SELECTOR
 3 rc/mysql   1         1         1         16h       mysql          192.168.110.133:5000/mysql:5.7.30    app=mysql
 4 rc/myweb   1         1         1         1h        myweb          192.168.110.133:5000/tomcat:latest   app=myweb
 5 
 6 NAME             CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE       SELECTOR
 7 svc/kubernetes   10.254.0.1       <none>        443/TCP          13d       <none>
 8 svc/mysql        10.254.245.194   <none>        3306/TCP         16h       app=mysql
 9 svc/myweb        10.254.222.197   <nodes>       8080:30008/TCP   1h        app=myweb
10 
11 NAME             READY     STATUS    RESTARTS   AGE       IP            NODE
12 po/mysql-537xf   1/1       Running   1          16h       172.16.16.3   k8s-master
13 po/myweb-8b7n3   1/1       Running   0          1h        172.16.32.3   k8s-node3
14 [root@k8s-master ~]# kubectl exec -it myweb-8b7n3 /bin/bas
15 base64      basename    bash        bashbug     bashbug-64  
16 [root@k8s-master ~]# kubectl exec -it myweb-8b7n3 /bin/bas
17 base64      basename    bash        bashbug     bashbug-64  
18 [root@k8s-master ~]# kubectl exec -it myweb-8b7n3 /bin/bash
19 root@myweb-8b7n3:/usr/local/tomcat# env 
20 KUBERNETES_SERVICE_PORT_HTTPS=443
21 MYSQL_PORT=tcp://10.254.245.194:3306
22 MYSQL_PORT_3306_TCP_ADDR=10.254.245.194
23 KUBERNETES_SERVICE_PORT=443
24 MYSQL_PORT_3306_TCP_PORT=3306
25 HOSTNAME=myweb-8b7n3
26 JAVA_HOME=/usr/local/openjdk-11
27 GPG_KEYS=05AB33110949707C93A279E3D3EFE6B686867BA6 07E48665A34DCAFAE522E5E6266191C37C037D42 47309207D818FFD8DCD3F83F1931D684307A10A5 541FBE7D8F78B25E055DDEE13C370389288584E7 61B832AC2F1C5A90F0F9B00A1C506407564C17A3 79F7026C690BAA50B92CD8B66A3AD3F4F22C4FED 9BA44C2621385CB966EBA586F72C284D731FABEE A27677289986DB50844682F8ACB77FC2E86E29AC A9C5DF4D22E99998D9875A5110C01C5A2F6059E7 DCFD35E0BF8CA7344752DE8B6FB21E8933C60243 F3A04C595DB5B6A5F1ECA43E3B7BBB100D811BBE F7DA48BB64BCB84ECBA7EE6935CD23C10D498E23
28 JAVA_BASE_URL=https://github.com/AdoptOpenJDK/openjdk11-upstream-binaries/releases/download/jdk-11.0.7%2B10/OpenJDK11U-jdk_
29 MYSQL_PORT_3306_TCP=tcp://10.254.245.194:3306
30 PWD=/usr/local/tomcat
31 JAVA_URL_VERSION=11.0.7_10
32 TOMCAT_SHA512=75e16a00e02782961a7753dc9afaf6d209afa5f22d320319778fd0ee5e3b47009da522ac735599f1739bff6e809c2da9081dbbd4b8de54a82cf5b8cfbd8030ff
33 TOMCAT_MAJOR=9
34 HOME=/root
35 LANG=C.UTF-8
36 KUBERNETES_PORT_443_TCP=tcp://10.254.0.1:443
37 TOMCAT_NATIVE_LIBDIR=/usr/local/tomcat/native-jni-lib
38 TERM=xterm
39 MYSQL_SERVICE_PORT=3306
40 CATALINA_HOME=/usr/local/tomcat
41 MYSQL_SERVICE_HOST=10.254.245.194
42 MYSQL_PORT_3306_TCP_PROTO=tcp
43 SHLVL=1
44 KUBERNETES_PORT_443_TCP_PROTO=tcp
45 KUBERNETES_PORT_443_TCP_ADDR=10.254.0.1
46 LD_LIBRARY_PATH=/usr/local/tomcat/native-jni-lib
47 KUBERNETES_SERVICE_HOST=10.254.0.1
48 KUBERNETES_PORT=tcp://10.254.0.1:443
49 KUBERNETES_PORT_443_TCP_PORT=443
50 PATH=/usr/local/tomcat/bin:/usr/local/openjdk-11/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
51 TOMCAT_VERSION=9.0.36
52 JAVA_VERSION=11.0.7
53 _=/usr/bin/env
54 root@myweb-8b7n3:/usr/local/tomcat# java -version
55 openjdk version "11.0.7" 2020-04-14
56 OpenJDK Runtime Environment 18.9 (build 11.0.7+10)
57 OpenJDK 64-Bit Server VM 18.9 (build 11.0.7+10, mixed mode)
58 root@myweb-8b7n3:/usr/local/tomcat#

如果有几个Service就会有几个环境变量，这里进行简单的过滤。

 1 root@myweb-hsdwn:/usr/local/tomcat# env | grep -iE 'kubernetes|mysql|myweb'
 2 KUBERNETES_SERVICE_PORT_HTTPS=443
 3 MYSQL_PORT=tcp://10.254.207.238:3306
 4 MYSQL_PORT_3306_TCP_ADDR=10.254.207.238
 5 KUBERNETES_SERVICE_PORT=443
 6 MYSQL_PORT_3306_TCP_PORT=3306
 7 HOSTNAME=myweb-hsdwn
 8 MYSQL_PORT_3306_TCP=tcp://10.254.207.238:3306
 9 KUBERNETES_PORT_443_TCP=tcp://10.254.0.1:443
10 MYSQL_SERVICE_PORT=3306
11 MYSQL_SERVICE_HOST=10.254.207.238
12 MYSQL_PORT_3306_TCP_PROTO=tcp
13 KUBERNETES_PORT_443_TCP_PROTO=tcp
14 KUBERNETES_PORT_443_TCP_ADDR=10.254.0.1
15 KUBERNETES_SERVICE_HOST=10.254.0.1
16 KUBERNETES_PORT=tcp://10.254.0.1:443
17 KUBERNETES_PORT_443_TCP_PORT=443
18 root@myweb-hsdwn:/usr/local/tomcat#

1 root@myweb-hsdwn:/usr/local/tomcat# env | grep -iE 'kubernetes|mysql|myweb' | wc -l
2 16
3 root@myweb-hsdwn:/usr/local/tomcat#

如果环境变量过多的话，会变得很复杂，所以引入了DNS服务。

3、下载DNS的资料包，如下所示：

 1 [root@k8s-master tomcat_demo]# wget https://www.qstack.com.cn/skydns.zip
 2 --2020-06-18 15:31:25--  https://www.qstack.com.cn/skydns.zip
 3 Resolving www.qstack.com.cn (www.qstack.com.cn)... 123.125.46.149, 111.202.85.37
 4 Connecting to www.qstack.com.cn (www.qstack.com.cn)|123.125.46.149|:443... connected.
 5 HTTP request sent, awaiting response... 200 OK
 6 Length: 3411 (3.3K) [application/zip]
 7 Saving to: ‘skydns.zip’
 8 
 9 100%[======================================================================================>] 3,411       --.-K/s   in 0s      
10 
11 2020-06-18 15:31:26 (40.5 MB/s) - ‘skydns.zip’ saved [3411/3411]
12 
13 [root@k8s-master tomcat_demo]# ls
14 mysql-rc.yml  mysql-svc.yml  skydns.zip  tomcat-rc.yml  tomcat-svc.yml
15 [root@k8s-master tomcat_demo]# cd ..
16 [root@k8s-master k8s]# mv tomcat_demo/skydns.zip .
17 [root@k8s-master k8s]# ls
18 book-master.war  deploy  pod  rc  skydns.zip  svc  tomcat_demo  tomcat_demo.zip
19 [root@k8s-master k8s]# unzip skydns.zip 
20 Archive:  skydns.zip
21    creating: skydns/
22   inflating: skydns/skydns-rc.yaml   
23   inflating: skydns/skydns-svc.yaml  
24   inflating: skydns/test_dns_pod.yaml  
25 [root@k8s-master k8s]# ls
26 book-master.war  deploy  pod  rc  skydns  skydns.zip  svc  tomcat_demo  tomcat_demo.zip
27 [root@k8s-master k8s]#

修改skydns-rc.yaml配置文件。

  1 # Copyright 2016 The Kubernetes Authors.
  2 #
  3 # Licensed under the Apache License, Version 2.0 (the "License");
  4 # you may not use this file except in compliance with the License.
  5 # You may obtain a copy of the License at
  6 #
  7 #     http://www.apache.org/licenses/LICENSE-2.0
  8 #
  9 # Unless required by applicable law or agreed to in writing, software
 10 # distributed under the License is distributed on an "AS IS" BASIS,
 11 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 12 # See the License for the specific language governing permissions and
 13 # limitations under the License.
 14 
 15 # TODO - At some point, we need to rename all skydns-*.yaml.* files to kubedns-*.yaml.*
 16 # Should keep target in cluster/addons/dns-horizontal-autoscaler/dns-horizontal-autoscaler.yaml
 17 # in sync with this file.
 18 
 19 # __MACHINE_GENERATED_WARNING__
 20 
 21 apiVersion: extensions/v1beta1
 22 kind: Deployment
 23 metadata:
 24   name: kube-dns
 25   namespace: kube-system
 26   labels:
 27     k8s-app: kube-dns
 28     kubernetes.io/cluster-service: "true"
 29 spec:
 30   replicas: 1
 31   # replicas: not specified here:
 32   # 1. In order to make Addon Manager do not reconcile this replicas parameter.
 33   # 2. Default is 1.
 34   # 3. Will be tuned in real time if DNS horizontal auto-scaling is turned on.
 35   strategy:
 36     rollingUpdate:
 37       maxSurge: 10%
 38       maxUnavailable: 0
 39   selector:
 40     matchLabels:
 41       k8s-app: kube-dns
 42   template:
 43     metadata:
 44       labels:
 45         k8s-app: kube-dns
 46       annotations:
 47         scheduler.alpha.kubernetes.io/critical-pod: ''
 48         scheduler.alpha.kubernetes.io/tolerations: '[{"key":"CriticalAddonsOnly", "operator":"Exists"}]'
 49     spec:
 50       containers:
 51       - name: kubedns
 52         image: myhub.fdccloud.com/library/kubedns-amd64:1.9
 53         resources:
 54           # TODO: Set memory limits when we've profiled the container for large
 55           # clusters, then set request = limit to keep this container in
 56           # guaranteed class. Currently, this container falls into the
 57           # "burstable" category so the kubelet doesn't backoff from restarting it.
 58           limits:
 59             memory: 170Mi
 60           requests:
 61             cpu: 100m
 62             memory: 70Mi
 63         livenessProbe:
 64           httpGet:
 65             path: /healthz-kubedns
 66             port: 8080
 67             scheme: HTTP
 68           initialDelaySeconds: 60
 69           timeoutSeconds: 5
 70           successThreshold: 1
 71           failureThreshold: 5
 72         readinessProbe:
 73           httpGet:
 74             path: /readiness
 75             port: 8081
 76             scheme: HTTP
 77           # we poll on pod startup for the Kubernetes master service and
 78           # only setup the /readiness HTTP server once that's available.
 79           initialDelaySeconds: 3
 80           timeoutSeconds: 5
 81         args:
 82         - --domain=cluster.local.
 83         - --dns-port=10053
 84         - --config-map=kube-dns
 85         - --kube-master-url=http://10.0.0.11:8080
 86         # This should be set to v=2 only after the new image (cut from 1.5) has
 87         # been released, otherwise we will flood the logs.
 88         - --v=0
 89         #__PILLAR__FEDERATIONS__DOMAIN__MAP__
 90         env:
 91         - name: PROMETHEUS_PORT
 92           value: "10055"
 93         ports:
 94         - containerPort: 10053
 95           name: dns-local
 96           protocol: UDP
 97         - containerPort: 10053
 98           name: dns-tcp-local
 99           protocol: TCP
100         - containerPort: 10055
101           name: metrics
102           protocol: TCP
103       - name: dnsmasq
104         image: myhub.fdccloud.com/library/kube-dnsmasq-amd64:1.4
105         livenessProbe:
106           httpGet:
107             path: /healthz-dnsmasq
108             port: 8080
109             scheme: HTTP
110           initialDelaySeconds: 60
111           timeoutSeconds: 5
112           successThreshold: 1
113           failureThreshold: 5
114         args:
115         - --cache-size=1000
116         - --no-resolv
117         - --server=127.0.0.1#10053
118         #- --log-facility=-
119         ports:
120         - containerPort: 53
121           name: dns
122           protocol: UDP
123         - containerPort: 53
124           name: dns-tcp
125           protocol: TCP
126         # see: https://github.com/kubernetes/kubernetes/issues/29055 for details
127         resources:
128           requests:
129             cpu: 150m
130             memory: 10Mi
131       - name: dnsmasq-metrics
132         image: myhub.fdccloud.com/library/dnsmasq-metrics-amd64:1.0
133         livenessProbe:
134           httpGet:
135             path: /metrics
136             port: 10054
137             scheme: HTTP
138           initialDelaySeconds: 60
139           timeoutSeconds: 5
140           successThreshold: 1
141           failureThreshold: 5
142         args:
143         - --v=2
144         - --logtostderr
145         ports:
146         - containerPort: 10054
147           name: metrics
148           protocol: TCP
149         resources:
150           requests:
151             memory: 10Mi
152       - name: healthz
153         image: myhub.fdccloud.com/library/exechealthz-amd64:1.2
154         resources:
155           limits:
156             memory: 50Mi
157           requests:
158             cpu: 10m
159             # Note that this container shouldn't really need 50Mi of memory. The
160             # limits are set higher than expected pending investigation on #29688.
161             # The extra memory was stolen from the kubedns container to keep the
162             # net memory requested by the pod constant.
163             memory: 50Mi
164         args:
165         - --cmd=nslookup kubernetes.default.svc.cluster.local 127.0.0.1 >/dev/null
166         - --url=/healthz-dnsmasq
167         - --cmd=nslookup kubernetes.default.svc.cluster.local 127.0.0.1:10053 >/dev/null
168         - --url=/healthz-kubedns
169         - --port=8080
170         - --quiet
171         ports:
172         - containerPort: 8080
173           protocol: TCP
174       dnsPolicy: Default  # Don't use cluster DNS.

需要将此地址的ip地址修改为自己api-server的地址。

开始创建RC，创建需要很长时间的。

1 [root@k8s-master skydns]# kubectl create -f skydns-rc.yaml 
2 deployment "kube-dns" created
3 [root@k8s-master skydns]#

开始创建SVC，修改配置文件，使用kubectl get svc查看CLUSTER-IP的地址，只要未被使用，就可以配置到skydns-svc.yaml 配置文件中即可。

1 [root@k8s-master ~]# kubectl get svc 
2 NAME         CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
3 kubernetes   10.254.0.1       <none>        443/TCP          13d
4 mysql        10.254.207.238   <none>        3306/TCP         42m
5 myweb        10.254.29.22     <nodes>       8080:30008/TCP   28m
6 [root@k8s-master ~]#

创建svc的命令，如下所示：

1 [root@k8s-master skydns]# vim skydns-svc.yaml 
2 [root@k8s-master skydns]# kubectl create -f skydns-svc.yaml 
3 service "kube-dns" created
4 [root@k8s-master skydns]#

根据命名空间来查看创建好的pod，可以看到这个Pod里面的四个容器都已经运行起来了。

1 [root@k8s-master skydns]# kubectl get pod --namespace=kube-system
2 NAME                       READY     STATUS    RESTARTS   AGE
3 kube-dns-778415672-f9ssw   4/4       Running   0          6m
4 [root@k8s-master skydns]#

 1 [root@k8s-master skydns]# kubectl get all --namespace=kube-system
 2 NAME              DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
 3 deploy/kube-dns   1         1         1            1           7m
 4 
 5 NAME           CLUSTER-IP       EXTERNAL-IP   PORT(S)         AGE
 6 svc/kube-dns   10.254.230.254   <none>        53/UDP,53/TCP   3m
 7 
 8 NAME                    DESIRED   CURRENT   READY     AGE
 9 rs/kube-dns-778415672   1         1         1         7m
10 
11 NAME                          READY     STATUS    RESTARTS   AGE
12 po/kube-dns-778415672-f9ssw   4/4       Running   0          7m
13 [root@k8s-master skydns]#

如果希望所有的Pod都使用DNS的话，还需要修改一下配置文件。修改kubelet配置，修改各个node节点上的/etc/kubernetes/kubelet配置文件，增加如下行。然后再重启各个node节点的kubelet。

4、修改kubelet配置，修改各个node节点上的/etc/kubernetes/kubelet配置文件，增加如下行。然后再重启各个node节点的kubelet。

1 KUBELET_ARGS="--cluster_dns=10.254.230.254 --cluster_domain=cluster.local"

这个地址就是刚才自己配置的svc的地址，可以使用命令进行查看。

1 [root@k8s-master skydns]# vim /etc/kubernetes/kubelet 
2 [root@k8s-master skydns]# kubectl get svc --namespace=kube-system
3 NAME       CLUSTER-IP       EXTERNAL-IP   PORT(S)         AGE
4 kube-dns   10.254.230.254   <none>        53/UDP,53/TCP   9m
5 [root@k8s-master skydns]#

然后重启三台Node节点的kubelet服务。

1 [root@k8s-master skydns]# systemctl restart kubelet.service 
2 
3 [root@k8s-node2 ~]# systemctl restart kubelet.service
4 
5 [root@k8s-node3 ~]# systemctl restart kubelet.service

如何测试DNS是否生效呢，可以使用下面的配置进行测试。

 1 [root@k8s-master skydns]# ls
 2 skydns-rc.yaml  skydns-svc.yaml  test_dns_pod.yaml
 3 [root@k8s-master skydns]# cat test_dns_pod.yaml 
 4 apiVersion: v1
 5 kind: Pod
 6 metadata:
 7   labels:
 8     name: busybox
 9     role: master
10   name: busybox2
11 spec:
12   containers:
13   - name: busybox
14     image: docker.io/busybox:latest
15     imagePullPolicy: IfNotPresent
16     command:
17     - sleep
18     - "3600"
19 [root@k8s-master skydns]# kubectl create -f test_dns_pod.yaml 
20 pod "busybox2" created
21 [root@k8s-master skydns]# kubectl get pods
22 NAME          READY     STATUS    RESTARTS   AGE
23 busybox2      1/1       Running   0          20s
24 mysql-lmx4s   1/1       Running   0          59m
25 myweb-hsdwn   1/1       Running   0          45m
26 [root@k8s-master skydns]# kubectl exec -it busybox2 sh
27 / #

可以检测一下，svc的vip是否可以解析出来。

1 / # nslookup mysql
2 Server:        10.254.230.254
3 Address:    10.254.230.254:53
4 
5 Name:    mysql.default.svc.cluster.local
6 Address: 10.254.207.238

可以看出来和SVC的地址一样。

1 [root@k8s-master ~]# kubectl get svc 
2 NAME         CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
3 kubernetes   10.254.0.1       <none>        443/TCP          13d
4 mysql        10.254.207.238   <none>        3306/TCP         1h
5 myweb        10.254.29.22     <nodes>       8080:30008/TCP   46m
6 [root@k8s-master ~]#