Helm是Kubernetes的包管理器,Helm 让我们能够像 yum 管理 rpm 包那样安装、部署、升级和删除容器化应用。
Helm主要由Helm客户端、Tiller服务器和Charts仓库组成:
简单的说:Helm 客户端负责管理 chart;Tiller 服务器负责管理 release。
部署文档:https://helm.sh/docs/using_helm/#quickstart-guide
下载:https://github.com/helm/helm/releases
[root@k8s-master ~]# wget https://get.helm.sh/helm-v2.14.1-linux-amd64.tar.gz
[root@k8s-master ~]# tar xf helm-v2.14.1-linux-amd64.tar.gz
[root@k8s-master ~]# mv linux-amd64/helm /usr/local/bin/
[root@k8s-master ~]# cat helm-service-account.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: tiller
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: tiller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: tiller
namespace: kube-system
[root@k8s-master ~]# kubectl apply -f helm-service-account.yaml
serviceaccount/tiller created
clusterrolebinding.rbac.authorization.k8s.io/tiller created
[root@k8s-master ~]# helm init --tiller-image gcr.azk8s.cn/kubernetes-helm/tiller:v2.14.1 --skip-refresh --service-account tiller
[root@k8s-master ~]# kubectl get pod -n kube-system |grep tiller
tiller-deploy-6b6bcb8f7c-pbnnc 1/1 Running 0 49s
[root@k8s-master ~]# helm version
Client: &version.Version{SemVer:"v2.14.1", GitCommit:"5270352a09c7e8b6e8c9593002a73535276507c0", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.14.1", GitCommit:"5270352a09c7e8b6e8c9593002a73535276507c0", GitTreeState:"clean"}
helm常用命令:
- helm search: 搜索charts
- helm fetch: 下载charts到本地目录
- helm install: 安装charts
- helm list: 列出charts的所有版本
用法:
helm [command]
命令可用选项:
completion 为指定的shell生成自动补全脚本(bash或zsh)
create 创建一个新的charts
delete 删除指定版本的release
dependency 管理charts的依赖
fetch 下载charts并解压到本地目录
get 下载一个release
history release历史信息
home 显示helm的家目录
init 在客户端和服务端初始化helm
inspect 查看charts的详细信息
install 安装charts
lint 检测包的存在问题
list 列出release
package 将chart目录进行打包
plugin add(增加), list(列出), or remove(移除) Helm 插件
repo add(增加), list(列出), remove(移除), update(更新), and index(索引) chart仓库
reset 卸载tiller
rollback release版本回滚
search 关键字搜索chart
serve 启动一个本地的http server
status 查看release状态信息
template 本地模板
test release测试
upgrade release更新
verify 验证chart的签名和有效期
version 打印客户端和服务端的版本信息
官方可用的Chart列表:https://hub.kubeapps.com
Charts是Helm的程序包,它们都存在在Charts仓库当中。Kubernetes官方的仓库保存了一系列的Charts,仓库默认的名称为stable
。安装Charts到集群时,Helm首先会到官方仓库获取相关的Charts,并创建release。可执行 helm search
查看当前可安装的 chart 。
[root@k8s-master ~]# helm search
NAME CHART VERSION APP VERSION DESCRIPTION
stable/acs-engine-autoscaler 2.2.2 2.1.1 DEPRECATED Scales worker nodes within agent pools
stable/aerospike 0.2.8 v4.5.0.5 A Helm chart for Aerospike in Kubernetes
stable/airflow 3.0.2 1.10.3 Airflow is a platform to programmatically author, schedul...
Helm 安装时已经默认配置好了两个仓库:stable
和 local
。stable
是官方仓库,local
是用户存放自己开发的chart
的本地仓库。可以通过helm repo list
进行查看。
由于网络原因,国内无法更新仓库源,这里更改为阿里云的仓库源。
[root@k8s-master ~]# helm repo list
NAME URL
stable https://kubernetes-charts.storage.googleapis.com
local http://127.0.0.1:8879/charts
[root@k8s-master ~]# helm repo remove stable
"stable" has been removed from your repositories
[root@k8s-master ~]# helm repo list
NAME URL
local http://127.0.0.1:8879/charts
[root@k8s-master ~]# helm repo add stable https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts
"stable" has been added to your repositories
[root@k8s-master ~]# helm repo list
NAME URL
local http://127.0.0.1:8879/charts
stable https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts
[root@k8s-master ~]# helm repo update
Hang tight while we grab the latest from your chart repositories...
...Skip local chart repository
...Successfully got an update from the "stable" chart repository
Update Complete.
[root@localhost ~]# helm search mysql
NAME CHART VERSION APP VERSION DESCRIPTION
stable/mysql 0.3.5 Fast, reliable, scalable, and easy to use open-source rel...
stable/percona 0.3.0 free, fully compatible, enhanced, open source drop-in rep...
stable/percona-xtradb-cluster 0.0.2 5.7.19 free, fully compatible, enhanced, open source drop-in rep...
stable/gcloud-sqlproxy 0.2.3 Google Cloud SQL Proxy
stable/mariadb 2.1.6 10.1.31 Fast, reliable, scalable, and easy to use open-source rel...
[root@localhost ~]# helm install stable/mysql
NAME: vigilant-clownfish
LAST DEPLOYED: Thu Jun 27 14:21:37 2019
NAMESPACE: default
STATUS: DEPLOYED
RESOURCES:
==> v1/PersistentVolumeClaim
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
vigilant-clownfish-mysql Pending 7s
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
vigilant-clownfish-mysql-757ff9c4fd-2jh58 0/1 Pending 0 4s
==> v1/Secret
NAME TYPE DATA AGE
vigilant-clownfish-mysql Opaque 2 7s
==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
vigilant-clownfish-mysql ClusterIP 10.0.0.26 <none> 3306/TCP 5s
==> v1beta1/Deployment
NAME READY UP-TO-DATE AVAILABLE AGE
vigilant-clownfish-mysql 0/1 1 0 5s
NOTES:
MySQL can be accessed via port 3306 on the following DNS name from within your cluster:
vigilant-clownfish-mysql.default.svc.cluster.local
To get your root password run:
MYSQL_ROOT_PASSWORD=$(kubectl get secret --namespace default vigilant-clownfish-mysql -o jsonpath="{.data.mysql-root-password}" | base64 --decode; echo)
To connect to your database:
1. Run an Ubuntu pod that you can use as a client:
kubectl run -i --tty ubuntu --image=ubuntu:16.04 --restart=Never -- bash -il
2. Install the mysql client:
$ apt-get update && apt-get install mysql-client -y
3. Connect using the mysql cli, then provide your password:
$ mysql -h vigilant-clownfish-mysql -p
To connect to your database directly from outside the K8s cluster:
MYSQL_HOST=127.0.0.1
MYSQL_PORT=3306
# Execute the following commands to route the connection:
export POD_NAME=$(kubectl get pods --namespace default -l "app=vigilant-clownfish-mysql" -o jsonpath="{.items[0].metadata.name}")
kubectl port-forward $POD_NAME 3306:3306
mysql -h ${MYSQL_HOST} -P${MYSQL_PORT} -u root -p${MYSQL_ROOT_PASSWORD}
安装 chart,Helm 支持四种安装方法:
helm install stable/nginx
helm install ./nginx-1.2.3.tgz
helm install ./nginx
helm install https://example.com/charts/nginx-1.2.3.tgz
[root@localhost ~]# helm list
NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE
vigilant-clownfish 1 Thu Jun 27 14:21:37 2019 DEPLOYED mysql-0.3.5 default
[root@localhost ~]# helm delete vigilant-clownfish
release "vigilant-clownfish" deleted
chart 是 Helm 的应用打包格式。chart 由一系列文件组成,这些文件描述了 Kubernetes 部署应用时所需要的资源,比如 Service、Deployment、PersistentVolumeClaim、Secret、ConfigMap 等。
单个的 chart 可以非常简单,只用于部署一个服务,比如 Memcached;chart 也可以很复杂,部署整个应用,比如包含 HTTP Servers、 Database、消息中间件、cache 等。
chart 将这些文件放置在预定义的目录结构中,通常整个 chart 被打成 tar 包,而且标注上版本信息,便于 Helm 部署。
以前面 MySQL chart 为例。一旦安装了某个 chart,我们就可以在 ~/.helm/cache/archive 中找到 chart 的 tar 包。
[root@localhost ~]# cd .helm/cache/archive/
[root@localhost archive]# ll
total 8
-rw-r--r-- 1 root root 5536 Jun 27 14:21 mysql-0.3.5.tgz
[root@localhost archive]# tar xf mysql-0.3.5.tgz
[root@localhost archive]# tree mysql
mysql
├── Chart.yaml
├── README.md
├── templates
│ ├── configmap.yaml
│ ├── deployment.yaml
│ ├── _helpers.tpl
│ ├── NOTES.txt
│ ├── pvc.yaml
│ ├── secrets.yaml
│ └── svc.yaml
└── values.yaml
Kubernetes 给我们提供了大量官方 chart,不过要部署微服务应用,还是需要开发自己的 chart
[root@localhost ~]# helm create mychart
Creating mychart
[root@localhost ~]# tree mychart/
mychart/
├── charts
├── Chart.yaml
├── templates
│ ├── deployment.yaml
│ ├── _helpers.tpl
│ ├── ingress.yaml
│ ├── NOTES.txt
│ ├── service.yaml
│ └── tests
│ └── test-connection.yaml
└── values.yaml
Helm 会帮我们创建目录 mychart
,并生成了各类 chart 文件。这样我们就可以在此基础上开发自己的 chart 了。
只要是程序就会有 bug,chart 也不例外。Helm 提供了 debug 的工具:helm lint
和 helm install --dry-run --debug
。
[root@localhost ~]# helm lint mychart
==> Linting mychart
[INFO] Chart.yaml: icon is recommended
1 chart(s) linted, no failures
#helm install --dry-run --debug会模拟安装 chart,并输出每个模板生成的 YAML 内容。
[root@localhost ~]# helm install --dry-run mychart --debug
[debug] Created tunnel using local port: '31734'
[debug] SERVER: "127.0.0.1:31734"
[debug] Original chart version: ""
[debug] CHART PATH: /root/mychart
NAME: invited-joey
REVISION: 1
RELEASED: Thu Jun 27 14:53:04 2019
CHART: mychart-0.1.0
USER-SUPPLIED VALUES:
{}
COMPUTED VALUES:
affinity: {}
fullnameOverride: ""
image:
pullPolicy: IfNotPresent
repository: nginx
tag: stable
imagePullSecrets: []
ingress:
annotations: {}
enabled: false
hosts:
- host: chart-example.local
paths: []
tls: []
nameOverride: ""
nodeSelector: {}
replicaCount: 1
resources: {}
service:
port: 80
type: ClusterIP
tolerations: []
HOOKS:
---
# invited-joey-mychart-test-connection
apiVersion: v1
kind: Pod
metadata:
name: "invited-joey-mychart-test-connection"
labels:
app.kubernetes.io/name: mychart
helm.sh/chart: mychart-0.1.0
app.kubernetes.io/instance: invited-joey
app.kubernetes.io/version: "1.0"
app.kubernetes.io/managed-by: Tiller
annotations:
"helm.sh/hook": test-success
spec:
containers:
- name: wget
image: busybox
command: ['wget']
args: ['invited-joey-mychart:80']
restartPolicy: Never
MANIFEST:
---
# Source: mychart/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
name: invited-joey-mychart
labels:
app.kubernetes.io/name: mychart
helm.sh/chart: mychart-0.1.0
app.kubernetes.io/instance: invited-joey
app.kubernetes.io/version: "1.0"
app.kubernetes.io/managed-by: Tiller
spec:
type: ClusterIP
ports:
- port: 80
targetPort: http
protocol: TCP
name: http
selector:
app.kubernetes.io/name: mychart
app.kubernetes.io/instance: invited-joey
---
# Source: mychart/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: invited-joey-mychart
labels:
app.kubernetes.io/name: mychart
helm.sh/chart: mychart-0.1.0
app.kubernetes.io/instance: invited-joey
app.kubernetes.io/version: "1.0"
app.kubernetes.io/managed-by: Tiller
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: mychart
app.kubernetes.io/instance: invited-joey
template:
metadata:
labels:
app.kubernetes.io/name: mychart
app.kubernetes.io/instance: invited-joey
spec:
containers:
- name: mychart
image: "nginx:stable"
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: 80
protocol: TCP
livenessProbe:
httpGet:
path: /
port: http
readinessProbe:
httpGet:
path: /
port: http
resources:
{
打包形成一个tgz文件,估计是每个项目一个chart,对应一个tgz
[root@localhost ~]# helm package mychart
Successfully packaged chart and saved it to: /root/mychart-0.1.0.tgz
上面我们是从本地的目录结构中的chart去进行部署,如果要集中管理chart,就需要涉及到repository的问题,因为helm repository都是指到外面的地址,接下来我们可以通过minio建立一个企业私有的存放仓库。
Minio提供对象存储服务。它的应用场景被设定在了非结构化的数据的存储之上了。众所周知,非结构化对象诸如图像/音频/视频/log文件/系统备份/镜像文件…等等保存起来管理总是不那么方便,size变化很大,类型很多,再有云端的结合会使得情况更加复杂,minio就是解决此种场景的一个解决方案。Minio号称其能很好的适应非结构化的数据,支持AWS的S3,非结构化的文件从数KB到5TB都能很好的支持。
Minio的使用比较简单,只有两个文件,服务端minio,客户访问端mc,比较简单。
在项目中,我们可以直接找一台虚拟机作为Minio Server,提供服务,当然minio也支持作为Pod部署。
在https://dl.minio.io/client/mc/release/linux-amd64/ 下载客户端程序mc和服务端程序minio
启动minio服务
[root@localhost ~]# minio server ./repository
针对使用得客户端加入security和token信息
[root@localhost ~]# ./mc config host add myminio http://192.168.0.225:9000 HF42ZU340MX6DPWZ426P YVkjGIEALDI8ZcaLyJYq5oNVWdkVwTUN0w3pYQB+
创建一个bucket同时设置权限
[root@localhost ~]# ./mc mb myminio/minio-helm-repo
Bucket created successfully `myminio/minio-helm-repo`.
[root@localhost ~]# ./mc policy download myminio/minio-helm-repo
Access permission for `myminio/minio-helm-repo` is set to `download`
将之前的tgz package上传minio
[root@localhost ~]# ./mc cp mychart-0.1.0.tgz myminio/minio-helm-repo
#index.yaml为了让helm对里面的包进行索引,找到各种entry,每个entry是一个项目,每个项目对应一个chart Package
[root@localhost ~]# cat index.yaml
apiVersion: v1
entries:
hello-svc:
- apiVersion: v1
description: Distributed object storage server built for cloud applications and devops.
digest: 8440f6f064ed91a75194e77d4b2be99c491c1cb04fb34bca4a36a5be67e1ef2c
name: mychart
urls:
- http://192.168.0.225:9000/minio-helm-repo/mychart-0.1.0.tgz
version: 0.1.0
[root@localhost ~]# ./mc cp ./index.yaml myminio/minio-helm-rep
浏览器访问minio的界面
[root@localhost ~]# helm repo add myrepo http://192.168.0.225:9000/minio-helm-repo
"myrepo" has been added to your repositories
[root@localhost ~]# helm repo list
NAME URL
local http://127.0.0.1:8879/charts
stable https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts
myrepo http://192.168.0.225:9000/minio-helm-repo
[root@localhost ~]# helm repo update
Hang tight while we grab the latest from your chart repositories...
...Skip local chart repository
...Successfully got an update from the "myrepo" chart repository
...Successfully got an update from the "stable" chart repository
Update Complete.