k8s 部署 - 使用helm3部署监控prometheus（普罗米修斯），从零到有，一文搞定

> 本文由 [简悦 SimpRead](http://ksria.com/simpread/) 转码， 原文地址 [mp.weixin.qq.com](https://mp.weixin.qq.com/s/VHXRclL_JT8PNze721ptag)

> 本文将带你从头到尾的来实现如何在 k8s 中部署一整套 Prometheus 监控系统，以及有哪些部署方式可以选择，让你轻松掌握。

1、Prometheus（普罗米修斯）部署方式
-----------------------

（1）手动部署

（2）Helm

*   类似于 centos 中的 yum 命令，是 k8s 中的包管理工具
    
*   一个包一个 Chart（一个目录）

（3）Prometheus Operator

（4）Helm + Prometheus Operator

*   我们这里采用这种方式来部署`Prometheus`监控

2、k8s 的 Helm 安装
---------------

> 版本我们选择最新的功能版本：3.8.0
>
> 下载地址可以从我微信公众号（运维家）后台回复：helm，即可获取下载地址。

上传到指定目录进行如下操作；

```
# 解压
[root@node1 ~]# tar xf helm-v3.8.0-linux-amd64.tar.gz 
# 移动到bin目录下
[root@node1 ~]# mv linux-amd64/helm /usr/local/bin/helm
# 环境变量根据个人情况进行配置如下
[root@node1 ~]# echo "export PATH=$PATH:/usr/local/bin/" >> /etc/profile
[root@node1 ~]# helm version
version.BuildInfo{Version:"v3.8.0", GitCommit:"d14138609b01886f544b2025f5000351c9eb092e", GitTreeState:"clean", GoVersion:"go1.17.5"}
[root@node1 ~]#

```

3、安装 Prometheus
---------------

> 由于 helm 默认会访问外网，有可能网速比较慢，那么我们采用直接把 charts 下载到本地的方式来安装；

### （1）从`github`上下载整个`charts`

地址如下：

```
https://github.com/helm/charts

```

登录之后复制`git`下载地址；

```
https://github.com/helm/charts.git

```

之后在我们的`master`节点上，使用如下命令进行下载；

```
[root@node1 ~]# git clone https://github.com/helm/charts.git

```

> 如果说上一步`git clone`执行失败的话，可以从微信公众号（运维家）后台回复：charts，即可获取软件包，解压即可使用；
>
> unzip charts-master.zip

### （2）安装 Prometheus

查看`charts`中的`prometheus`；

```
[root@node1 ~]# ls charts-master/stable/prometheus-operator/
Chart.yaml  ci  CONTRIBUTING.md  crds  hack  README.md  requirements.lock  requirements.yaml  templates  values.yaml
[root@node1 ~]#

```

安装；

```
# 复制程序包
[root@node1 ~]# cp -ra charts-master/stable/prometheus-operator .
# 创建存放依赖的目录
[root@node1 ~]# mkdir prometheus-operator/charts
# 复制依赖
[root@node1 ~]# cp -ra charts-master/stable/kube-state-metrics/ prometheus-operator/charts/
[root@node1 ~]# cp -ra charts-master/stable/prometheus-node-exporter/ prometheus-operator/charts/
[root@node1 ~]# cp -ra charts-master/stable/grafana/ prometheus-operator/charts/
# 安装，并命名为 monitor，这样子方便查找
[root@node1 ~]# helm install monitor ./prometheus-operator/

```

出现如下信息就表示安装成功了；

```
NOTES:
*******************
*** DEPRECATED ****
*******************
* stable/prometheus-operator chart is deprecated.
* Further development has moved to https://github.com/prometheus-community/helm-charts
* The chart has been renamed kube-prometheus-stack to more clearly reflect
* that it installs the `kube-prometheus` project stack, within which Prometheus
* Operator is only one component.

The Prometheus Operator has been installed. Check its status by running:
  kubectl --namespace default get pods -l "release=monitor"

Visit https://github.com/coreos/prometheus-operator for instructions on how
to create & configure Alertmanager and Prometheus instances using the Operator.
[root@node1 ~]#

```

但是需要注意的是，安装成功只是表明相关配置做好了，我们还需要实际运行下上面的命令来检查下状态的；

```
[root@node1 ~]# kubectl --namespace default get pods -l "release=monitor"
NAME                                                   READY   STATUS              RESTARTS   AGE
monitor-prometheus-node-exporter-9pdwj                 0/1     ContainerCreating   0          3m48s
monitor-prometheus-node-exporter-v9qhp                 1/1     Running             0          3m48s
monitor-prometheus-operato-operator-79458977df-5jgkb   0/2     ContainerCreating   0          3m48s
[root@node1 ~]# 
[root@node1 ~]# kubectl get all 
NAME                                                         READY   STATUS              RESTARTS   AGE
pod/alertmanager-monitor-prometheus-operato-alertmanager-0   0/2     ContainerCreating   0          47s
pod/monitor-grafana-775969b799-6tldl                         0/2     PodInitializing     0          8m42s
pod/monitor-kube-state-metrics-574788cc57-r9gtf              0/1     ImagePullBackOff    0          8m42s
pod/monitor-prometheus-node-exporter-9pdwj                   1/1     Running             0          8m42s
pod/monitor-prometheus-node-exporter-v9qhp                   1/1     Running             0          8m42s
pod/monitor-prometheus-operato-operator-79458977df-5jgkb     0/2     ContainerCreating   0          8m42s
pod/prometheus-monitor-prometheus-operato-prometheus-0       0/3     ContainerCreating   0          37s

NAME                                              TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
service/alertmanager-operated                     ClusterIP   None             <none>        9093/TCP,9094/TCP,9094/UDP   48s
service/kubernetes                                ClusterIP   10.233.0.1       <none>        443/TCP                      6d19h
service/monitor-grafana                           ClusterIP   10.233.10.12     <none>        80/TCP                       8m44s
service/monitor-kube-state-metrics                ClusterIP   10.233.199.229   <none>        8080/TCP                     8m44s
service/monitor-prometheus-node-exporter          ClusterIP   10.233.69.10     <none>        9100/TCP                     8m44s
service/monitor-prometheus-operato-alertmanager   ClusterIP   10.233.51.110    <none>        9093/TCP                     8m44s
service/monitor-prometheus-operato-operator       ClusterIP   10.233.209.238   <none>        8080/TCP,443/TCP             8m44s
service/monitor-prometheus-operato-prometheus     ClusterIP   10.233.40.187    <none>        9090/TCP                     8m44s
service/prometheus-operated                       ClusterIP   None             <none>        9090/TCP                     38s
service/springboot-web-demo                       ClusterIP   10.233.16.229    <none>        80/TCP                       5d19h
service/springboot-web-svc                        ClusterIP   None             <none>        80/TCP                       6d19h

NAME                                              DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/monitor-prometheus-node-exporter   2         2         2       2            2           <none>          8m44s

NAME                                                  READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/monitor-grafana                       0/1     1            0           8m44s
deployment.apps/monitor-kube-state-metrics            0/1     1            0           8m44s
deployment.apps/monitor-prometheus-operato-operator   0/1     1            0           8m44s

NAME                                                             DESIRED   CURRENT   READY   AGE
replicaset.apps/monitor-grafana-775969b799                       1         1         0       8m43s
replicaset.apps/monitor-kube-state-metrics-574788cc57            1         1         0       8m43s
replicaset.apps/monitor-prometheus-operato-operator-79458977df   1         1         0       8m43s

NAME                                                                    READY   AGE
statefulset.apps/alertmanager-monitor-prometheus-operato-alertmanager   0/1     48s
statefulset.apps/prometheus-monitor-prometheus-operato-prometheus       0/1     38s
[root@node1 ~]#

```

可以看到状态还是不对的，等待状态正常即可；

（3）长时间处于`ContainerCreating`状态；

> 在我这边的时候，有两个`pod`无法正常运行，分别是`alertmanager-monitor-prometheus-operato-alertmanager`和`prometheus-monitor-prometheus-operato-prometheus`，主要原因就是第一个`pod`的镜像无法下载，这个时候我们可以改一下这个镜像的下载地址，然后在更新一下；

更新`alertmanager-monitor-prometheus-operato-alertmanager`镜像地址；

```
[root@node1 ~]# cd prometheus-operator/
[root@node1 prometheus-operator]# vim values.yaml 
[root@node1 prometheus-operator]# 
# 原信息
image:
  repository: quay.io/prometheus/alertmanager
  tag: v0.21.0
  sha: ""
# 改成如下信息
image:
  repository: registry.cn-beijing.aliyuncs.com/yunweijia0909/alertmanager
  tag: v0.21.0
  sha: ""
[root@node1 prometheus-operator]# helm upgrade monitor -f values.yaml ../prometheus-operator/

# 或者采用如下方式，直接来修改这个pod的images地址，也可以
[root@node1 ~]# kubectl edit pod alertmanager-monitor-prometheus-operato-alertmanager-0
# 地址修改如下：
image: registry.cn-beijing.aliyuncs.com/yunweijia0909/alertmanager:v0.21.0

```

更新`prometheus-monitor-prometheus-operato-prometheus`，这里在上一步操作完毕之后，他就自动成功了；

最后状态如下：

```
[root@node1 ~]# kubectl get pod
NAME                                                     READY   STATUS    RESTARTS   AGE
alertmanager-monitor-prometheus-operato-alertmanager-0   2/2     Running   0          26m
monitor-grafana-775969b799-6tldl                         2/2     Running   2          3h26m
monitor-kube-state-metrics-574788cc57-r9gtf              1/1     Running   1          3h26m
monitor-prometheus-node-exporter-9pdwj                   1/1     Running   1          3h26m
monitor-prometheus-node-exporter-v9qhp                   1/1     Running   1          3h26m
monitor-prometheus-operato-operator-79458977df-5jgkb     2/2     Running   2          3h26m
prometheus-monitor-prometheus-operato-prometheus-0       3/3     Running   1          3h18m
[root@node1 ~]#

```

4、访问 Prometheus（普罗米修斯）
----------------------

```
[root@node1 ~]# kubectl get svc
NAME                                      TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
alertmanager-operated                     ClusterIP   None             <none>        9093/TCP,9094/TCP,9094/UDP   3h26m
kubernetes                                ClusterIP   10.233.0.1       <none>        443/TCP                      6d22h
monitor-grafana                           ClusterIP   10.233.10.12     <none>        80/TCP                       3h34m
monitor-kube-state-metrics                ClusterIP   10.233.199.229   <none>        8080/TCP                     3h34m
monitor-prometheus-node-exporter          ClusterIP   10.233.69.10     <none>        9100/TCP                     3h34m
monitor-prometheus-operato-alertmanager   ClusterIP   10.233.51.110    <none>        9093/TCP                     3h34m
monitor-prometheus-operato-operator       ClusterIP   10.233.209.238   <none>        8080/TCP,443/TCP             3h34m
monitor-prometheus-operato-prometheus     ClusterIP   10.233.40.187    <none>        9090/TCP                     3h34m
prometheus-operated                       ClusterIP   None             <none>        9090/TCP                     3h25m
springboot-web-demo                       ClusterIP   10.233.16.229    <none>        80/TCP                       5d22h
springboot-web-svc                        ClusterIP   None             <none>        80/TCP                       6d22h
[root@node1 ~]#

```

从上面可以看到我们需要的是`monitor-prometheus-operato-prometheus`，然后我们查看下这个服务的`yaml`文件；

```
[root@node1 ~]# kubectl get svc monitor-prometheus-operato-prometheus -o yaml
apiVersion: v1
kind: Service
metadata:
  annotations:
    meta.helm.sh/release-name: monitor
    meta.helm.sh/release-namespace: default
  creationTimestamp: "2022-05-03T10:08:32Z"
  labels:
    app: prometheus-operator-prometheus
    app.kubernetes.io/managed-by: Helm
    chart: prometheus-operator-9.3.2
    heritage: Helm
    release: monitor
    self-monitor: "true"
  managedFields:
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:meta.helm.sh/release-name: {}
          f:meta.helm.sh/release-namespace: {}
        f:labels:
          .: {}
          f:app: {}
          f:app.kubernetes.io/managed-by: {}
          f:chart: {}
          f:heritage: {}
          f:release: {}
          f:self-monitor: {}
      f:spec:
        f:ports:
          .: {}
          k:{"port":9090,"protocol":"TCP"}:
            .: {}
            f:name: {}
            f:port: {}
            f:protocol: {}
            f:targetPort: {}
        f:selector:
          .: {}
          f:app: {}
          f:prometheus: {}
        f:sessionAffinity: {}
        f:type: {}
    manager: helm
    operation: Update
    time: "2022-05-03T10:08:32Z"
  name: monitor-prometheus-operato-prometheus
  namespace: default
  resourceVersion: "783791"
  selfLink: /api/v1/namespaces/default/services/monitor-prometheus-operato-prometheus
  uid: 6e50f705-6bc2-4b4c-a46f-e43c53e1724a
spec:
  clusterIP: 10.233.40.187
  clusterIPs:
  - 10.233.40.187
  ports:
  - name: web
    port: 9090
    protocol: TCP
    targetPort: 9090
  selector:
    app: prometheus
    prometheus: monitor-prometheus-operato-prometheus
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}
[root@node1 ~]#

```

从上面可以看到这个服务的一些信息，比如`label`、`port`等；

我们要想访问的话，是不是还要使用`ingress-nginx`呢？我们来配置下；

```
[root@node1 ~]# mkdir namespace/prometheus
[root@node1 ~]# cd namespace/prometheus/
[root@node1 prometheus]# vim ingress-prometheus.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: prometheus
spec:
  rules:
  - host: prometheus.yunweijia.com
    http:
      paths:
      - backend:
          serviceName: monitor-prometheus-operato-prometheus
          servicePort: web
        path: /
[root@node1 prometheus]# 
[root@node1 prometheus]# kubectl apply -f ingress-prometheus.yaml 
ingress.extensions/prometheus created
[root@node1 prometheus]#

```

然后在`windows`客户端上修改下`hosts`文件，添加如下内容；

```
192.168.112.131 prometheus.yunweijia.com

```

浏览器直接访问如下地址：

```
http://prometheus.yunweijia.com

```

就出现了如下界面；

![](http://kmgy.top:9000/image/2022/10/17/202210171522286.jpeg)

至此，本文结束，下文我们将介绍下具体如何使用`prometheus`（普罗米修斯）。

![](http://kmgy.top:9000/image/2022/10/17/202210171522291.jpeg)