Kube-static-metrics监控
kube-static-metrics组件可以通过监听apiserver生成有关资源对象的状态指标,比如Node、Pod,需要注意的是kube-state-metrics只是简单的提供一个metrics数据,并不会存储这些指标数据,所以可以使用Prometheus来抓取这些数据然后存储,主要关注的是业务相关的一些元数据,比如Pod副本状态等;调度了多少个replicas?现在可用的有几个?多少个Pod是running/stopped/terminated状态?Pod重启了多少次?有多少job在运行中。
安装kibe-state-metrics组件
# harbor创建仓库,装载metrics镜像,tag镜像名称,push镜像到harbor
docker load -i kube-state-metrics_1_9_0.tar.gz
docker tag quay.io/coreos/kube-state-metrics:v1.9.0 172.16.80.140/metrics/metrics:v1.9.0
docker push 172.16.80.140/metrics/metrics:v1.9.0# 创建sa,授权
vi kube-state-metrics-rbac.yaml
---
apiVersion: v1
kind: ServiceAccount
metadata:name: kube-state-metricsnamespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:name: kube-state-metrics
rules:
- apiGroups: [""]resources: ["nodes", "pods", "services", "resourcequotas", "replicationcontrollers", "limitranges", "persistentv
olumeclaims", "persistentvolumes", "namespaces", "endpoints"]verbs: ["list", "watch"]
- apiGroups: ["extensions"]resources: ["daemonsets", "deployments", "replicasets"]verbs: ["list", "watch"]
- apiGroups: ["apps"]resources: ["statefulsets"]verbs: ["list", "watch"]
- apiGroups: ["batch"]resources: ["cronjobs", "jobs"]verbs: ["list", "watch"]
- apiGroups: ["autoscaling"]resources: ["horizontalpodautoscalers"]verbs: ["list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: kube-state-metrics
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: kube-state-metrics
subjects:
- kind: ServiceAccountname: kube-state-metricsnamespace: kube-system# 创建deployment、svc
vi kube-state-metrics-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: kube-state-metricsnamespace: kube-system
spec:replicas: 1selector:matchLabels:app: kube-state-metricstemplate:metadata:labels:app: kube-state-metricsspec:serviceAccountName: kube-state-metricscontainers:- name: kube-state-metricsimage: 172.16.80.140/metrics/metrics:v1.9.0imagePullPolicy: IfNotPresentports:- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:annotations:prometheus.io/scrape: 'true'name: kube-state-metricsnamespace: kube-systemlabels:app: kube-state-metrics
spec:ports:- name: kube-state-metricsport: 8080protocol: TCPselector:app: kube-state-metrics[root@mast01 1]# kubectl get pods -A | grep metrics
kube-system kube-state-metrics-bf9b8785b-7fbrq 1/1 Running 0 25s
[root@mast01 1]# kubectl get svc -A | grep metrics
kube-system kube-state-metrics ClusterIP 10.108.11.25 <none> 8080/TCP 27s
granafa导入Kubernetes Cluster (Prometheus)-1577674936972,有的指标为N/A,可通过比对promethers修改指标校正