Kibana
Kibana是elasticsearch的可视化界面。
首先创建kibana的服务，yaml文件如下。k8s里的服务分为四种，clusterIP为仅仅为pod分配k8s集群内部的一个虚拟ip，用于集群内的pod通信，而不对外暴露。elasticsearch的服务就是clusterIP。NodePort是对外开放一个端口。所以kibana就是NordPort。其余还有LoadBalancer和ExternalName。
kibana部署代码
[root@master 31efk]# kubectl get svc -n kube-logging31
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
elasticsearch31 ClusterIP None <none> 9200/TCP,9300/TCP 37d
kibana31 NodePort 10.96.72.47 <none> 5601:30159/TCP 33m
[root@master 31efk]# cat kibana_svc.yaml
apiVersion: v1
kind: Service
metadata: name: kibana31 namespace: kube-logging31 labels: app: kibana
spec: type: NodePortports: - port: 5601 selector: app: kibana
Kibana Pod 通过 ELASTICSEARCH_URL 连接到 Elasticsearch 的 Service，这个 Service 会将流量路由到实际运行的 Elasticsearch Pod。这种通信是在集群内部通过 ClusterIP 完成的，Kibana 不需要知道 Elasticsearch Pod 的具体 IP 地址，只需通过 Service 的 DNS 名称访问它。
[root@master 31efk]# cat kibana_deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: kibana31namespace: kube-logging31labels:app: kibana
spec:replicas: 1selector:matchLabels:app: kibanatemplate:metadata:labels:app: kibanaspec:containers:- name: kibanaimage: docker.elastic.co/kibana/kibana:7.2.0imagePullPolicy: IfNotPresentresources:limits:cpu: 1000mrequests:cpu: 200menv:- name: ELASTICSEARCH_URLvalue: http://elasticsearch31.kube-logging31.svc.cluster.local:9200ports:- containerPort: 5601
调错
在执行的过程中，我们发现kinaba连接不到elasticsearch的服务，查看kibana pod的日志，发现kibana并没有使用我们yaml文件中定义的elasticsearch的服务名称，而是使用了默认的名称：
elasticsearch:9200
[root@master 31efk]# kubectl exec -it kibana31-859c9dcfb7-ptl9z -n kube-logging31 -- cat /usr/share/kibana/config/kibana.yml
#
# ** THIS IS AN AUTO-GENERATED FILE **
## Default Kibana configuration for docker target
server.name: kibana
server.host: "0"
elasticsearch.hosts: [ "http://elasticsearch:9200" ]
xpack.monitoring.ui.container.elasticsearch.enabled所以在yaml文件中，在ELASTICSEARCH_URL下面，再加一个环境变量：
name:ELASTICSEARCH_HOSTS
value: http://elasticsearch31.kube-logging31,svc.cluster.local:9200
kibana界面
kibana和elasticsearch连接成功
fluentd
fluentd部署代码
第一部分
首先对fluentd进行角色绑定：
apiVersion: v1
kind: ServiceAccount
metadata:name: fluentdnamespace: kube-logging31labels:app: fluentd
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:name: fluentdlabels:app: fluentd
rules:
- apiGroups:- ""resources:- pods- namespacesverbs:- get- list- watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: fluentd
roleRef:kind: ClusterRolename: fluentdapiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccountname: fluentdnamespace: kube-logging31
第二部分
使用deamonset来部署fluentd的pod，而且通过卷挂载的方式把每一个物理节点上的日志和容器日志收集起来：
path: /var/log
path: /var/lib/docker/containers
---
apiVersion: apps/v1
kind: DaemonSet
metadata:name: fluentdnamespace: kube-logging31labels:app: fluentd
spec:selector:matchLabels:app: fluentdtemplate:metadata:labels:app: fluentdspec:serviceAccount: fluentdserviceAccountName: fluentdtolerations:- key: node-role.kubernetes.io/mastereffect: NoSchedulecontainers:- name: fluentdimage: fluent/fluentd-kubernetes-daemonset:v1.4.2-debian-elasticsearch-1.1imagePullPolicy: IfNotPresentenv:- name: FLUENT_ELASTICSEARCH_HOSTvalue: "elasticsearch31.kube-logging31.svc.cluster.local"- name: FLUENT_ELASTICSEARCH_PORTvalue: "9200"- name: FLUENT_ELASTICSEARCH_SCHEMEvalue: "http"- name: FLUENTD_SYSTEMD_CONFvalue: disableresources:limits:memory: 512Mirequests:cpu: 100mmemory: 200MivolumeMounts:- name: varlogmountPath: /var/log- name: varlibdockercontainersmountPath: /var/lib/docker/containersreadOnly: trueterminationGracePeriodSeconds: 30volumes:- name: varloghostPath:path: /var/log- name: varlibdockercontainershostPath:path: /var/lib/docker/containers
部署结果
[root@master 31efk]# kubectl get pods -n kube-logging31
NAME READY STATUS RESTARTS AGE
elastic-cluster31-0 1/1 Running 0 3h31m
elastic-cluster31-1 1/1 Running 0 3h31m
elastic-cluster31-2 1/1 Running 0 3h31m
fluentd-8255m 1/1 Running 0 3m58s
fluentd-8rwbz 1/1 Running 0 3m58s
fluentd-sffhh 1/1 Running 0 3m58s
kibana31-8669689dbf-mvntd 1/1 Running 0 101m
kibana界面
此时经过一些配置，kibana的界面就可以显示log数据：
这里的索引为logstash，是因为fluentd默认情况下会将日志写入 Elasticsearch 中的索引名称与 Logstash 命名规则一致。
