kubernet在prometheus+alertmanager+grafana框架下新增部署loki模块

整个框架拓扑图

1、Loki的安装部署

1）编写有关角色的yaml
root@iZj6c72dzbei17o2cuksmeZ:~/loki# cat LokiServiceAccount.yaml

---
apiVersion: v1
kind: ServiceAccount
metadata:name: loki-promtail-serviceaccountnamespace: monitoring---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: loki-promtail-clusterrolebindingnamespace: monitoring
subjects:- kind: ServiceAccountname: loki-promtail-serviceaccountnamespace: monitoring
roleRef:kind: ClusterRolename: loki-promtail-clusterroleapiGroup: rbac.authorization.k8s.io

2）创建loki-configmap

---
apiVersion: v1
data:loki.yaml: |-auth_enabled: falsechunk_store_config:max_look_back_period: 30dcompactor:shared_store: filesystemworking_directory: /data/loki/boltdb-shipper-compactorcompaction_interval: 60mretention_enabled: trueretention_delete_delay: 2hretention_delete_worker_count: 150ingester:chunk_block_size: 262144chunk_idle_period: 3mchunk_retain_period: 1mlifecycler:ring:kvstore:store: inmemoryreplication_factor: 1max_transfer_retries: 0wal:dir: /data/loki/wal          limits_config:enforce_metric_name: falsereject_old_samples: truereject_old_samples_max_age: 30dretention_period: 30dschema_config:configs:- from: "2020-10-24"index:period: 24hprefix: index_object_store: filesystemschema: v11store: boltdb-shipperserver:http_listen_port: 3100storage_config:boltdb_shipper:active_index_directory: /data/loki/boltdb-shipper-activecache_location: /data/loki/boltdb-shipper-cachecache_ttl: 24hshared_store: filesystemfilesystem:directory: /data/loki/chunkstable_manager:retention_deletes_enabled: trueretention_period: 30d
kind: ConfigMap
metadata:name: loki-configmapnamespace: monitoringresourceVersion: '17040827'

3）创建存储类yaml文件

---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:annotations:k8s.kuboard.cn/storageType: nfs_client_provisionername: cluster-storageresourceVersion: '17031467'
parameters:archiveOnDelete: 'false'
provisioner: nfs-cluster-storage
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer

4）创建loki的yaml文件

---
apiVersion: apps/v1
kind: StatefulSet
metadata:annotations: {}labels:app: lokik8s.kuboard.cn/addon: system-logsk8s.kuboard.cn/name: lokiname: lokinamespace: monitoringresourceVersion: '17042746'
spec:persistentVolumeClaimRetentionPolicy:whenDeleted: RetainwhenScaled: RetainpodManagementPolicy: OrderedReadyreplicas: 1revisionHistoryLimit: 10selector:matchLabels:app: lokik8s.kuboard.cn/addon: system-logsk8s.kuboard.cn/layer: monitorserviceName: lokitemplate:metadata:annotations:kubectl.kubernetes.io/restartedAt: '2025-03-27T16:07:47+08:00'prometheus.io/port: http-metricsprometheus.io/scrape: 'true'creationTimestamp: nulllabels:app: lokik8s.kuboard.cn/addon: system-logsk8s.kuboard.cn/layer: monitorspec:containers:- args:- '-config.file=/etc/loki/loki.yaml'image: 'swr.cn-east-2.myhuaweicloud.com/kuboard-dependency/loki:2.7.1'imagePullPolicy: IfNotPresentlivenessProbe:failureThreshold: 3httpGet:path: /readyport: http-metricsscheme: HTTPinitialDelaySeconds: 45periodSeconds: 10successThreshold: 1timeoutSeconds: 1name: lokiports:- containerPort: 3100name: http-metricsprotocol: TCPreadinessProbe:failureThreshold: 3httpGet:path: /readyport: http-metricsscheme: HTTPinitialDelaySeconds: 45periodSeconds: 10successThreshold: 1timeoutSeconds: 1resources: {}securityContext:readOnlyRootFilesystem: trueterminationMessagePath: /dev/termination-logterminationMessagePolicy: FilevolumeMounts:- mountPath: /etc/lokiname: config- mountPath: /dataname: storagednsPolicy: ClusterFirstrestartPolicy: AlwaysschedulerName: default-schedulersecurityContext:fsGroup: 10001runAsGroup: 10001runAsNonRoot: truerunAsUser: 10001serviceAccount: loki-promtail-serviceaccountserviceAccountName: loki-promtail-serviceaccountterminationGracePeriodSeconds: 4800volumes:- configMap:defaultMode: 420name: loki-configmapname: configupdateStrategy:type: RollingUpdatevolumeClaimTemplates:- apiVersion: v1kind: PersistentVolumeClaimmetadata:annotations:k8s.kuboard.cn/pvcType: DynamiccreationTimestamp: nullname: storagespec:accessModes:- ReadWriteManyresources:requests:storage: 5GistorageClassName: cluster-storagevolumeMode: Filesystemstatus:phase: Pending---
apiVersion: v1
kind: Service
metadata:annotations: {}labels:app: lokik8s.kuboard.cn/addon: system-logsk8s.kuboard.cn/name: lokiname: lokinamespace: monitoringresourceVersion: '17029178'
spec:clusterIP: 10.68.62.80clusterIPs:- 10.68.62.80internalTrafficPolicy: ClusteripFamilies:- IPv4ipFamilyPolicy: SingleStackports:- name: http-metricsport: 3100protocol: TCPtargetPort: http-metricsselector:app: lokik8s.kuboard.cn/addon: system-logsk8s.kuboard.cn/layer: monitorsessionAffinity: Nonetype: ClusterIP

5）启动部署deployment

root@iZj6c72dzbei17o2cuksmeZ:~/loki# kubectl apply -f LokiServiceAccount.yaml
root@iZj6c72dzbei17o2cuksmeZ:~/loki# kubectl apply -f cluster-storage.yaml
root@iZj6c72dzbei17o2cuksmeZ:~/loki# kubectl apply -f loki.yaml

2、Promtail的部署

1）编写serviceaccount文件
root@iZj6c72dzbei17o2cuksmeZ:~/loki# cat promtailServiceAccount.yaml

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:name: loki-promtail-clusterrolenamespace: monitoring
rules:- apiGroups: [""]resources:- nodes- services- podsverbs:- get- watch- list
---
apiVersion: v1
kind: ServiceAccount
metadata:name: loki-promtail-serviceaccountnamespace: monitoring---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: loki-promtail-clusterrolebindingnamespace: monitoring 
subjects:- kind: ServiceAccountname: loki-promtail-serviceaccountnamespace: monitoring 
roleRef:kind: ClusterRolename: loki-promtail-clusterroleapiGroup: rbac.authorization.k8s.io```2）编写promtail-configmap文件```yaml
---
apiVersion: v1
data:promtail.yaml: |-client:backoff_config:max_period: 5smax_retries: 20min_period: 100msbatchsize: 102400batchwait: 1sexternal_labels: {}timeout: 10spositions:filename: /run/promtail/positions.yamlserver:http_listen_port: 3101target_config:sync_period: 10sscrape_configs:- job_name: kubernetes-pods-namepipeline_stages:- docker: {}kubernetes_sd_configs:- role: podrelabel_configs:- source_labels:- __meta_kubernetes_pod_label_nametarget_label: __service__- source_labels:- __meta_kubernetes_pod_node_nametarget_label: __host__- action: dropregex: ''source_labels:- __service__- action: labelmapregex: __meta_kubernetes_pod_label_(.+)- action: replacereplacement: $1separator: /source_labels:- __meta_kubernetes_namespace- __service__target_label: job- action: replacesource_labels:- __meta_kubernetes_namespacetarget_label: namespace- action: replacesource_labels:- __meta_kubernetes_pod_nametarget_label: pod- action: replacesource_labels:- __meta_kubernetes_pod_container_nametarget_label: container- replacement: /var/log/pods/*$1/*.logseparator: /source_labels:- __meta_kubernetes_pod_uid- __meta_kubernetes_pod_container_nametarget_label: __path__- job_name: kubernetes-pods-apppipeline_stages:- docker: {}kubernetes_sd_configs:- role: podrelabel_configs:- action: dropregex: .+source_labels:- __meta_kubernetes_pod_label_name- source_labels:- __meta_kubernetes_pod_label_apptarget_label: __service__- source_labels:- __meta_kubernetes_pod_node_nametarget_label: __host__- action: dropregex: ''source_labels:- __service__- action: labelmapregex: __meta_kubernetes_pod_label_(.+)- action: replacereplacement: $1separator: /source_labels:- __meta_kubernetes_namespace- __service__target_label: job- action: replacesource_labels:- __meta_kubernetes_namespacetarget_label: namespace- action: replacesource_labels:- __meta_kubernetes_pod_nametarget_label: pod- action: replacesource_labels:- __meta_kubernetes_pod_container_nametarget_label: container- replacement: /var/log/pods/*$1/*.logseparator: /source_labels:- __meta_kubernetes_pod_uid- __meta_kubernetes_pod_container_nametarget_label: __path__- job_name: kubernetes-pods-direct-controllerspipeline_stages:- docker: {}kubernetes_sd_configs:- role: podrelabel_configs:- action: dropregex: .+separator: ''source_labels:- __meta_kubernetes_pod_label_name- __meta_kubernetes_pod_label_app- action: dropregex: '[0-9a-z-.]+-[0-9a-f]{8,10}'source_labels:- __meta_kubernetes_pod_controller_name- source_labels:- __meta_kubernetes_pod_controller_nametarget_label: __service__- source_labels:- __meta_kubernetes_pod_node_nametarget_label: __host__- action: dropregex: ''source_labels:- __service__- action: labelmapregex: __meta_kubernetes_pod_label_(.+)- action: replacereplacement: $1separator: /source_labels:- __meta_kubernetes_namespace- __service__target_label: job- action: replacesource_labels:- __meta_kubernetes_namespacetarget_label: namespace- action: replacesource_labels:- __meta_kubernetes_pod_nametarget_label: pod- action: replacesource_labels:- __meta_kubernetes_pod_container_nametarget_label: container- replacement: /var/log/pods/*$1/*.logseparator: /source_labels:- __meta_kubernetes_pod_uid- __meta_kubernetes_pod_container_nametarget_label: __path__- job_name: kubernetes-pods-indirect-controllerpipeline_stages:- docker: {}kubernetes_sd_configs:- role: podrelabel_configs:- action: dropregex: .+separator: ''source_labels:- __meta_kubernetes_pod_label_name- __meta_kubernetes_pod_label_app- action: keepregex: '[0-9a-z-.]+-[0-9a-f]{8,10}'source_labels:- __meta_kubernetes_pod_controller_name- action: replaceregex: '([0-9a-z-.]+)-[0-9a-f]{8,10}'source_labels:- __meta_kubernetes_pod_controller_nametarget_label: __service__- source_labels:- __meta_kubernetes_pod_node_nametarget_label: __host__- action: dropregex: ''source_labels:- __service__- action: labelmapregex: __meta_kubernetes_pod_label_(.+)- action: replacereplacement: $1separator: /source_labels:- __meta_kubernetes_namespace- __service__target_label: job- action: replacesource_labels:- __meta_kubernetes_namespacetarget_label: namespace- action: replacesource_labels:- __meta_kubernetes_pod_nametarget_label: pod- action: replacesource_labels:- __meta_kubernetes_pod_container_nametarget_label: container- replacement: /var/log/pods/*$1/*.logseparator: /source_labels:- __meta_kubernetes_pod_uid- __meta_kubernetes_pod_container_nametarget_label: __path__- job_name: kubernetes-pods-staticpipeline_stages:- docker: {}kubernetes_sd_configs:- role: podrelabel_configs:- action: dropregex: ''source_labels:- __meta_kubernetes_pod_annotation_kubernetes_io_config_mirror- action: replacesource_labels:- __meta_kubernetes_pod_label_componenttarget_label: __service__- source_labels:- __meta_kubernetes_pod_node_nametarget_label: __host__- action: dropregex: ''source_labels:- __service__- action: labelmapregex: __meta_kubernetes_pod_label_(.+)- action: replacereplacement: $1separator: /source_labels:- __meta_kubernetes_namespace- __service__target_label: job- action: replacesource_labels:- __meta_kubernetes_namespacetarget_label: namespace- action: replacesource_labels:- __meta_kubernetes_pod_nametarget_label: pod- action: replacesource_labels:- __meta_kubernetes_pod_container_nametarget_label: container- replacement: /var/log/pods/*$1/*.logseparator: /source_labels:- __meta_kubernetes_pod_annotation_kubernetes_io_config_mirror- __meta_kubernetes_pod_container_nametarget_label: __path__
kind: ConfigMap
metadata:name: promtail-configmapnamespace: monitoringresourceVersion: '17042330'

3）启动deployment

root@iZj6c72dzbei17o2cuksmeZ:~/loki# kubectl apply -f promtailServiceAccount.yaml
root@iZj6c72dzbei17o2cuksmeZ:~/loki# kubectl apply -f promtail.yaml

3、web界面操作

1）登录web界面，导入loki数据源
点击Data sources --> +Add new data source

选择loki项

自定义相关的参数项

滑倒最下面点击保存

2）自定义条件查看日志信息

3）导入Dashboards
选中Dashboards–>New–>import

输入模块id–>加载

自定义Dashboard名称–id–选择loki数据源

最后点击导入