docker-compose部署prometheus+grafana+node_exporter+alertmanager规则+邮件告警

目录

一.docker-compose文件

二.配置文件

三.文件层级关系，docker-compose和配置文件位于同级目录

四.node_exporter页面json文件

五.效果展示

prometheus+alertmanager+邮件告警

grafana面板效果

六.涉及离线包

---

 

一.docker-compose文件

[root@sulibao prometheus]# cat docker-compose.yml 
version: '3'services:prometheus:image: registry.cn-chengdu.aliyuncs.com/su03/prometheus:2.46.0-debian-11-r5container_name: prometheusports:- "9090:9090"volumes:- ./prometheus.yml:/etc/prometheus/prometheus.yml- ./rules.yml:/etc/prometheus/rules.ymlcommand:- '--config.file=/etc/prometheus/prometheus.yml'restart: alwaysalertmanager:image: registry.cn-chengdu.aliyuncs.com/su03/alertmanager:0.25.0-debian-11-r171container_name: alertmanagerports:- "9093:9093"volumes:- ./alertmanager.yml:/etc/alertmanager/alertmanager.ymlcommand:- '--config.file=/etc/alertmanager/alertmanager.yml'restart: alwaysgrafana:image: registry.cn-chengdu.aliyuncs.com/su03/grafana:9.3.6container_name: grafanaports:- "3000:3000"volumes:- grafana_data:/var/lib/grafana- ./dashboards:/usr/share/grafana/conf/provisioning/dashboardsenvironment:- GF_SECURITY_ADMIN_USER=admin- GF_SECURITY_ADMIN_PASSWORD=adminrestart: alwayspushgateway:image: registry.cn-chengdu.aliyuncs.com/su03/pushgateway:v1.6.2 container_name: pushgatewayports:- "9091:9091"restart: alwaysblackbox-exporter:image: registry.cn-chengdu.aliyuncs.com/su03/blackbox-exporter:0.25.0container_name: blackbox-exporterports:- "9115:9115"volumes:- ./blackbox.yml:/etc/blackbox_exporter/config.ymlcommand:- '--config.file=/etc/blackbox_exporter/config.yml'restart: alwaysnode-exporter:image: registry.cn-chengdu.aliyuncs.com/su03/node-exporter:1.6.1-debian-11-r8container_name: node-exporterports:- "9100:9100"restart: alwaysvolumes:grafana_data:    

二.配置文件

• prometheus.yml，监控和告警配置

[root@sulibao prometheus]# cat prometheus.yml 
global:scrape_interval: 15sevaluation_interval: 15sscrape_configs:- job_name: 'prometheus'static_configs:- targets: ['localhost:9090']- job_name: 'alertmanager'static_configs:- targets: ['alertmanager:9093']- job_name: 'pushgateway'static_configs:- targets: ['pushgateway:9091']- job_name: 'blackbox'metrics_path: /probeparams:module: [http_2xx]static_configs:- targets:- http://example.comrelabel_configs:- source_labels: [__address__]target_label: __param_target- source_labels: [__param_target]target_label: instance- target_label: __address__replacement: blackbox-exporter:9115- job_name: 'node-exporter'static_configs:- targets: ['node-exporter:9100']alerting:alertmanagers:- static_configs:- targets: ['alertmanager:9093']rule_files:- "rules.yml"

• blackbox.yml，黑盒测试文件

[root@sulibao prometheus]# cat blackbox.yml 
modules:http_2xx:prober: httptimeout: 5shttp:valid_status_codes: [200]

• alertmanager.yml，告警通知配置文件

[root@sulibao prometheus]# cat alertmanager.yml 
global:resolve_timeout: 5msmtp_smarthost: 'smtp.qq.com:465'   #邮件服务器地址smtp_from: 'xxx'                #邮件发送地址smtp_auth_username: 'xxx'       #邮箱认证用户地址，一般和smtp_from一致smtp_auth_password: 'xxx'          #邮箱授权码smtp_require_tls: falseroute:group_by: ['alertname']group_wait: 30sgroup_interval: 5mrepeat_interval: 12hreceiver: 'test-alertmanager'receivers:
- name: 'test-alertmanager'email_configs:- to: 'xxx'               #告警接收邮箱地址

•   rules.yml，告警规则配置文件

[root@sulibao prometheus]# cat rules.yml 
groups:- name: test-rulesrules:- alert: HighCPUUsageexpr: avg by (instance) (rate(node_cpu_seconds_total{mode!="idle"}[5m])) > 0.8for: 5mlabels:severity: warningannotations:summary: "High CPU usage on {{ $labels.instance }}"description: "{{ $labels.instance }} has a CPU usage of {{ $value }}"

三.文件层级关系，docker-compose和配置文件位于同级目录

[root@sulibao prometheus]# ll
total 20
-rw-r--r-- 1 root root  421 Mar 31 13:39 alertmanager.yml
-rw-r--r-- 1 root root  100 Mar 30 18:28 blackbox.yml
-rw-r--r-- 1 root root 1793 Mar 31 13:25 docker-compose.yml
-rw-r--r-- 1 root root  945 Mar 31 13:21 prometheus.yml
-rw-r--r-- 1 root root  378 Mar 31 13:48 rules.yml
[root@sulibao prometheus]# pwd
/data/prometheus

四.node_exporter页面json文件

{"__inputs": [{"name": "DS__VICTORIAMETRICS","label": " VictoriaMetrics","description": "","type": "datasource","pluginId": "prometheus","pluginName": "Prometheus"}],"__requires": [{"type": "panel","id": "bargauge","name": "Bar gauge","version": ""},{"type": "grafana","id": "grafana","name": "Grafana","version": "7.2.0"},{"type": "panel","id": "graph","name": "Graph","version": ""},{"type": "datasource","id": "prometheus","name": "Prometheus","version": "1.0.0"},{"type": "panel","id": "stat","name": "Stat","version": ""},{"type": "panel","id": "table-old","name": "Table (old)","version": ""}],"annotations": {"list": [{"$$hashKey": "object:2875","builtIn": 1,"datasource": "-- Grafana --","enable": true,"hide": true,"iconColor": "rgba(0, 211, 255, 1)","name": "Annotations & Alerts","type": "dashboard"}]},"description": "服务器监控","editable": true,"gnetId": 11074,"graphTooltip": 0,"id": null,"iteration": 1602304673501,"links": [{"$$hashKey": "object:2300","icon": "bolt","tags": [],"targetBlank": true,"title": "Update","tooltip": "Update dashboard","type": "link","url": "https://grafana.com/grafana/dashboards/11074"},{"$$hashKey": "object:2301","icon": "question","tags": [],"targetBlank": true,"title": "GitHub","tooltip": "more dashboard","type": "link","url": "https://github.com/starsliao"},{"$$hashKey": "object:2302","asDropdown": true,"icon": "external link","tags": [],"targetBlank": true,"title": "","type": "dashboards"}],"panels": [{"collapsed": false,"datasource": "${DS__VICTORIAMETRICS}","gridPos": {"h": 1,"w": 24,"x": 0,"y": 0},"id": 187,"panels": [],"title": "Resource Overview (associated JOB)，Host：$show_hostname，Instance：$node","type": "row"},{"columns": [],"datasource": "${DS__VICTORIAMETRICS}","description": "Partition utilization, disk read, disk write, download bandwidth, upload bandwidth, if there are multiple network cards or multiple partitions, it is the value of the network card or partition with the highest utilization rate collected.\n\nCurrEstab: The number of TCP connections whose current status is ESTABLISHED or CLOSE-WAIT.","fieldConfig": {"defaults": {"custom": {}},"overrides": []},"fontSize": "80%","gridPos": {"h": 7,"w": 24,"x": 0,"y": 1},"id": 185,"pageSize": null,"showHeader": true,"sort": {"col": 31,"desc": false},"styles": [{"$$hashKey": "object:1600","alias": "Hostname","align": "auto","colorMode": null,"colors": ["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat": "YYYY-MM-DD HH:mm:ss","decimals": 1,"link": false,"linkTooltip": "","linkUrl": "","mappingType": 1,"pattern": "nodename","thresholds": [],"type": "string","unit": "bytes"},{"$$hashKey": "object:1601","alias": "IP（Link to details）","align": "auto","colorMode": null,"colors": ["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat": "YYYY-MM-DD HH:mm:ss","decimals": 2,"link": true,"linkTargetBlank": false,"linkTooltip": "Browse host details","linkUrl": "d/xfpJB9FGz/node-exporter?orgId=1&var-job=${job}&var-hostname=All&var-node=${__cell}&var-device=All&var-origin_prometheus=$origin_prometheus","mappingType": 1,"pattern": "instance","thresholds": [],"type": "number","unit": "short"},{"$$hashKey": "object:1602","alias": "Memory","align": "auto","colorMode": null,"colors": ["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat": "YYYY-MM-DD HH:mm:ss","decimals": 2,"link": false,"mappingType": 1,"pattern": "Value #B","thresholds": [],"type": "number","unit": "bytes"},{"$$hashKey": "object:1603","alias": "CPU Cores","align": "auto","colorMode": null,"colors": ["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat": "YYYY-MM-DD HH:mm:ss","decimals": null,"mappingType": 1,"pattern": "Value #C","thresholds": [],"type": "number","unit": "short"},{"$$hashKey": "object:1604","alias": " Uptime","align": "auto","colorMode": null,"colors": ["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat": "YYYY-MM-DD HH:mm:ss","decimals": 2,"mappingType": 1,"pattern": "Value #D","thresholds": [],"type": "number","unit": "s"},{"$$hashKey": "object:1605","alias": "Partition used%*","align": "auto","colorMode": "cell","colors": ["rgba(50, 172, 45, 0.97)","rgba(237, 129, 40, 0.89)","rgba(245, 54, 54, 0.9)"],"dateFormat": "YYYY-MM-DD HH:mm:ss","decimals": 2,"mappingType": 1,"pattern": "Value #E","thresholds": ["70","85"],"type": "number","unit": "percent"},{"$$hashKey": "object:1606","alias": "CPU used%","align": "auto","colorMode": "cell","colors": ["rgba(50, 172, 45, 0.97)","rgba(237, 129, 40, 0.89)","rgba(245, 54, 54, 0.9)"],"dateFormat": "YYYY-MM-DD HH:mm:ss","decimals": 2,"mappingType": 1,"pattern": "Value #F","thresholds": ["70","85"],"type": "number","unit": "percent"},{"$$hashKey": "object:1607","alias": "Memory used%","align": "auto","colorMode": "cell","colors": ["rgba(50, 172, 45, 0.97)","rgba(237, 129, 40, 0.89)","rgba(245, 54, 54, 0.9)"],"dateFormat": "YYYY-MM-DD HH:mm:ss","decimals": 2,"mappingType": 1,"pattern": "Value #G","thresholds": ["70","85"],"type": "number","unit": "percent"},{"$$hashKey": "object:1608","alias": "Disk read*","align": "auto","colorMode": "cell","colors": ["rgba(50, 172, 45, 0.97)","rgba(237, 129, 40, 0.89)","rgba(245, 54, 54, 0.9)"],"dateFormat": "YYYY-MM-DD HH:mm:ss","decimals": 2,"mappingType": 1,"pattern": "Value #H","thresholds": ["10485760","20485760"],"type": "number","unit": "Bps"},{"$$hashKey": "object:1609","alias": "Disk write*","align": "auto","colorMode": "cell","colors": ["rgba(50, 172, 45, 0.97)","rgba(237, 129, 40, 0.89)","rgba(245, 54, 54, 0.9)"],"dateFormat": "YYYY-MM-DD HH:mm:ss","decimals": 2,"mappingType": 1,"pattern": "Value #I","thresholds": ["10485760","20485760"],"type": "number","unit": "Bps"},{"$$hashKey": "object:1610","alias": "Download*","align": "auto","colorMode": "cell","colors": ["rgba(50, 172, 45, 0.97)","rgba(237, 129, 40, 0.89)","rgba(245, 54, 54, 0.9)"],"dateFormat": "YYYY-MM-DD HH:mm:ss","decimals": 2,"mappingType": 1,"pattern": "Value #J","thresholds": ["30485760","104857600"],"type": "number","unit": "bps"},{"$$hashKey": "object:1611","alias": "Upload*","align": "auto","colorMode": "cell","colors": ["rgba(50, 172, 45, 0.97)","rgba(237, 129, 40, 0.89)","rgba(245, 54, 54, 0.9)"],"dateFormat": "YYYY-MM-DD HH:mm:ss","decimals": 2,"mappingType": 1,"pattern": "Value #K","thresholds": ["30485760","104857600"],"type": "number","unit": "bps"},{"$$hashKey": "object:1612","alias": "5m load","align": "auto","colorMode": null,"colors": ["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat": "YYYY-MM-DD HH:mm:ss","decimals": 2,"mappingType": 1,"pattern": "Value #L","thresholds": [],"type": "number","unit": "short"},{"$$hashKey": "object:1613","alias": "CurrEstab","align": "auto","colorMode": "cell","colors": ["rgba(50, 172, 45, 0.97)","rgba(237, 129, 40, 0.89)","rgba(245, 54, 54, 0.9)"],"dateFormat": "YYYY-MM-DD HH:mm:ss","decimals": 2,"mappingType": 1,"pattern": "Value #M","thresholds": ["1000","1500"],"type": "string","unit": "short"},{"$$hashKey": "object:1614","alias": "TCP_tw","align": "center","colorMode": "cell","colors": ["rgba(50, 172, 45, 0.97)","rgba(237, 129, 40, 0.89)","rgba(245, 54, 54, 0.9)"],"dateFormat": "YYYY-MM-DD HH:mm:ss","decimals": null,"mappingType": 1,"pattern": "Value #N","thresholds": ["5000","20000"],"type": "number","unit": "short"},{"$$hashKey": "object:1615","alias": "","align": "right","colorMode": null,"colors": ["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"decimals": 2,"pattern": "/.*/","thresholds": [],"type": "hidden","unit": "short"}],"targets": [{"expr": "node_uname_info{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"} - 0","format": "table","instant": true,"interval": "","legendFormat": "主机名","refId": "A"},{"expr": "sum(time() - node_boot_time_seconds{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"})by(instance)","format": "table","hide": false,"instant": true,"interval": "","legendFormat": "运行时间","refId": "D"},{"expr": "node_memory_MemTotal_bytes{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"} - 0","format": "table","hide": false,"instant": true,"interval": "","legendFormat": "总内存","refId": "B"},{"expr": "count(node_cpu_seconds_total{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\",mode='system'}) by (instance)","format": "table","hide": false,"instant": true,"interval": "","legendFormat": "总核数","refId": "C"},{"expr": "node_load5{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"}","format": "table","instant": true,"interval": "","legendFormat": "5分钟负载","refId": "L"},{"expr": "(1 - avg(rate(node_cpu_seconds_total{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\",mode=\"idle\"}[$interval])) by (instance)) * 100","format": "table","hide": false,"instant": true,"interval": "","legendFormat": "CPU使用率","refId": "F"},{"expr": "(1 - (node_memory_MemAvailable_bytes{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"} / (node_memory_MemTotal_bytes{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"})))* 100","format": "table","hide": false,"instant": true,"interval": "","legendFormat": "内存使用率","refId": "G"},{"expr": "max((node_filesystem_size_bytes{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\",fstype=~\"ext.?|xfs\"}-node_filesystem_free_bytes{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\",fstype=~\"ext.?|xfs\"}) *100/(node_filesystem_avail_bytes {origin_prometheus=~\"$origin_prometheus\",job=~\"$job\",fstype=~\"ext.?|xfs\"}+(node_filesystem_size_bytes{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\",fstype=~\"ext.?|xfs\"}-node_filesystem_free_bytes{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\",fstype=~\"ext.?|xfs\"})))by(instance)","format": "table","hide": false,"instant": true,"interval": "","legendFormat": "分区使用率","refId": "E"},{"expr": "max(rate(node_disk_read_bytes_total{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"}[$interval])) by (instance)","format": "table","hide": false,"instant": true,"interval": "","legendFormat": "最大读取","refId": "H"},{"expr": "max(rate(node_disk_written_bytes_total{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"}[$interval])) by (instance)","format": "table","hide": false,"instant": true,"interval": "","legendFormat": "最大写入","refId": "I"},{"expr": "node_netstat_Tcp_CurrEstab{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"} - 0","format": "table","hide": false,"instant": true,"interval": "","legendFormat": "连接数","refId": "M"},{"expr": "node_sockstat_TCP_tw{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"} - 0","format": "table","hide": false,"instant": true,"interval": "","legendFormat": "TIME_WAIT","refId": "N"},{"expr": "max(rate(node_network_receive_bytes_total{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"}[$interval])*8) by (instance)","format": "table","hide": false,"instant": true,"interval": "","legendFormat": "下载带宽","refId": "J"},{"expr": "max(rate(node_network_transmit_bytes_total{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"}[$interval])*8) by (instance)","format": "table","hide": false,"instant": true,"interval": "","legendFormat": "上传带宽","refId": "K"}],"timeFrom": null,"timeShift": null,"title": "Server Resource Overview【JOB：$job，Total：$total】","transform": "table","type": "table-old"},{"aliasColors": {"192.168.200.241:9100_Total": "dark-red","Idle - Waiting for something to happen": "#052B51","guest": "#9AC48A","idle": "#052B51","iowait": "#EAB839","irq": "#BF1B00","nice": "#C15C17","sdb_每秒I/O操作%": "#d683ce","softirq": "#E24D42","steal": "#FCE2DE","system": "#508642","user": "#5195CE","磁盘花费在I/O操作占比": "#ba43a9"},"bars": false,"dashLength": 10,"dashes": false,"datasource": "${DS__VICTORIAMETRICS}","decimals": null,"description": "","fieldConfig": {"defaults": {"custom": {},"links": []},"overrides": []},"fill": 0,"fillGradient": 0,"gridPos": {"h": 7,"w": 8,"x": 0,"y": 8},"hiddenSeries": false,"id": 191,"legend": {"alignAsTable": false,"avg": false,"current": true,"hideEmpty": true,"hideZero": true,"max": false,"min": false,"rightSide": false,"show": true,"sideWidth": null,"sort": "current","sortDesc": true,"total": false,"values": true},"lines": true,"linewidth": 2,"links": [],"maxPerRow": 6,"nullPointMode": "null","options": {"alertThreshold": true},"percentage": false,"pluginVersion": "7.2.0","pointradius": 5,"points": false,"renderer": "flot","repeat": null,"seriesOverrides": [{"$$hashKey": "object:2312","alias": "Overall average used%","lines": false,"pointradius": 1,"points": true,"yaxis": 2},{"$$hashKey": "object:2313","alias": "CPU Cores","color": "#C4162A"}],"spaceLength": 10,"stack": false,"steppedLine": false,"targets": [{"expr": "count(node_cpu_seconds_total{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\", mode='system'})","format": "time_series","hide": false,"instant": false,"interval": "30m","intervalFactor": 1,"legendFormat": "CPU Cores","refId": "B","step": 240},{"expr": "sum(node_load5{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"})","format": "time_series","hide": false,"interval": "30m","intervalFactor": 1,"legendFormat": "Total 5m load","refId": "A","step": 240},{"expr": "avg(1 - avg(rate(node_cpu_seconds_total{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\",mode=\"idle\"}[$interval])) by (instance)) * 100","format": "time_series","hide": false,"interval": "30m","intervalFactor": 1,"legendFormat": "Overall average used%","refId": "F","step": 240}],"thresholds": [],"timeFrom": null,"timeRegions": [],"timeShift": null,"title": "$job：Overall total 5m load & average CPU used%","tooltip": {"shared": true,"sort": 2,"value_type": "individual"},"type": "graph","xaxis": {"buckets": null,"mode": "time","name": null,"show": true,"values": []},"yaxes": [{"$$hashKey": "object:8791","decimals": null,"format": "short","label": "Total 5m load","logBase": 1,"max": null,"min": null,"show": true},{"$$hashKey": "object:8792","decimals": 0,"format": "percent","label": "Overall average used%","logBase": 1,"max": null,"min": null,"show": true}],"yaxis": {"align": false,"alignLevel": null}},{"aliasColors": {"192.168.200.241:9100_总内存": "dark-red","内存_Avaliable": "#6ED0E0","内存_Cached": "#EF843C","内存_Free": "#629E51","内存_Total": "#6d1f62","内存_Used": "#eab839","可用": "#9ac48a","总内存": "#bf1b00"},"bars": false,"dashLength": 10,"dashes": false,"datasource": "${DS__VICTORIAMETRICS}","decimals": 1,"fieldConfig": {"defaults": {"custom": {},"links": []},"overrides": []},"fill": 0,"fillGradient": 0,"gridPos": {"h": 7,"w": 8,"x": 8,"y": 8},"height": "300","hiddenSeries": false,"id": 195,"legend": {"alignAsTable": false,"avg": false,"current": true,"max": false,"min": false,"rightSide": false,"show": true,"sort": "current","sortDesc": false,"total": false,"values": true},"lines": true,"linewidth": 2,"links": [],"nullPointMode": "null","options": {"alertThreshold": true},"percentage": false,"pluginVersion": "7.2.0","pointradius": 5,"points": false,"renderer": "flot","seriesOverrides": [{"$$hashKey": "object:2494","alias": "Total","color": "#C4162A","fill": 0},{"$$hashKey": "object:2495","alias": "Overall Average Used%","lines": false,"pointradius": 1,"points": true,"yaxis": 2}],"spaceLength": 10,"stack": false,"steppedLine": false,"targets": [{"expr": "sum(node_memory_MemTotal_bytes{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"})","format": "time_series","hide": false,"instant": false,"interval": "30m","intervalFactor": 1,"legendFormat": "Total","refId": "A","step": 4},{"expr": "sum(node_memory_MemTotal_bytes{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"} - node_memory_MemAvailable_bytes{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"})","format": "time_series","hide": false,"interval": "30m","intervalFactor": 1,"legendFormat": "Total Used","refId": "B","step": 4},{"expr": "(sum(node_memory_MemTotal_bytes{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"} - node_memory_MemAvailable_bytes{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"}) / sum(node_memory_MemTotal_bytes{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"}))*100","format": "time_series","hide": false,"interval": "30m","intervalFactor": 1,"legendFormat": "Overall Average Used%","refId": "H"}],"thresholds": [],"timeFrom": null,"timeRegions": [],"timeShift": null,"title": "$job：Overall total memory & average memory used%","tooltip": {"shared": true,"sort": 2,"value_type": "individual"},"type": "graph","xaxis": {"buckets": null,"mode": "time","name": null,"show": true,"values": []},"yaxes": [{"$$hashKey": "object:8938","decimals": null,"format": "bytes","label": "Total","logBase": 1,"max": null,"min": "0","show": true},{"$$hashKey": "object:8939","decimals": null,"format": "percent","label": "Overall Average Used%","logBase": 1,"max": null,"min": null,"show": true}],"yaxis": {"align": false,"alignLevel": null}},{"aliasColors": {},"bars": false,"dashLength": 10,"dashes": false,"datasource": "${DS__VICTORIAMETRICS}","decimals": 1,"description": "","fieldConfig": {"defaults": {"custom": {},"links": []},"overrides": []},"fill": 0,"fillGradient": 0,"gridPos": {"h": 7,"w": 8,"x": 16,"y": 8},"hiddenSeries": false,"id": 197,"legend": {"alignAsTable": false,"avg": false,"current": true,"hideEmpty": false,"hideZero": false,"max": false,"min": false,"rightSide": false,"show": true,"sideWidth": null,"sort": "current","sortDesc": true,"total": false,"values": true},"lines": true,"linewidth": 2,"links": [],"nullPointMode": "null","options": {"alertThreshold": true},"percentage": false,"pluginVersion": "7.2.0","pointradius": 5,"points": false,"renderer": "flot","seriesOverrides": [{"$$hashKey": "object:2617","alias": "Overall Average Used%","lines": false,"pointradius": 1,"points": true,"yaxis": 2},{"$$hashKey": "object:2618","alias": "Total","color": "#C4162A"}],"spaceLength": 10,"stack": false,"steppedLine": false,"targets": [{"expr": "sum(avg(node_filesystem_size_bytes{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\",fstype=~\"xfs|ext.*\"})by(device,instance))","format": "time_series","instant": false,"interval": "30m","intervalFactor": 1,"legendFormat": "Total","refId": "E"},{"expr": "sum(avg(node_filesystem_size_bytes{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\",fstype=~\"xfs|ext.*\"})by(device,instance)) - sum(avg(node_filesystem_free_bytes{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\",fstype=~\"xfs|ext.*\"})by(device,instance))","format": "time_series","instant": false,"interval": "30m","intervalFactor": 1,"legendFormat": "Total Used","refId": "C"},{"expr": "(sum(avg(node_filesystem_size_bytes{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\",fstype=~\"xfs|ext.*\"})by(device,instance)) - sum(avg(node_filesystem_free_bytes{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\",fstype=~\"xfs|ext.*\"})by(device,instance))) *100/(sum(avg(node_filesystem_avail_bytes{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\",fstype=~\"xfs|ext.*\"})by(device,instance))+(sum(avg(node_filesystem_size_bytes{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\",fstype=~\"xfs|ext.*\"})by(device,instance)) - sum(avg(node_filesystem_free_bytes{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\",fstype=~\"xfs|ext.*\"})by(device,instance))))","format": "time_series","instant": false,"interval": "30m","intervalFactor": 1,"legendFormat": "Overall Average Used%","refId": "A"}],"thresholds": [],"timeFrom": null,"timeRegions": [],"timeShift": null,"title": "$job：Overall total disk & average disk used%","tooltip": {"shared": true,"sort": 2,"value_type": "individual"},"type": "graph","xaxis": {"buckets": null,"mode": "time","name": null,"show": true,"values": []},"yaxes": [{"$$hashKey": "object:8990","decimals": 1,"format": "bytes","label": "Total","logBase": 1,"max": null,"min": "0","show": true},{"$$hashKey": "object:8991","decimals": null,"format": "percent","label": "Overall Average Used%","logBase": 1,"max": null,"min": null,"show": true}],"yaxis": {"align": false,"alignLevel": null}},{"collapsed": false,"datasource": "${DS__VICTORIAMETRICS}","gridPos": {"h": 1,"w": 24,"x": 0,"y": 15},"id": 189,"panels": [],"title": "Resource Details：【$show_hostname】","type": "row"},{"cacheTimeout": null,"datasource": "${DS__VICTORIAMETRICS}","description": "","fieldConfig": {"defaults": {"custom": {},"decimals": 0,"mappings": [{"id": 0,"op": "=","text": "N/A","type": 1,"value": "null"}],"nullValueMode": "null","thresholds": {"mode": "absolute","steps": [{"color": "rgba(245, 54, 54, 0.9)","value": null},{"color": "rgba(237, 129, 40, 0.89)","value": 1},{"color": "rgba(50, 172, 45, 0.97)","value": 3}]},"unit": "s"},"overrides": []},"gridPos": {"h": 2,"w": 2,"x": 0,"y": 16},"hideTimeOverride": true,"id": 15,"interval": null,"links": [],"maxDataPoints": 100,"options": {"colorMode": "value","graphMode": "none","justifyMode": "auto","orientation": "horizontal","reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": false},"textMode": "auto"},"pluginVersion": "7.2.0","targets": [{"expr": "avg(time() - node_boot_time_seconds{instance=~\"$node\"})","format": "time_series","hide": false,"instant": true,"interval": "","intervalFactor": 1,"legendFormat": "","refId": "A","step": 40}],"title": "Uptime","type": "stat"},{"datasource": "${DS__VICTORIAMETRICS}","fieldConfig": {"defaults": {"color": {"mode": "thresholds"},"custom": {},"decimals": 1,"mappings": [{"from": "","id": 1,"operator": "","text": "N/A","to": "","type": 1,"value": "0"}],"max": 100,"min": 0.1,"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "#EAB839","value": 70},{"color": "red","value": 90}]},"unit": "percent"},"overrides": []},"gridPos": {"h": 6,"w": 3,"x": 2,"y": 16},"id": 177,"options": {"displayMode": "lcd","orientation": "horizontal","reduceOptions": {"calcs": ["last"],"fields": "","values": false},"showUnfilled": true},"pluginVersion": "7.2.0","targets": [{"expr": "100 - (avg(rate(node_cpu_seconds_total{instance=~\"$node\",mode=\"idle\"}[$interval])) * 100)","instant": true,"interval": "","legendFormat": "CPU Busy","refId": "A"},{"expr": "avg(rate(node_cpu_seconds_total{instance=~\"$node\",mode=\"iowait\"}[$interval])) * 100","hide": true,"instant": true,"interval": "","legendFormat": "IOwait使用率","refId": "C"},{"expr": "(1 - (node_memory_MemAvailable_bytes{instance=~\"$node\"} / (node_memory_MemTotal_bytes{instance=~\"$node\"})))* 100","instant": true,"interval": "","legendFormat": "Used RAM Memory","refId": "B"},{"expr": "(node_filesystem_size_bytes{instance=~'$node',fstype=~\"ext.*|xfs\",mountpoint=\"$maxmount\"}-node_filesystem_free_bytes{instance=~'$node',fstype=~\"ext.*|xfs\",mountpoint=\"$maxmount\"})*100 /(node_filesystem_avail_bytes {instance=~'$node',fstype=~\"ext.*|xfs\",mountpoint=\"$maxmount\"}+(node_filesystem_size_bytes{instance=~'$node',fstype=~\"ext.*|xfs\",mountpoint=\"$maxmount\"}-node_filesystem_free_bytes{instance=~'$node',fstype=~\"ext.*|xfs\",mountpoint=\"$maxmount\"}))","hide": false,"instant": true,"interval": "","legendFormat": "Used Max Mount($maxmount)","refId": "D"},{"expr": "(1 - ((node_memory_SwapFree_bytes{instance=~\"$node\"} + 1)/ (node_memory_SwapTotal_bytes{instance=~\"$node\"} + 1))) * 100","instant": true,"interval": "","legendFormat": "Used SWAP","refId": "F"}],"timeFrom": null,"timeShift": null,"title": "","transformations": [],"type": "bargauge"},{"columns": [],"datasource": "${DS__VICTORIAMETRICS}","description": "In this kanban: the total disk, usage, available, and usage rate are consistent with the values of the Size, Used, Avail, and Use% columns of the df command, and the value of Use% will be rounded to one decimal place, which will be more accurate .\n\nNote: The Use% algorithm in df is: (size-free) * 100 / (avail + (size-free)), the result is divisible by this value, non-divisible by this value is +1, and the unit of the result is %.\nRefer to the df command source code:","fieldConfig": {"defaults": {"custom": {}},"overrides": []},"fontSize": "80%","gridPos": {"h": 6,"w": 10,"x": 5,"y": 16},"id": 181,"links": [{"targetBlank": true,"title": "https://github.com/coreutils/coreutils/blob/master/src/df.c","url": "https://github.com/coreutils/coreutils/blob/master/src/df.c"}],"pageSize": null,"scroll": true,"showHeader": true,"sort": {"col": 6,"desc": false},"styles": [{"$$hashKey": "object:307","alias": "Mounted on","align": "auto","colorMode": null,"colors": ["rgba(50, 172, 45, 0.97)","rgba(237, 129, 40, 0.89)","rgba(245, 54, 54, 0.9)"],"dateFormat": "YYYY-MM-DD HH:mm:ss","decimals": 2,"mappingType": 1,"pattern": "mountpoint","thresholds": [""],"type": "string","unit": "bytes"},{"$$hashKey": "object:308","alias": "Avail","align": "auto","colorMode": "value","colors": ["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat": "YYYY-MM-DD HH:mm:ss","decimals": 1,"mappingType": 1,"pattern": "Value #A","thresholds": ["10000000000","20000000000"],"type": "number","unit": "bytes"},{"$$hashKey": "object:309","alias": "Used","align": "auto","colorMode": "cell","colors": ["rgba(50, 172, 45, 0.97)","rgba(237, 129, 40, 0.89)","rgba(245, 54, 54, 0.9)"],"dateFormat": "YYYY-MM-DD HH:mm:ss","decimals": 1,"mappingType": 1,"pattern": "Value #B","thresholds": ["70","85"],"type": "number","unit": "percent"},{"$$hashKey": "object:310","alias": "Size","align": "auto","colorMode": null,"colors": ["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat": "YYYY-MM-DD HH:mm:ss","decimals": 0,"link": false,"mappingType": 1,"pattern": "Value #C","thresholds": [],"type": "number","unit": "bytes"},{"$$hashKey": "object:311","alias": "Filesystem","align": "auto","colorMode": null,"colors": ["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat": "YYYY-MM-DD HH:mm:ss","decimals": 2,"link": false,"mappingType": 1,"pattern": "fstype","thresholds": [],"type": "string","unit": "short"},{"$$hashKey": "object:312","alias": "Device","align": "auto","colorMode": null,"colors": ["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"dateFormat": "YYYY-MM-DD HH:mm:ss","decimals": 2,"link": false,"mappingType": 1,"pattern": "device","preserveFormat": false,"sanitize": false,"thresholds": [],"type": "string","unit": "short"},{"$$hashKey": "object:313","alias": "","align": "auto","colorMode": null,"colors": ["rgba(245, 54, 54, 0.9)","rgba(237, 129, 40, 0.89)","rgba(50, 172, 45, 0.97)"],"decimals": 2,"pattern": "/.*/","preserveFormat": true,"sanitize": false,"thresholds": [],"type": "hidden","unit": "short"}],"targets": [{"expr": "node_filesystem_size_bytes{instance=~'$node',fstype=~\"ext.*|xfs\",mountpoint !~\".*pod.*\"}-0","format": "table","hide": false,"instant": true,"interval": "","intervalFactor": 1,"legendFormat": "总量","refId": "C"},{"expr": "node_filesystem_avail_bytes {instance=~'$node',fstype=~\"ext.*|xfs\",mountpoint !~\".*pod.*\"}-0","format": "table","hide": false,"instant": true,"interval": "10s","intervalFactor": 1,"legendFormat": "","refId": "A"},{"expr": "(node_filesystem_size_bytes{instance=~'$node',fstype=~\"ext.*|xfs\",mountpoint !~\".*pod.*\"}-node_filesystem_free_bytes{instance=~'$node',fstype=~\"ext.*|xfs\",mountpoint !~\".*pod.*\"}) *100/(node_filesystem_avail_bytes {instance=~'$node',fstype=~\"ext.*|xfs\",mountpoint !~\".*pod.*\"}+(node_filesystem_size_bytes{instance=~'$node',fstype=~\"ext.*|xfs\",mountpoint !~\".*pod.*\"}-node_filesystem_free_bytes{instance=~'$node',fstype=~\"ext.*|xfs\",mountpoint !~\".*pod.*\"}))","format": "table","hide": false,"instant": true,"interval": "","intervalFactor": 1,"legendFormat": "","refId": "B"}],"title": "【$show_hostname】：Disk Space Used Basic(EXT?/XFS)","transform": "table","type": "table-old"},{"cacheTimeout": null,"datasource": "${DS__VICTORIAMETRICS}","description": "","fieldConfig": {"defaults": {"custom": {},"decimals": 2,"mappings": [{"id": 0,"op": "=","text": "N/A","type": 1,"value": "null"}],"nullValueMode": "connected","thresholds": {"mode": "absolute","steps": [{"color": "rgba(50, 172, 45, 0.97)","value": null},{"color": "rgba(237, 129, 40, 0.89)","value": 20},{"color": "#d44a3a","value": 50}]},"unit": "percent"},"overrides": []},"gridPos": {"h": 2,"w": 2,"x": 15,"y": 16},"id": 20,"interval": null,"links": [],"maxDataPoints": 100,"options": {"colorMode": "value","graphMode": "area","justifyMode": "auto","orientation": "horizontal","reduceOptions": {"calcs": ["last"],"fields": "","values": false},"textMode": "auto"},"pluginVersion": "7.2.0","targets": [{"expr": "avg(rate(node_cpu_seconds_total{instance=~\"$node\",mode=\"iowait\"}[$interval])) * 100","format": "time_series","hide": false,"instant": false,"interval": "","intervalFactor": 1,"legendFormat": "","refId": "A","step": 20}],"timeFrom": null,"timeShift": null,"title": "CPU iowait","type": "stat"},{"aliasColors": {"cn-shenzhen.i-wz9cq1dcb6zwc39ehw59_cni0_in": "light-red","cn-shenzhen.i-wz9cq1dcb6zwc39ehw59_cni0_in下载": "green","cn-shenzhen.i-wz9cq1dcb6zwc39ehw59_cni0_out上传": "yellow","cn-shenzhen.i-wz9cq1dcb6zwc39ehw59_eth0_in下载": "purple","cn-shenzhen.i-wz9cq1dcb6zwc39ehw59_eth0_out": "purple","cn-shenzhen.i-wz9cq1dcb6zwc39ehw59_eth0_out上传": "blue"},"bars": true,"dashLength": 10,"dashes": false,"datasource": "${DS__VICTORIAMETRICS}","editable": true,"error": false,"fieldConfig": {"defaults": {"custom": {},"links": []},"overrides": []},"fill": 1,"fillGradient": 0,"grid": {},"gridPos": {"h": 6,"w": 7,"x": 17,"y": 16},"hiddenSeries": false,"id": 183,"legend": {"alignAsTable": true,"avg": true,"current": true,"hideEmpty": true,"hideZero": true,"max": true,"min": false,"show": false,"sort": "current","sortDesc": true,"total": true,"values": true},"lines": false,"linewidth": 2,"links": [],"nullPointMode": "null as zero","options": {"alertThreshold": true},"percentage": false,"pluginVersion": "7.2.0","pointradius": 1,"points": false,"renderer": "flot","repeat": null,"seriesOverrides": [{"$$hashKey": "object:2970","alias": "/.*_transmit$/","transform": "negative-Y"}],"spaceLength": 10,"stack": false,"steppedLine": false,"targets": [{"expr": "increase(node_network_receive_bytes_total{instance=~\"$node\",device=~\"$device\"}[60m])","interval": "60m","intervalFactor": 1,"legendFormat": "{{device}}_receive","metric": "","refId": "A","step": 600,"target": ""},{"expr": "increase(node_network_transmit_bytes_total{instance=~\"$node\",device=~\"$device\"}[60m])","hide": false,"interval": "60m","intervalFactor": 1,"legendFormat": "{{device}}_transmit","refId": "B","step": 600}],"thresholds": [],"timeFrom": null,"timeRegions": [],"timeShift": null,"title": "Internet traffic per hour $device","tooltip": {"msResolution": false,"shared": true,"sort": 0,"value_type": "cumulative"},"type": "graph","xaxis": {"buckets": null,"mode": "time","name": null,"show": true,"values": []},"yaxes": [{"$$hashKey": "object:2977","format": "bytes","label": "transmit（-）/receive（+）","logBase": 1,"max": null,"min": null,"show": true},{"$$hashKey": "object:2978","format": "short","logBase": 1,"max": null,"min": null,"show": false}],"yaxis": {"align": false,"alignLevel": null}},{"cacheTimeout": null,"datasource": "${DS__VICTORIAMETRICS}","description": "","fieldConfig": {"defaults": {"custom": {},"mappings": [{"id": 0,"op": "=","text": "N/A","type": 1,"value": "null"}],"nullValueMode": "null","thresholds": {"mode": "absolute","steps": [{"color": "rgba(245, 54, 54, 0.9)","value": null},{"color": "rgba(237, 129, 40, 0.89)","value": 1},{"color": "rgba(50, 172, 45, 0.97)","value": 2}]},"unit": "short"},"overrides": []},"gridPos": {"h": 2,"w": 2,"x": 0,"y": 18},"id": 14,"interval": null,"links": [],"maxDataPoints": 100,"options": {"colorMode": "value","graphMode": "none","justifyMode": "auto","orientation": "horizontal","reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": false},"textMode": "value"},"pluginVersion": "7.2.0","targets": [{"expr": "count(node_cpu_seconds_total{instance=~\"$node\", mode='system'})","format": "time_series","instant": true,"interval": "","intervalFactor": 1,"legendFormat": "","refId": "A","step": 20}],"title": "CPU Cores","type": "stat"},{"cacheTimeout": null,"datasource": "${DS__VICTORIAMETRICS}","description": "","fieldConfig": {"defaults": {"custom": {},"mappings": [{"$$hashKey": "object:193","id": 0,"op": "=","text": "N/A","type": 1,"value": "null"}],"nullValueMode": "null","thresholds": {"mode": "absolute","steps": [{"color": "rgba(245, 54, 54, 0.9)","value": null},{"color": "rgba(237, 129, 40, 0.89)","value": 100000},{"color": "rgba(50, 172, 45, 0.97)","value": 1000000}]},"unit": "short"},"overrides": []},"gridPos": {"h": 2,"w": 2,"x": 15,"y": 18},"id": 179,"interval": null,"links": [],"maxDataPoints": 100,"options": {"colorMode": "value","graphMode": "none","justifyMode": "auto","orientation": "horizontal","reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": false},"textMode": "auto"},"pluginVersion": "7.2.0","targets": [{"expr": "avg(node_filesystem_files_free{instance=~\"$node\",mountpoint=\"$maxmount\",fstype=~\"ext.?|xfs\"})","format": "time_series","instant": true,"interval": "","intervalFactor": 1,"legendFormat": "","refId": "A","step": 20}],"title": "Free inodes:$maxmount ","type": "stat"},{"cacheTimeout": null,"datasource": "${DS__VICTORIAMETRICS}","description": "","fieldConfig": {"defaults": {"custom": {},"decimals": 0,"mappings": [{"id": 0,"op": "=","text": "N/A","type": 1,"value": "null"}],"nullValueMode": "null","thresholds": {"mode": "absolute","steps": [{"color": "rgba(245, 54, 54, 0.9)","value": null},{"color": "rgba(237, 129, 40, 0.89)","value": 2},{"color": "rgba(50, 172, 45, 0.97)","value": 3}]},"unit": "bytes"},"overrides": []},"gridPos": {"h": 2,"w": 2,"x": 0,"y": 20},"id": 75,"interval": null,"links": [],"maxDataPoints": 100,"options": {"colorMode": "value","graphMode": "none","justifyMode": "auto","orientation": "horizontal","reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": false},"textMode": "auto"},"pluginVersion": "7.2.0","targets": [{"expr": "sum(node_memory_MemTotal_bytes{instance=~\"$node\"})","format": "time_series","instant": true,"interval": "","intervalFactor": 1,"legendFormat": "{{instance}}","refId": "A","step": 20}],"title": "Total RAM","type": "stat"},{"cacheTimeout": null,"datasource": "${DS__VICTORIAMETRICS}","description": "","fieldConfig": {"defaults": {"custom": {},"mappings": [{"$$hashKey": "object:271","id": 0,"op": "=","text": "N/A","type": 1,"value": "null"}],"nullValueMode": "null","thresholds": {"mode": "absolute","steps": [{"color": "rgba(245, 54, 54, 0.9)","value": null},{"color": "rgba(237, 129, 40, 0.89)","value": 1024},{"color": "rgba(50, 172, 45, 0.97)","value": 10000}]},"unit": "locale"},"overrides": []},"gridPos": {"h": 2,"w": 2,"x": 15,"y": 20},"id": 178,"interval": null,"links": [],"maxDataPoints": 100,"options": {"colorMode": "value","graphMode": "none","justifyMode": "auto","orientation": "horizontal","reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": false},"textMode": "auto"},"pluginVersion": "7.2.0","targets": [{"expr": "avg(node_filefd_maximum{instance=~\"$node\"})","format": "time_series","instant": true,"intervalFactor": 1,"legendFormat": "","refId": "A","step": 20}],"title": "Total filefd","type": "stat"},{"aliasColors": {"192.168.200.241:9100_Total": "dark-red","Idle - Waiting for something to happen": "#052B51","guest": "#9AC48A","idle": "#052B51","iowait": "#EAB839","irq": "#BF1B00","nice": "#C15C17","sdb_每秒I/O操作%": "#d683ce","softirq": "#E24D42","steal": "#FCE2DE","system": "#508642","user": "#5195CE","磁盘花费在I/O操作占比": "#ba43a9"},"bars": false,"dashLength": 10,"dashes": false,"datasource": "${DS__VICTORIAMETRICS}","decimals": 2,"description": "","fieldConfig": {"defaults": {"custom": {},"links": []},"overrides": []},"fill": 1,"fillGradient": 0,"gridPos": {"h": 8,"w": 8,"x": 0,"y": 22},"hiddenSeries": false,"id": 7,"legend": {"alignAsTable": true,"avg": true,"current": true,"hideEmpty": true,"hideZero": true,"max": true,"min": true,"rightSide": false,"show": true,"sideWidth": null,"sort": "current","sortDesc": true,"total": false,"values": true},"lines": true,"linewidth": 2,"links": [],"maxPerRow": 6,"nullPointMode": "null","options": {"alertThreshold": true},"percentage": false,"pluginVersion": "7.2.0","pointradius": 5,"points": false,"renderer": "flot","repeat": null,"seriesOverrides": [{"$$hashKey": "object:3051","alias": "/.*Total/","color": "#C4162A","fill": 0}],"spaceLength": 10,"stack": false,"steppedLine": false,"targets": [{"expr": "avg(rate(node_cpu_seconds_total{instance=~\"$node\",mode=\"system\"}[$interval])) by (instance) *100","format": "time_series","hide": false,"instant": false,"interval": "","intervalFactor": 1,"legendFormat": "System","refId": "A","step": 20},{"expr": "avg(rate(node_cpu_seconds_total{instance=~\"$node\",mode=\"user\"}[$interval])) by (instance) *100","format": "time_series","hide": false,"interval": "","intervalFactor": 1,"legendFormat": "User","refId": "B","step": 240},{"expr": "avg(rate(node_cpu_seconds_total{instance=~\"$node\",mode=\"iowait\"}[$interval])) by (instance) *100","format": "time_series","hide": false,"instant": false,"interval": "","intervalFactor": 1,"legendFormat": "Iowait","refId": "D","step": 240},{"expr": "(1 - avg(rate(node_cpu_seconds_total{instance=~\"$node\",mode=\"idle\"}[$interval])) by (instance))*100","format": "time_series","hide": false,"interval": "","intervalFactor": 1,"legendFormat": "Total","refId": "F","step": 240}],"thresholds": [],"timeFrom": null,"timeRegions": [],"timeShift": null,"title": "CPU% Basic","tooltip": {"shared": true,"sort": 2,"value_type": "individual"},"type": "graph","xaxis": {"buckets": null,"mode": "time","name": null,"show": true,"values": []},"yaxes": [{"$$hashKey": "object:11294","decimals": 0,"format": "percent","label": "","logBase": 1,"max": null,"min": null,"show": true},{"$$hashKey": "object:11295","format": "short","label": null,"logBase": 1,"max": null,"min": null,"show": false}],"yaxis": {"align": false,"alignLevel": null}},{"aliasColors": {"192.168.200.241:9100_总内存": "dark-red","使用率": "yellow","内存_Avaliable": "#6ED0E0","内存_Cached": "#EF843C","内存_Free": "#629E51","内存_Total": "#6d1f62","内存_Used": "#eab839","可用": "#9ac48a","总内存": "#bf1b00"},"bars": false,"dashLength": 10,"dashes": false,"datasource": "${DS__VICTORIAMETRICS}","decimals": 2,"fieldConfig": {"defaults": {"custom": {},"links": []},"overrides": []},"fill": 1,"fillGradient": 0,"gridPos": {"h": 8,"w": 8,"x": 8,"y": 22},"height": "300","hiddenSeries": false,"id": 156,"legend": {"alignAsTable": true,"avg": true,"current": true,"hideEmpty": true,"hideZero": true,"max": true,"min": true,"rightSide": false,"show": true,"sort": "current","sortDesc": true,"total": false,"values": true},"lines": true,"linewidth": 2,"links": [],"nullPointMode": "null","options": {"alertThreshold": true},"percentage": false,"pluginVersion": "7.2.0","pointradius": 5,"points": false,"renderer": "flot","seriesOverrides": [{"$$hashKey": "object:3234","alias": "Total","color": "#C4162A","fill": 0},{"$$hashKey": "object:3235","alias": "Used%","color": "rgb(0, 209, 255)","lines": false,"pointradius": 1,"points": true,"yaxis": 2}],"spaceLength": 10,"stack": false,"steppedLine": false,"targets": [{"expr": "node_memory_MemTotal_bytes{instance=~\"$node\"}","format": "time_series","hide": false,"instant": false,"interval": "","intervalFactor": 1,"legendFormat": "Total","refId": "A","step": 4},{"expr": "node_memory_MemTotal_bytes{instance=~\"$node\"} - node_memory_MemAvailable_bytes{instance=~\"$node\"}","format": "time_series","hide": false,"interval": "","intervalFactor": 1,"legendFormat": "Used","refId": "B","step": 4},{"expr": "node_memory_MemAvailable_bytes{instance=~\"$node\"}","format": "time_series","hide": false,"interval": "","intervalFactor": 1,"legendFormat": "Avaliable","refId": "F","step": 4},{"expr": "node_memory_Buffers_bytes{instance=~\"$node\"}","format": "time_series","hide": true,"interval": "","intervalFactor": 1,"legendFormat": "内存_Buffers","refId": "D","step": 4},{"expr": "node_memory_MemFree_bytes{instance=~\"$node\"}","format": "time_series","hide": true,"intervalFactor": 1,"legendFormat": "内存_Free","refId": "C","step": 4},{"expr": "node_memory_Cached_bytes{instance=~\"$node\"}","format": "time_series","hide": true,"intervalFactor": 1,"legendFormat": "内存_Cached","refId": "E","step": 4},{"expr": "node_memory_MemTotal_bytes{instance=~\"$node\"} - (node_memory_Cached_bytes{instance=~\"$node\"} + node_memory_Buffers_bytes{instance=~\"$node\"} + node_memory_MemFree_bytes{instance=~\"$node\"})","format": "time_series","hide": true,"intervalFactor": 1,"refId": "G"},{"expr": "(1 - (node_memory_MemAvailable_bytes{instance=~\"$node\"} / (node_memory_MemTotal_bytes{instance=~\"$node\"})))* 100","format": "time_series","hide": false,"interval": "30m","intervalFactor": 10,"legendFormat": "Used%","refId": "H"}],"thresholds": [],"timeFrom": null,"timeRegions": [],"timeShift": null,"title": "Memory Basic","tooltip": {"shared": true,"sort": 2,"value_type": "individual"},"type": "graph","xaxis": {"buckets": null,"mode": "time","name": null,"show": true,"values": []},"yaxes": [{"$$hashKey": "object:3130","format": "bytes","label": null,"logBase": 1,"max": null,"min": "0","show": true},{"$$hashKey": "object:3131","format": "percent","label": "Utilization%","logBase": 1,"max": "100","min": "0","show": true}],"yaxis": {"align": false,"alignLevel": null}},{"aliasColors": {"192.168.10.227:9100_em1_in下载": "super-light-green","192.168.10.227:9100_em1_out上传": "dark-blue"},"bars": false,"dashLength": 10,"dashes": false,"datasource": "${DS__VICTORIAMETRICS}","decimals": 2,"fieldConfig": {"defaults": {"custom": {},"links": []},"overrides": []},"fill": 1,"fillGradient": 0,"gridPos": {"h": 8,"w": 8,"x": 16,"y": 22},"height": "300","hiddenSeries": false,"id": 157,"legend": {"alignAsTable": true,"avg": true,"current": true,"hideEmpty": true,"hideZero": true,"max": true,"min": true,"rightSide": false,"show": true,"sort": "current","sortDesc": true,"total": false,"values": true},"lines": true,"linewidth": 1,"links": [],"nullPointMode": "null","options": {"alertThreshold": true},"percentage": false,"pluginVersion": "7.2.0","pointradius": 2,"points": false,"renderer": "flot","seriesOverrides": [{"$$hashKey": "object:3308","alias": "/.*_transmit$/","transform": "negative-Y"}],"spaceLength": 10,"stack": false,"steppedLine": false,"targets": [{"expr": "rate(node_network_receive_bytes_total{instance=~'$node',device=~\"$device\"}[$interval])*8","format": "time_series","interval": "","intervalFactor": 1,"legendFormat": "{{device}}_receive","refId": "A","step": 4},{"expr": "rate(node_network_transmit_bytes_total{instance=~'$node',device=~\"$device\"}[$interval])*8","format": "time_series","interval": "","intervalFactor": 1,"legendFormat": "{{device}}_transmit","refId": "B","step": 4}],"thresholds": [],"timeFrom": null,"timeRegions": [],"timeShift": null,"title": "Network bandwidth usage per second $device","tooltip": {"shared": true,"sort": 2,"value_type": "individual"},"type": "graph","xaxis": {"buckets": null,"mode": "time","name": null,"show": true,"values": []},"yaxes": [{"$$hashKey": "object:3315","format": "bps","label": "transmit（-）/receive（+）","logBase": 1,"max": null,"min": null,"show": true},{"$$hashKey": "object:3316","format": "short","label": null,"logBase": 1,"max": null,"min": null,"show": false}],"yaxis": {"align": false,"alignLevel": null}},{"aliasColors": {"15分钟": "#6ED0E0","1分钟": "#BF1B00","5分钟": "#CCA300"},"bars": false,"dashLength": 10,"dashes": false,"datasource": "${DS__VICTORIAMETRICS}","decimals": 2,"editable": true,"error": false,"fieldConfig": {"defaults": {"custom": {},"links": []},"overrides": []},"fill": 1,"fillGradient": 1,"grid": {},"gridPos": {"h": 8,"w": 8,"x": 0,"y": 30},"height": "300","hiddenSeries": false,"id": 13,"legend": {"alignAsTable": true,"avg": true,"current": true,"hideEmpty": true,"hideZero": true,"max": true,"min": true,"rightSide": false,"show": true,"sort": "current","sortDesc": true,"total": false,"values": true},"lines": true,"linewidth": 2,"links": [],"maxPerRow": 6,"nullPointMode": "null","options": {"alertThreshold": true},"percentage": false,"pluginVersion": "7.2.0","pointradius": 5,"points": false,"renderer": "flot","repeat": null,"seriesOverrides": [{"$$hashKey": "object:3389","alias": "/.*CPU cores/","color": "#C4162A"}],"spaceLength": 10,"stack": false,"steppedLine": false,"targets": [{"expr": "node_load1{instance=~\"$node\"}","format": "time_series","instant": false,"interval": "","intervalFactor": 1,"legendFormat": "1m","metric": "","refId": "A","step": 20,"target": ""},{"expr": "node_load5{instance=~\"$node\"}","format": "time_series","instant": false,"interval": "","intervalFactor": 1,"legendFormat": "5m","refId": "B","step": 20},{"expr": "node_load15{instance=~\"$node\"}","format": "time_series","instant": false,"interval": "","intervalFactor": 1,"legendFormat": "15m","refId": "C","step": 20},{"expr": " sum(count(node_cpu_seconds_total{instance=~\"$node\", mode='system'}) by (cpu,instance)) by(instance)","format": "time_series","instant": false,"interval": "","intervalFactor": 1,"legendFormat": "CPU cores","refId": "D","step": 20}],"thresholds": [],"timeFrom": null,"timeRegions": [],"timeShift": null,"title": "System Load","tooltip": {"msResolution": false,"shared": true,"sort": 2,"value_type": "cumulative"},"type": "graph","xaxis": {"buckets": null,"mode": "time","name": null,"show": true,"values": []},"yaxes": [{"$$hashKey": "object:3396","format": "short","logBase": 1,"max": null,"min": null,"show": true},{"$$hashKey": "object:3397","format": "short","logBase": 1,"max": null,"min": null,"show": true}],"yaxis": {"align": false,"alignLevel": null}},{"aliasColors": {"vda_write": "#6ED0E0"},"bars": false,"dashLength": 10,"dashes": false,"datasource": "${DS__VICTORIAMETRICS}","decimals": 2,"description": "Per second read / write bytes ","fieldConfig": {"defaults": {"custom": {},"links": []},"overrides": []},"fill": 1,"fillGradient": 1,"gridPos": {"h": 8,"w": 8,"x": 8,"y": 30},"height": "300","hiddenSeries": false,"id": 168,"legend": {"alignAsTable": true,"avg": true,"current": true,"hideEmpty": true,"hideZero": true,"max": true,"min": true,"show": true,"sort": "current","sortDesc": true,"total": false,"values": true},"lines": true,"linewidth": 2,"links": [],"nullPointMode": "null","options": {"alertThreshold": true},"percentage": false,"pluginVersion": "7.2.0","pointradius": 5,"points": false,"renderer": "flot","seriesOverrides": [{"$$hashKey": "object:3474","alias": "/.*_Read bytes$/","transform": "negative-Y"}],"spaceLength": 10,"stack": false,"steppedLine": false,"targets": [{"expr": "rate(node_disk_read_bytes_total{instance=~\"$node\"}[$interval])","format": "time_series","interval": "","intervalFactor": 1,"legendFormat": "{{device}}_Read bytes","refId": "A","step": 10},{"expr": "rate(node_disk_written_bytes_total{instance=~\"$node\"}[$interval])","format": "time_series","hide": false,"interval": "","intervalFactor": 1,"legendFormat": "{{device}}_Written bytes","refId": "B","step": 10}],"thresholds": [],"timeFrom": null,"timeRegions": [],"timeShift": null,"title": "Disk R/W Data","tooltip": {"shared": true,"sort": 2,"value_type": "individual"},"type": "graph","xaxis": {"buckets": null,"mode": "time","name": null,"show": true,"values": []},"yaxes": [{"$$hashKey": "object:3481","decimals": null,"format": "Bps","label": "Bytes read (-) / write (+)","logBase": 1,"max": null,"min": null,"show": true},{"$$hashKey": "object:3482","format": "short","label": null,"logBase": 1,"max": null,"min": null,"show": false}],"yaxis": {"align": false,"alignLevel": null}},{"aliasColors": {},"bars": false,"dashLength": 10,"dashes": false,"datasource": "${DS__VICTORIAMETRICS}","decimals": 1,"description": "","fieldConfig": {"defaults": {"custom": {},"links": []},"overrides": []},"fill": 0,"fillGradient": 0,"gridPos": {"h": 8,"w": 8,"x": 16,"y": 30},"hiddenSeries": false,"id": 174,"legend": {"alignAsTable": true,"avg": true,"current": true,"hideEmpty": true,"hideZero": true,"max": true,"min": true,"rightSide": false,"show": true,"sideWidth": null,"sort": "current","sortDesc": true,"total": false,"values": true},"lines": true,"linewidth": 2,"links": [],"nullPointMode": "null","options": {"alertThreshold": true},"percentage": false,"pluginVersion": "7.2.0","pointradius": 5,"points": false,"renderer": "flot","seriesOverrides": [{"$$hashKey": "object:3554","alias": "/Inodes.*/","yaxis": 2}],"spaceLength": 10,"stack": false,"steppedLine": false,"targets": [{"expr": "(node_filesystem_size_bytes{instance=~'$node',fstype=~\"ext.*|xfs\",mountpoint !~\".*pod.*\"}-node_filesystem_free_bytes{instance=~'$node',fstype=~\"ext.*|xfs\",mountpoint !~\".*pod.*\"}) *100/(node_filesystem_avail_bytes {instance=~'$node',fstype=~\"ext.*|xfs\",mountpoint !~\".*pod.*\"}+(node_filesystem_size_bytes{instance=~'$node',fstype=~\"ext.*|xfs\",mountpoint !~\".*pod.*\"}-node_filesystem_free_bytes{instance=~'$node',fstype=~\"ext.*|xfs\",mountpoint !~\".*pod.*\"}))","format": "time_series","instant": false,"interval": "","intervalFactor": 1,"legendFormat": "{{mountpoint}}","refId": "A"},{"expr": "node_filesystem_files_free{instance=~'$node',fstype=~\"ext.?|xfs\"} / node_filesystem_files{instance=~'$node',fstype=~\"ext.?|xfs\"}","hide": true,"interval": "","legendFormat": "Inodes：{{instance}}：{{mountpoint}}","refId": "B"}],"thresholds": [],"timeFrom": null,"timeRegions": [],"timeShift": null,"title": "Disk Space Used% Basic","tooltip": {"shared": true,"sort": 2,"value_type": "individual"},"type": "graph","xaxis": {"buckets": null,"mode": "time","name": null,"show": true,"values": []},"yaxes": [{"$$hashKey": "object:3561","decimals": null,"format": "percent","label": "","logBase": 1,"max": "100","min": "0","show": true},{"$$hashKey": "object:3562","decimals": 2,"format": "percentunit","label": null,"logBase": 1,"max": "1","min": null,"show": false}],"yaxis": {"align": false,"alignLevel": null}},{"aliasColors": {"vda_write": "#6ED0E0"},"bars": false,"dashLength": 10,"dashes": false,"datasource": "${DS__VICTORIAMETRICS}","decimals": 2,"description": "Read/write completions per second\n\nWrites completed: 每个磁盘分区每秒写完成次数\n\nIO now 每个磁盘分区每秒正在处理的输入/输出请求数","fieldConfig": {"defaults": {"custom": {},"links": []},"overrides": []},"fill": 0,"fillGradient": 0,"gridPos": {"h": 9,"w": 8,"x": 0,"y": 38},"height": "300","hiddenSeries": false,"id": 161,"legend": {"alignAsTable": true,"avg": true,"current": true,"hideEmpty": true,"hideZero": true,"max": true,"min": true,"show": true,"sort": "current","sortDesc": true,"total": false,"values": true},"lines": true,"linewidth": 1,"links": [],"nullPointMode": "null","options": {"alertThreshold": true},"percentage": false,"pluginVersion": "7.2.0","pointradius": 5,"points": false,"renderer": "flot","seriesOverrides": [{"$$hashKey": "object:3711","alias": "/.*_Reads completed$/","transform": "negative-Y"}],"spaceLength": 10,"stack": false,"steppedLine": false,"targets": [{"expr": "rate(node_disk_reads_completed_total{instance=~\"$node\"}[$interval])","format": "time_series","hide": false,"interval": "","intervalFactor": 1,"legendFormat": "{{device}}_Reads completed","refId": "A","step": 10},{"expr": "rate(node_disk_writes_completed_total{instance=~\"$node\"}[$interval])","format": "time_series","hide": false,"interval": "","intervalFactor": 1,"legendFormat": "{{device}}_Writes completed","refId": "B","step": 10},{"expr": "node_disk_io_now{instance=~\"$node\"}","format": "time_series","hide": true,"interval": "","intervalFactor": 1,"legendFormat": "{{device}}","refId": "C"}],"thresholds": [],"timeFrom": null,"timeRegions": [],"timeShift": null,"title": "Disk IOps Completed（IOPS）","tooltip": {"shared": true,"sort": 2,"value_type": "individual"},"type": "graph","xaxis": {"buckets": null,"mode": "time","name": null,"show": true,"values": []},"yaxes": [{"$$hashKey": "object:3718","decimals": null,"format": "iops","label": "IO read (-) / write (+)","logBase": 1,"max": null,"min": null,"show": true},{"$$hashKey": "object:3719","format": "short","label": null,"logBase": 1,"max": null,"min": null,"show": true}],"yaxis": {"align": false,"alignLevel": null}},{"aliasColors": {"Idle - Waiting for something to happen": "#052B51","guest": "#9AC48A","idle": "#052B51","iowait": "#EAB839","irq": "#BF1B00","nice": "#C15C17","sdb_每秒I/O操作%": "#d683ce","softirq": "#E24D42","steal": "#FCE2DE","system": "#508642","user": "#5195CE","磁盘花费在I/O操作占比": "#ba43a9"},"bars": false,"dashLength": 10,"dashes": false,"datasource": "${DS__VICTORIAMETRICS}","decimals": null,"description": "The time spent on I/O in the natural time of each second.（wall-clock time）","fieldConfig": {"defaults": {"custom": {},"links": []},"overrides": []},"fill": 1,"fillGradient": 0,"gridPos": {"h": 9,"w": 8,"x": 8,"y": 38},"hiddenSeries": false,"id": 175,"legend": {"alignAsTable": true,"avg": true,"current": true,"hideEmpty": true,"hideZero": true,"max": true,"min": false,"rightSide": false,"show": true,"sideWidth": null,"sort": null,"sortDesc": null,"total": false,"values": true},"lines": true,"linewidth": 1,"links": [],"maxPerRow": 6,"nullPointMode": "null","options": {"alertThreshold": true},"percentage": false,"pluginVersion": "7.2.0","pointradius": 5,"points": false,"renderer": "flot","seriesOverrides": [],"spaceLength": 10,"stack": false,"steppedLine": false,"targets": [{"expr": "rate(node_disk_io_time_seconds_total{instance=~\"$node\"}[$interval])","format": "time_series","interval": "","intervalFactor": 1,"legendFormat": "{{device}}_ IO time","refId": "C"}],"thresholds": [],"timeFrom": null,"timeRegions": [],"timeShift": null,"title": "Time Spent Doing I/Os","tooltip": {"shared": true,"sort": 2,"value_type": "individual"},"type": "graph","xaxis": {"buckets": null,"mode": "time","name": null,"show": true,"values": []},"yaxes": [{"$$hashKey": "object:3796","decimals": null,"format": "percentunit","label": "","logBase": 1,"max": null,"min": null,"show": true},{"$$hashKey": "object:3797","format": "short","label": null,"logBase": 1,"max": null,"min": null,"show": false}],"yaxis": {"align": false,"alignLevel": null}},{"aliasColors": {"vda": "#6ED0E0"},"bars": false,"dashLength": 10,"dashes": false,"datasource": "${DS__VICTORIAMETRICS}","decimals": 2,"description": "Time spent on each read/write operation","fieldConfig": {"defaults": {"custom": {},"links": []},"overrides": []},"fill": 1,"fillGradient": 1,"gridPos": {"h": 9,"w": 8,"x": 16,"y": 38},"height": "300","hiddenSeries": false,"id": 160,"legend": {"alignAsTable": true,"avg": true,"current": true,"hideEmpty": true,"hideZero": true,"max": true,"min": true,"show": true,"sort": "current","sortDesc": true,"total": false,"values": true},"lines": true,"linewidth": 2,"links": [],"nullPointMode": "null as zero","options": {"alertThreshold": true},"percentage": false,"pluginVersion": "7.2.0","pointradius": 5,"points": false,"renderer": "flot","seriesOverrides": [{"$$hashKey": "object:4023","alias": "/,*_Read time$/","transform": "negative-Y"}],"spaceLength": 10,"stack": false,"steppedLine": false,"targets": [{"expr": "rate(node_disk_read_time_seconds_total{instance=~\"$node\"}[$interval]) / rate(node_disk_reads_completed_total{instance=~\"$node\"}[$interval])","format": "time_series","hide": false,"instant": false,"interval": "","intervalFactor": 1,"legendFormat": "{{device}}_Read time","refId": "B"},{"expr": "rate(node_disk_write_time_seconds_total{instance=~\"$node\"}[$interval]) / rate(node_disk_writes_completed_total{instance=~\"$node\"}[$interval])","format": "time_series","hide": false,"instant": false,"interval": "","intervalFactor": 1,"legendFormat": "{{device}}_Write time","refId": "C"},{"expr": "rate(node_disk_io_time_seconds_total{instance=~\"$node\"}[$interval])","format": "time_series","hide": true,"interval": "","intervalFactor": 1,"legendFormat": "{{device}}","refId": "A","step": 10},{"expr": "rate(node_disk_io_time_weighted_seconds_total{instance=~\"$node\"}[$interval])","format": "time_series","hide": true,"interval": "","intervalFactor": 1,"legendFormat": "{{device}}_加权","refId": "D"}],"thresholds": [],"timeFrom": null,"timeRegions": [],"timeShift": null,"title": "Disk R/W Time(Reference: less than 100ms)(beta)","tooltip": {"shared": true,"sort": 2,"value_type": "individual"},"type": "graph","xaxis": {"buckets": null,"mode": "time","name": null,"show": true,"values": []},"yaxes": [{"$$hashKey": "object:4030","format": "s","label": "Time read (-) / write (+)","logBase": 1,"max": null,"min": null,"show": true},{"$$hashKey": "object:4031","format": "short","label": null,"logBase": 1,"max": null,"min": null,"show": false}],"yaxis": {"align": false,"alignLevel": null}},{"aliasColors": {"192.168.200.241:9100_TCP_alloc": "semi-dark-blue","TCP": "#6ED0E0","TCP_alloc": "blue"},"bars": false,"dashLength": 10,"dashes": false,"datasource": "${DS__VICTORIAMETRICS}","decimals": 2,"description": "Sockets_used - Sockets currently in use\n\nCurrEstab - TCP connections for which the current state is either ESTABLISHED or CLOSE- WAIT\n\nTCP_alloc - Allocated sockets\n\nTCP_tw - Sockets wating close\n\nUDP_inuse - Udp sockets currently in use\n\nRetransSegs - TCP retransmission packets\n\nOutSegs - Number of packets sent by TCP\n\nInSegs - Number of packets received by TCP","fieldConfig": {"defaults": {"custom": {},"links": []},"overrides": []},"fill": 0,"fillGradient": 0,"gridPos": {"h": 8,"w": 16,"x": 0,"y": 47},"height": "300","hiddenSeries": false,"id": 158,"interval": "","legend": {"alignAsTable": true,"avg": false,"current": true,"hideEmpty": true,"hideZero": true,"max": true,"min": false,"rightSide": true,"show": true,"sideWidth": null,"sort": "current","sortDesc": true,"total": false,"values": true},"lines": true,"linewidth": 1,"links": [],"nullPointMode": "null","options": {"alertThreshold": true},"percentage": false,"pluginVersion": "7.2.0","pointradius": 5,"points": false,"renderer": "flot","seriesOverrides": [{"$$hashKey": "object:4103","alias": "/.*Sockets_used/","color": "#E02F44","lines": false,"pointradius": 1,"points": true,"yaxis": 2}],"spaceLength": 10,"stack": false,"steppedLine": false,"targets": [{"expr": "node_netstat_Tcp_CurrEstab{instance=~'$node'}","format": "time_series","hide": false,"instant": false,"interval": "","intervalFactor": 1,"legendFormat": "CurrEstab","refId": "A","step": 20},{"expr": "node_sockstat_TCP_tw{instance=~'$node'}","format": "time_series","interval": "","intervalFactor": 1,"legendFormat": "TCP_tw","refId": "D"},{"expr": "node_sockstat_sockets_used{instance=~'$node'}","hide": false,"interval": "30m","intervalFactor": 1,"legendFormat": "Sockets_used","refId": "B"},{"expr": "node_sockstat_UDP_inuse{instance=~'$node'}","interval": "","legendFormat": "UDP_inuse","refId": "C"},{"expr": "node_sockstat_TCP_alloc{instance=~'$node'}","interval": "","legendFormat": "TCP_alloc","refId": "E"},{"expr": "rate(node_netstat_Tcp_PassiveOpens{instance=~'$node'}[$interval])","hide": true,"interval": "","legendFormat": "{{instance}}_Tcp_PassiveOpens","refId": "G"},{"expr": "rate(node_netstat_Tcp_ActiveOpens{instance=~'$node'}[$interval])","hide": true,"interval": "","legendFormat": "{{instance}}_Tcp_ActiveOpens","refId": "F"},{"expr": "rate(node_netstat_Tcp_InSegs{instance=~'$node'}[$interval])","interval": "","legendFormat": "Tcp_InSegs","refId": "H"},{"expr": "rate(node_netstat_Tcp_OutSegs{instance=~'$node'}[$interval])","interval": "","legendFormat": "Tcp_OutSegs","refId": "I"},{"expr": "rate(node_netstat_Tcp_RetransSegs{instance=~'$node'}[$interval])","hide": false,"interval": "","legendFormat": "Tcp_RetransSegs","refId": "J"},{"expr": "rate(node_netstat_TcpExt_ListenDrops{instance=~'$node'}[$interval])","hide": true,"interval": "","legendFormat": "","refId": "K"}],"thresholds": [],"timeFrom": null,"timeRegions": [],"timeShift": null,"title": "Network Sockstat","tooltip": {"shared": true,"sort": 2,"value_type": "individual"},"transformations": [],"type": "graph","xaxis": {"buckets": null,"mode": "time","name": null,"show": true,"values": []},"yaxes": [{"$$hashKey": "object:4118","format": "short","label": null,"logBase": 1,"max": null,"min": null,"show": true},{"$$hashKey": "object:4119","format": "short","label": "Total_Sockets_used","logBase": 1,"max": null,"min": null,"show": true}],"yaxis": {"align": false,"alignLevel": null}},{"aliasColors": {"filefd_192.168.200.241:9100": "super-light-green","switches_192.168.200.241:9100": "semi-dark-red","使用的文件描述符_10.118.72.128:9100": "red","每秒上下文切换次数_10.118.71.245:9100": "yellow","每秒上下文切换次数_10.118.72.128:9100": "yellow"},"bars": false,"cacheTimeout": null,"dashLength": 10,"dashes": false,"datasource": "${DS__VICTORIAMETRICS}","description": "","fieldConfig": {"defaults": {"custom": {},"links": []},"overrides": []},"fill": 0,"fillGradient": 1,"gridPos": {"h": 8,"w": 8,"x": 16,"y": 47},"hiddenSeries": false,"hideTimeOverride": false,"id": 16,"legend": {"alignAsTable": false,"avg": false,"current": true,"max": false,"min": false,"rightSide": false,"show": true,"total": false,"values": true},"lines": true,"linewidth": 2,"links": [],"nullPointMode": "null","options": {"alertThreshold": true},"percentage": false,"pluginVersion": "7.2.0","pointradius": 1,"points": false,"renderer": "flot","seriesOverrides": [{"$$hashKey": "object:4197","alias": "switches","color": "#FADE2A","lines": false,"pointradius": 1,"points": true,"yaxis": 2},{"$$hashKey": "object:4198","alias": "used filefd","color": "#F2495C"}],"spaceLength": 10,"stack": false,"steppedLine": false,"targets": [{"expr": "node_filefd_allocated{instance=~\"$node\"}","format": "time_series","instant": false,"interval": "","intervalFactor": 5,"legendFormat": "used filefd","refId": "B"},{"expr": "rate(node_context_switches_total{instance=~\"$node\"}[$interval])","interval": "","intervalFactor": 5,"legendFormat": "switches","refId": "A"},{"expr": "  (node_filefd_allocated{instance=~\"$node\"}/node_filefd_maximum{instance=~\"$node\"}) *100","format": "time_series","hide": true,"instant": false,"interval": "","intervalFactor": 5,"legendFormat": "使用的文件描述符占比_{{instance}}","refId": "C"}],"thresholds": [],"timeFrom": null,"timeRegions": [],"timeShift": null,"title": "Open  File Descriptor(left)/Context switches(right)","tooltip": {"shared": true,"sort": 2,"value_type": "individual"},"type": "graph","xaxis": {"buckets": null,"mode": "time","name": null,"show": true,"values": []},"yaxes": [{"$$hashKey": "object:4219","format": "short","label": "used filefd","logBase": 1,"max": null,"min": null,"show": true},{"$$hashKey": "object:4220","format": "short","label": "context_switches","logBase": 1,"max": null,"min": null,"show": true}],"yaxis": {"align": false,"alignLevel": null}}],"refresh": "","schemaVersion": 26,"style": "dark","tags": ["Prometheus","node_exporter","StarsL.cn"],"templating": {"list": [{"allValue": "","current": {},"datasource": "${DS__VICTORIAMETRICS}","definition": "label_values(origin_prometheus)","hide": 0,"includeAll": false,"label": "Origin_prom","multi": false,"name": "origin_prometheus","options": [],"query": "label_values(origin_prometheus)","refresh": 1,"regex": "","skipUrlSync": false,"sort": 5,"tagValuesQuery": "","tags": [],"tagsQuery": "","type": "query","useTags": false},{"allValue": null,"current": {},"datasource": "${DS__VICTORIAMETRICS}","definition": "label_values(node_uname_info{origin_prometheus=~\"$origin_prometheus\"}, job)","hide": 0,"includeAll": false,"label": "JOB","multi": false,"name": "job","options": [],"query": "label_values(node_uname_info{origin_prometheus=~\"$origin_prometheus\"}, job)","refresh": 1,"regex": "","skipUrlSync": false,"sort": 5,"tagValuesQuery": "","tags": [],"tagsQuery": "","type": "query","useTags": false},{"allValue": null,"current": {},"datasource": "${DS__VICTORIAMETRICS}","definition": "label_values(node_uname_info{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"}, nodename)","hide": 0,"includeAll": true,"label": "Host","multi": false,"name": "hostname","options": [],"query": "label_values(node_uname_info{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"}, nodename)","refresh": 1,"regex": "","skipUrlSync": false,"sort": 5,"tagValuesQuery": "","tags": [],"tagsQuery": "","type": "query","useTags": false},{"allFormat": "glob","allValue": null,"current": {},"datasource": "${DS__VICTORIAMETRICS}","definition": "label_values(node_uname_info{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\",nodename=~\"$hostname\"},instance)","hide": 0,"includeAll": false,"label": "Instance","multi": true,"multiFormat": "regex values","name": "node","options": [],"query": "label_values(node_uname_info{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\",nodename=~\"$hostname\"},instance)","refresh": 1,"regex": "","skipUrlSync": false,"sort": 5,"tagValuesQuery": "","tags": [],"tagsQuery": "","type": "query","useTags": false},{"allFormat": "glob","allValue": null,"current": {},"datasource": "${DS__VICTORIAMETRICS}","definition": "label_values(node_network_info{origin_prometheus=~\"$origin_prometheus\",device!~'tap.*|veth.*|br.*|docker.*|virbr.*|lo.*|cni.*'},device)","hide": 0,"includeAll": true,"label": "NIC","multi": true,"multiFormat": "regex values","name": "device","options": [],"query": "label_values(node_network_info{origin_prometheus=~\"$origin_prometheus\",device!~'tap.*|veth.*|br.*|docker.*|virbr.*|lo.*|cni.*'},device)","refresh": 1,"regex": "","skipUrlSync": false,"sort": 1,"tagValuesQuery": "","tags": [],"tagsQuery": "","type": "query","useTags": false},{"auto": false,"auto_count": 100,"auto_min": "10s","current": {"selected": false,"text": "2m","value": "2m"},"hide": 0,"label": "Interval","name": "interval","options": [{"selected": false,"text": "30s","value": "30s"},{"selected": false,"text": "1m","value": "1m"},{"selected": true,"text": "2m","value": "2m"},{"selected": false,"text": "3m","value": "3m"},{"selected": false,"text": "5m","value": "5m"},{"selected": false,"text": "10m","value": "10m"},{"selected": false,"text": "30m","value": "30m"}],"query": "30s,1m,2m,3m,5m,10m,30m","queryValue": "","refresh": 2,"skipUrlSync": false,"type": "interval"},{"allValue": null,"current": {},"datasource": "${DS__VICTORIAMETRICS}","definition": "query_result(topk(1,sort_desc (max(node_filesystem_size_bytes{origin_prometheus=~\"$origin_prometheus\",instance=~'$node',fstype=~\"ext.?|xfs\",mountpoint!~\".*pods.*\"}) by (mountpoint))))","hide": 2,"includeAll": false,"label": "maxmount","multi": false,"name": "maxmount","options": [],"query": "query_result(topk(1,sort_desc (max(node_filesystem_size_bytes{origin_prometheus=~\"$origin_prometheus\",instance=~'$node',fstype=~\"ext.?|xfs\",mountpoint!~\".*pods.*\"}) by (mountpoint))))","refresh": 2,"regex": "/.*\\\"(.*)\\\".*/","skipUrlSync": false,"sort": 5,"tagValuesQuery": "","tags": [],"tagsQuery": "","type": "query","useTags": false},{"allValue": null,"current": {},"datasource": "${DS__VICTORIAMETRICS}","definition": "label_values(node_uname_info{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\",instance=~\"$node\"}, nodename)","hide": 2,"includeAll": false,"label": "show_hostname","multi": false,"name": "show_hostname","options": [],"query": "label_values(node_uname_info{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\",instance=~\"$node\"}, nodename)","refresh": 1,"regex": "","skipUrlSync": false,"sort": 5,"tagValuesQuery": "","tags": [],"tagsQuery": "","type": "query","useTags": false},{"allValue": null,"current": {},"datasource": "${DS__VICTORIAMETRICS}","definition": "query_result(count(node_uname_info{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"}))","hide": 2,"includeAll": false,"label": "total_servers","multi": false,"name": "total","options": [],"query": "query_result(count(node_uname_info{origin_prometheus=~\"$origin_prometheus\",job=~\"$job\"}))","refresh": 1,"regex": "/{} (.*) .*/","skipUrlSync": false,"sort": 0,"tagValuesQuery": "","tags": [],"tagsQuery": "","type": "query","useTags": false}]},"time": {"from": "now-12h","to": "now"},"timepicker": {"hidden": false,"now": true,"refresh_intervals": ["15s","30s","1m","5m","15m","30m"],"time_options": ["5m","15m","1h","6h","12h","24h","2d","7d","30d"]},"timezone": "browser","title": "服务器监控","uid": "xfpJB9FGz","version": 2
}

通过网盘分享的文件：node-exporter.json
链接: https://pan.baidu.com/s/1OkaIlBjQ4JknkJcw7BF5Uw?pwd=9gd9 提取码: 9gd9

五.效果展示

prometheus+alertmanager+邮件告警

 

grafana面板效果

 

六.涉及离线包

包含镜像包(X86版本)、脚本文件

通过网盘分享的文件：prometheus.tgz
链接: https://pan.baidu.com/s/1ShVWSpPB2OhPdxsFTcdu8g?pwd=g7zd 提取码: g7zd