Zabbix对Kafka topic积压数据监控的问题(bug优化)
简述
《Zabbix对Kafka topic积压数据监控》一文的目的是通过Zabbix自动发现实现对多个消费者组的Topic及Partition的Lag进行监控。因在实际监控中发现有问题,为给感兴趣的读者不留坑,特通过此文对监控进行优化调整。
分区自动发现
# 未优化前的计算方式: # 自动发现配置文件 vim consumer-groups.conf #按消费者组(Group)|Topic格式,写入自动发现配置文件 test-group|test # 执行脚本自动发现指定消费者和topic的分区 bash consumer-groups.sh discovery { "data": [ { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"0" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"1" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"3" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"2" } ] }
经过上线验证,当自动发现配置文件只有一个test-group|test
是没有问题的,但当我们按需求再接入test-group|test1
(即test-group消费者组的第二个Topic)时,自动发现的结果如下:
# 未优化前的计算方式: vim consumer-groups.conf #按消费者组(Group)|Topic格式,写入自动发现配置文件 test-group|test test-group|test1 # 执行脚本自动发现指定消费者和topic的分区 bash consumer-groups.sh discovery { "data": [ { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"0" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"1" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"3" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"2" } { "{#GROUP}":"test-group", "{#TOPICP}":"test1", "{#PARTITION}":"0" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test2", "{#PARTITION}":"1" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test3", "{#PARTITION}":"2" } ] }
了解Zabbix自动发现格式的同学会发现,每个Topic的Partition会出现','
,这种格式是不符合规范,这就是导致我们的监控项会出现问题,因此我们需要进一步修改脚本。
经修改后,最终效果应该如下:
# 优化后的计算方式: vim consumer-groups.conf #按消费者组(Group)|Topic格式,写入自动发现配置文件 test-group|test test-group|test1 # 执行脚本自动发现指定消费者和topic的分区 bash consumer-groups.sh discovery { "data": [ { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"0" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"1" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"3" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"2" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test1", "{#PARTITION}":"0" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test1", "{#PARTITION}":"1" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test1", "{#PARTITION}":"2" } ] }
获取监控项“test-group/test/分区X”的Lag
经过自动发现后的数据,我们可以进一步获取不同分区的lag
# 优化后的计算方式: # test-group test分区0 lag bash consumer-groups.sh lag test-group test 0 # test-group test分区1 lag bash consumer-groups.sh lag test-group test 1 # test-group test1分区0 lag bash consumer-groups.sh lag test-group test1 0
通过命令可以看到,我们的参数通过消费者组、Topic、Partition来获取最终的lag值,如果不加消费者区分,那么无法区分不同消费者组和不同Topic相应的lag结果:
# 未优化前的计算方式: # 获取分区0 lag bash consumer-groups.sh lag 0 # 获取分区1 lag bash consumer-groups.sh lag 1 # 获取分区2 lag bash consumer-groups.sh lag 2 # 获取分区3 lag bash consumer-groups.sh lag 3
最终优化后脚本
# 自动发现配置文件 vim consumer-groups.conf #按消费者组(Group)|Topic格式,写入自动发现配置文件 test-group|test test-group|test1 # 自动发现、lag计算脚本 vim consumer-groups.sh #!/bin/bash ##comment: 根据消费者组监控topic lag,进行监控告警 #配置文件说明 #消费者组|Topic #test-group|test #获取topic 信息 cal_topic() { if [ $# -ne 2 ]; then echo "parameter num error, 读取topic信息失败" exit 1 else /usr/local/kafka/bin/./kafka-consumer-groups.sh --bootstrap-server 192.168.3.55:9092 --describe --group $1 |grep -w $2|grep -v none fi } #topic+分区自动发现 topic_discovery() { printf "{\n" printf "\t\"data\": [\n" m=0 num=`cat /etc/zabbix/monitor_scripts/consumer-groups.conf|wc -l` for line in `cat /etc/zabbix/monitor_scripts/consumer-groups.conf` do m=`expr $m + 1` group=`echo ${line} | awk -F'|' '{print $1}'` topic=`echo ${line} | awk -F'|' '{print $2}'` cal_topic $group $topic > /tmp/consumer-group-tmp count=`cat /tmp/consumer-group-tmp|wc -l` n=0 while read line do n=`expr $n + 1` #判断最后一行 if [ $n -eq $count ] && [ $m -eq $num ]; then topicp=`echo $line | awk '{print $1}'` partition=`echo $line | awk '{print $2}'` printf "\t\t{ \"{#GROUP}\":\"${group}\", \"{#TOPICP}\":\"${topicp}\", \"{#PARTITION}\":\"${partition}\" }\n" else topicp=`echo $line | awk '{print $1}'` partition=`echo $line | awk '{print $2}'` printf "\t\t{ \"{#GROUP}\":\"${group}\", \"{#TOPICP}\":\"${topicp}\", \"{#PARTITION}\":\"${partition}\" },\n" fi done < /tmp/consumer-group-tmp done printf "\t]\n" printf "}\n" } if [ $1 == "discovery" ]; then topic_discovery elif [ $1 == "lag" ];then cal_topic $2 $3 > /tmp/consumer-group cat /tmp/consumer-group |awk -v t=$3 -v p=$4 '{if($1==t && $2==p ){print $5}}' else echo "Usage: /data/scripts/consumer-group.sh discovery | lag" fi # 手动运行 ## 自动发现 bash consumer-groups.sh discovery ## test-group test分区0 lag bash consumer-groups.sh lag test-group test 0
接入Zabbix
1.Zabbix配置文件
vim userparameter_kafka.conf UserParameter=topic_discovery,bash /data/scripts/consumer-groups.sh discovery UserParameter=topic_log[*],bash /data/scripts/consumer-groups.sh lag "$1" "$2" "$3"
2.Zabbix自动发现
3.监控项配置
4.告警信息
告警主机:Kafka_192.168.3.55 主机IP:192.168.3.55 主机组:Kafka 告警时间:2022.03.21 00:23:10 告警等级:Average 告警信息:test-group/test/分区1:数据积压100 告警项目:topic_lag[test-group,test,1] 问题详情: test-group/test/1: 62
到此这篇关于Zabbix对Kafka topic积压数据监控的文章就介绍到这了,更多相关Zabbix积压数据监控内容请搜索脚本之家以前的文章或继续浏览下面的相关文章希望大家以后多多支持脚本之家!
相关文章
Zabbix对Kafka topic积压数据监控的问题(bug优化)
这篇文章主要介绍了Zabbix对Kafka topic积压数据监控,本文给大家介绍的非常详细,对大家的学习或工作具有一定的参考借鉴价值,需要的朋友可以参考下2022-07-07
最新评论