Linux shell命令统计某列去重后的值

更新时间：2019年02月26日 14:08:37 作者：sxf_0123

今天小编就为大家分享一篇关于Linux shell命令统计某列去重后的值，小编觉得内容挺不错的，现在分享给大家，具有很好的参考价值，需要的朋友一起跟随小编来看看吧

需求：根据MAC获取，MAC对应的硬件版本和软件版本。在根据硬件版本统计MAC的数量

$ head test_1.txt

00:07:67,EC2108,HWV010537P0000
00:07:63,Z83,ZNV861010P1225
00:07:67,EC6108V9U_ca_sccdx,HWV218012P0000
00:07:63,SBox8900hisi3716V3,ZGV865012P0001
00:07:67,EC6108V9U_ca_sccdx,HWV218012P0000
00:21:26,EC6108V9U_ca_sccdx,HWV218012P0000
00:07:67,EC6108V9U_ca_sccdx,HWV218012P0000
00:07:63,STBHD-ST7105-000,ZTV70433089P0002
00:21:26,EC6108V9U_ca_sccdx,HWV215012P0000
00:21:26,EC6108V9U_ca_sccdx,HWV218012P0000

第二列去重

$ cat test_2.txt | awk -F ',' '{print $2}' | sort | uniq

EC2108
EC6108V9U_ca_sccdx
SBox8900hisi3716V3
STBHD-ST7105-000
Z83

将字符串转化为列表

import os
path = "C:\\Users\\Administrator\\Desktop\\stb\\dd.txt"
read = open(path,"r")
list_ = []
for line in read:
  line_ = line.strip()
  list_.append(line_)
print(list_)

根据厂商和硬件版本获取对应的软件版本

方法一：

GET gather-012-20180109/_search
{
 "size": 0, 
 "aggs": {
  "1": {
   "terms": {
    "field": "dev_hard_ver",
    "size": 200
   },
   "aggs": {
    "2": {
     "terms": {
      "field": "soft_ver",
      "size": 1
     }
    }
   }
  }
 }
}

方法二：

import os
from elasticsearch import Elasticsearch
path = "C:\\Users\\Administrator\\Desktop\\stb\\fact_stb.txt"
write_file = open(path,"a+")
es = Elasticsearch()
def get_Data(dev_hard_ver,dev_vendor_name):
  index_ = "gather-012-20180109"
  try:
    rs = es.search(index=index_,body={
      "size": 1,
       "query": {
        "bool": {
         "must": [
          {"term": {
           "dev_hard_ver": {
            "value": dev_hard_ver
           }
          }
         },
         {
          "term": {
           "dev_vendor_name": {
            "value": dev_vendor_name
           }
          }
         }
         ]
        }
       },
       "_source": ["soft_ver","dev_vendor_name","dev_hard_ver"]
    })
    return rs
  except:
    return -1
dev_hard_ver=["02L0B61701MTAS0001", "0x209", "0x441", "0x485", "0xE0A", "0xE17", "16060102L0B61701MTAS0001", "1828", "1903", "2035", "5202", "99007002L0B6170635100791", "9900B002L0B61701IHO3000T", "990104010004102300000001", "99010402L0B6083100000001", "990501010003170908A5C800", "A7.32.64.A0", "B860AV2.1", "BJDS2Z-52", "CHEP3000SC", "CHI619112", "CHIHO3000", "CHIHO3000A", "CHIHO3000JX2", "CHIHO33", "CHIHO3300A", "CHIHO3300AD", "CUB-ANDROIDSTB", "EC2106V1", "EC2108", "EC6108V9_pub_gdydx", "EC6108V9U_ca_sccdx", "EC6108V9U_pub_hbjdx", "EC6108V9U_pub_sccdx", "G2-40_2544", "HG510I", "HG650-TDSWH", "HG650-TDSWH-11", "HT670-V", "HY_V1.0", "HY107", "HY112", "HY127", "HY2021", "HY34E71C02L0B6V003HG510P04", "HY34E71C02L0B6V3.0HG510P10", "HY44170102L0B6V003HG510P04", "HY9900E0013798KK0120170001", "HY990104010004102300000001", "HYC37.E", "HYCHEP920SC", "HYCHEP921SC", "HYEE", "HYHG510IV1.0", "HYHG510PV3.0", "HYTY_H1.00.00", "HYV10-ANDROIDIGW-068", "HYYiX-G210V1.1", "IBHN-ANDROIDSTB-068", "IBHR-ANDROIDSTBGA-069", "IBHU-ANDROIDSTB-068", "IBHX-ANDROIDSTBGA-068", "IBHX-ANDROIDSTBGA-069", "IPTV-S63-V1.1", "IPTV-S65-V1.1", "IPTV-S65-V1.2", "ITV628 HD", "KB1.100.01H", "M8043V02", "MR222-DSGH-19", "mx_ca_sccdx", "S80-E-R1.01.01", "SBox8900hisi3716V3", "SBox8900hisi3716V386501007", "STBAB-ANDROIDSTB-068", "STBAB-ANDROIDSTBGA-069", "STBHD-AML8626X-000", "STBHD-ST7105-000", "STBHD-ST7105-128-256-000", "STBHO-AML8626X-000", "tgh0001", "TY_A1.00.00", "TY_H1.00.00", "V1", "V10-ANDROIDIGW-068", "V2.1", "Z82", "Z83", "Z85E", "ZXB700V5C(IBHL-000-000)", "ZXB860A(IBHR-ANDROIDSTB-068)", "ZXB860A(IBHR-ANDROIDSTBGA-068)"]
dev_vendor_name=["SCTY", "CHANGHONG", "Jiuzhou", "Jiuzhou", "Hisense", "Hisense", "SCTY", "SKYWORTH", "SKYWORTH", "SKYWORTH", "Huawei", "ZTE", "CHANGHONG", "SKYWORTH", "SKYWORTH", "Zhaoneng", "UTSTARCOM", "ZTE", "Fiberhome", "CHANGHONG", "CHANGHONG", "CHANGHONG", "CHANGHONG", "CHANGHONG", "CHANGHONG", "CHANGHONG", "CHANGHONG", "YOUHUA", "Huawei", "Huawei", "Huawei", "Huawei", "Huawei", "Huawei", "YINHE", "UTStarcom", "Fiberhome", "Fiberhome", "Zhaoneng", "SCTY", "Hisense", "Hisense", "Hisense", "SKYWORTH", "YHTC", "YHTC", "YHTC", "FiberHome", "SKYWORTH", "Huawei", "CHANGHONG", "CHANGHONG", "Hisense", "UTStarcom", "YHTC", "SCTY", "ZTE", "CCS", "ZTE", "ZTE", "ZTE", "ZTE", "ZTE", "ChinaGCI", "ChinaGCI", "ChinaGCI", "ChangHong", "30kt", "Huawei", "FiberHome", "Huawei", "ChinaGCI", "Sunniwell", "Sunniwell", "ZTE", "ZTE", "ZTE", "ZTE", "ZTE", "ZTE", "Yihekeji", "SCTY", "SCTY", "ZTE", "ZTE", "FiberHome", "Zhaoneng", "Zhaoneng", "Zhaoneng", "ZTE", "ZTE", "ZTE"]
for i in range(len(dev_vendor_name)):
  rs = get_Data(dev_hard_ver[i],dev_vendor_name[i])
  if rs!=-1:
    for hit in rs['hits']['hits']:
      soft_ver=hit['_source']['soft_ver']
      if soft_ver is None:
        soft_ver='None'
      write_file.write(soft_ver+"\n")
    else:
      write_file.write("-1" + "\n")
write_file.close()

总结

以上就是这篇文章的全部内容了，希望本文的内容对大家的学习或者工作具有一定的参考学习价值，谢谢大家对脚本之家的支持。如果你想了解更多相关内容请查看下面相关链接

您可能感兴趣的文章:

fedora 开机启动/禁止开机启动服务的实现
下面小编就为大家带来一篇fedora 开机启动/禁止开机启动服务的实现。小编觉得挺不错的，现在就分享给大家，也给大家做个参考。一起跟随小编过来看看吧
2017-02-02
Shell脚本编写的八条可靠建议(值得收藏)
这八个建议，来源于键者几年来编写 shell 脚本的一些经验和教训，不夸张地说，每条都是精挑细选的，虽然有几点算是老生常谈了。好了，对shell脚本编写八条建议感兴趣的朋友一起看看吧
2016-11-11
linux下passwd命令设置修改用户密码的方法
下面小编就为大家带来一篇linux下passwd命令设置修改用户密码的方法。小编觉得挺不错的，现在就分享给大家，也给大家做个参考。一起跟随小编过来看看吧
2017-02-02
Shell命令批量杀死进程的方法实现
本文主要介绍了Shell命令批量杀死进程的方法实现，文中通过示例代码介绍的非常详细，对大家的学习或者工作具有一定的参考学习价值，需要的朋友们下面随着小编来一起学习学习吧
2023-06-06
Linux中cat命令的使用详解
cat 命令的用途不仅仅是显示文件内容,还可以打印文本文件的文件内容，下面就为大家详细介绍一下Linux中cat命令的具体用法，需要的小伙伴可以了解一下
2023-08-08
shell判断文件是否为空的代码
这篇文章主要介绍了shell判断文件是否为空的代码,需要的朋友可以参考下
2017-05-05
shell script获取文件名或者目录名称的方法
本文主要介绍了shell script获取文件名或者目录名称的方法，文中通过示例代码介绍的非常详细，对大家的学习或者工作具有一定的参考学习价值，需要的朋友们下面随着小编来一起学习学习吧
2022-06-06
linux shell中单引号、双引号、反引号、反斜杠的区别
shell可以识别4种不同类型的引字符号: 单引号字符' 双引号字符" 反斜杠字符\ 反引号字符`的区别，学习shell编程的朋友可以看下
2013-01-01
bash shell命令行选项与修传入参数处理
本文介绍了bash shell命令行参数与命令行选项的操作方法，有关shell 命令行参数的实例教程，有需要的朋友参考下。
2014-04-04
用shell脚本和c语言将大写字母转成小写的代码
最近在学shell编程，在网上看到tr '[:upper:]' '[:lower:]' 可以把字符串中大写字母转换成小写字母，我就在这个的基础上写了一个脚本来自己学习学习
2013-08-08