python写的一个squid访问日志分析的小程序
作者:乡村运维 时间:2021-08-02 14:09:09
这两周组里面几位想学习python,于是我们就创建了一个这样的环境和氛围来给大家学习。
昨天在群里,贴了一个需求,就是统计squid访问日志中ip 访问数和url的访问数并排序,不少同学都大体实现了相应的功能,我把我简单实现的贴出来,欢迎拍砖:
日志格式如下:
%ts.%03tu %6tr %{X-Forwarded-For}>h %Ss/%03Hs %<st %rm %ru %un %Sh/%<A %mt "%{Referer}>h" "%{User-Agent}>h" %{Cookie}>h
1372776321.285 0 100.64.19.225 TCP_HIT/200 8560 GET http://img1.jb51.net/games/0908/19/1549401_3_80x100.jpg - NONE/- image/jpeg "https://www.jb51.net/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; QQDownload 734; .NET4.0C; .NET CLR 2.0.50727)" pcsuv=0;%20pcuvdata=lastAccessTime=1372776317582;%20u4ad=33480hn;%20c=14arynt;%20uf=1372776310453
#!/usr/bin/python
# -*- coding: utf-8 -*-
import sys
from optparse import OptionParser
'''
仅仅是一个关于日志文件的测试,统计处access.log 的ip数目
'''
try:
f = open('/data/proclog/log/squid/access.log')
except IOError,e:
print "can't open the file:%s" %(e)
def log_report(field):
'''
return the field of the access log
'''
if field == "ip":
return [line.split()[2] for line in f]
if field == "url":
return [line.split()[6] for line in f]
def log_count(field):
'''
return a dict of like {field:number}
'''
fields2 = {}
fields = log_report(field)
for field_tmp in fields:
if field_tmp in fields2:
fields2[field_tmp] += 1
else:
fields2[field_tmp] = 1
return fields2
def log_sort(field,number = 10 ,reverse = True):
'''
print the sorted fields to output
'''
for v in sorted(log_count(field).iteritems(),key = lambda x:x[1] , reverse = reverse )[0:int(number)]:
print v[1],v[0]
if __name__ == "__main__":
parser =OptionParser(usage="%prog [-i|-u] [-n num | -r]" ,version = "1.0")
parser.add_option('-n','--number',dest="number",type=int,default=10,help=" print top line of the ouput")
parser.add_option('-i','--ip',dest="ip",action = "store_true",help="print ip information of access log")
parser.add_option('-u','--url',dest="url",action = "store_true",help="print url information of access log")
parser.add_option('-r','--reverse',action = "store_true",dest="reverse",help="reverse output ")
(options,args) = parser.parse_args()
if len(sys.argv) < 2:
parser.print_help()
if options.ip and options.url:
parser.error(' -i and -u can not be execute at the same time ')
if options.ip :
log_sort("ip", options.number , True and options.reverse or False)
if options.url:
log_sort("url", options.number , True and options.reverse or False)
f.close()
效果如下:
标签:python,squid,访问日志分析
![](/images/zang.png)
![](/images/jiucuo.png)
猜你喜欢
对python .txt文件读取及数据处理方法总结
2022-08-04 02:36:07
![](https://img.aspxhome.com/file/2023/5/65975_0s.jpg)
python文件转为exe文件的方法及用法详解
2022-08-18 00:14:17
PHP header()函数使用详细(301、404等错误设置)
2023-11-02 17:28:23
Python如何根据时间序列数据作图
2022-06-30 01:41:01
![](https://img.aspxhome.com/file/2023/2/107032_0s.png)
利用Python脚本实现传递参数的三种方式分享
2023-07-11 03:45:44
![](https://img.aspxhome.com/file/2023/0/83710_0s.png)
Oracle关于时间/日期的操作
2009-02-26 10:37:00
细化解析:SQL Server数据库的集群设计
2009-02-05 15:59:00
Python基于jieba分词实现snownlp情感分析
2023-11-14 21:43:38
![](https://img.aspxhome.com/file/2023/0/63590_0s.jpg)
Python7个爬虫小案例详解(附源码)下篇
2023-11-23 03:47:38
![](https://img.aspxhome.com/file/2023/7/86647_0s.png)
Python实现简单的获取图片爬虫功能示例
2023-01-31 06:15:13
windowns使用PySpark环境配置和基本操作
2021-04-12 06:43:06
![](https://img.aspxhome.com/file/2023/3/95123_0s.png)
Python得到弹幕并保存到Excel中怎么设置
2021-04-04 16:42:22
![](https://img.aspxhome.com/file/2023/9/99289_0s.jpg)
asp封装dll源码分享
2008-09-25 17:20:00
![](https://img.aspxhome.com/file/UploadPic/20089/25/asp-dll_50s.jpg)
ASP利用Google实现在线翻译功能
2010-03-07 17:28:00
Favicon图标小常识
2010-04-01 12:01:00
用Python实现最速下降法求极值的方法
2023-05-04 20:03:34
![](https://img.aspxhome.com/file/2023/7/112977_0s.jpg)
Oracle PL/SQL语言入门基础
2010-07-20 13:28:00
如何在Python 中使用 join() 函数把列表拼接成一个字符串
2022-10-15 14:58:38
Python3访问并下载网页内容的方法
2022-11-12 06:08:39
详解python itertools功能
2022-12-05 23:07:31