背景和目标
这次我想要处理的是一个txt文件，里面的内容是一台机器定时ping另一台机器的输出结果，想要提取出的内容是时间和rtt值，最后还要把结果输出到csv文件。
1. 明确要提取的内容，编写正则表达式
要提取的文本如下：
第一步是要编写正则表达式，此时可以先不要读取数据文件。先复制一部分数据到str中，方便测试。
编写正则表达式用到了re模块，因为每个人要处理的文本是不一样的，所以需要自己去学习基本的使用方法。re具体使用方法可以参考这篇文章：
/p/139596371
关键就是弄清楚.*?和{}的作用，还有re.S可以匹配到换行符，就可以比较容易地写出正确的表达式。
import re# 为了方便测试，我把一部分文本先放到str里str='''-03-11 15:21:481PING 81.71.51.181 (81.71.51.181) 56(84) bytes of data.64 bytes from 81.71.51.181: icmp_seq=1 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=2 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=3 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=4 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=5 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=6 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=7 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=8 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=9 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=10 ttl=45 time=253 ms--- 81.71.51.181 ping statistics ---10 packets transmitted, 10 received, 0% packet loss, time 9000msrtt min/avg/max/mdev = 250.203/250.563/253.202/0.961 ms-03-11 15:22:402PING 81.71.51.181 (81.71.51.181) 56(84) bytes of data.64 bytes from 81.71.51.181: icmp_seq=1 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=2 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=3 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=4 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=5 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=6 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=7 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=8 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=9 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=10 ttl=45 time=250 ms--- 81.71.51.181 ping statistics ---10 packets transmitted, 10 received, 0% packet loss, time 9009msrtt min/avg/max/mdev = 250.181/250.256/250.434/0.636 ms-03-11 15:23:443PING 81.71.51.181 (81.71.51.181) 56(84) bytes of data.64 bytes from 81.71.51.181: icmp_seq=1 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=2 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=3 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=4 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=5 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=6 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=7 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=8 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=9 ttl=45 time=250 ms64 bytes from 81.71.51.181: icmp_seq=10 ttl=45 time=250 ms--- 81.71.51.181 ping statistics ---10 packets transmitted, 10 received, 0% packet loss, time 9009msrtt min/avg/max/mdev = 250.209/250.320/250.658/0.563 ms'''# print(re.findall(r'(\d{4}-\d{2}-\d{2}\s\d{2}:\d{2})', str)) # 提取时间# print(re.findall(r'mdev = (.*?) ms', str)) # 提取rttprint(re.findall(r'(\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}).*?mdev = (.*?) ms', data, re.S)) # 提取时间和rtt 包括换行
输出：
D:\python37\python.exe D:/test/data_process.py['-03-11 15:21', '-03-11 15:22', '-03-11 15:23']['250.203/250.563/253.202/0.961', '250.181/250.256/250.434/0.636', '250.209/250.320/250.658/0.563'][('-03-11 15:21', '250.203/250.563/253.202/0.961'), ('-03-11 15:22', '250.181/250.256/250.434/0.636'), ('-03-11 15:23', '250.209/250.320/250.658/0.563')]Process finished with exit code 0
2. 从文件中读入数据
编写出正确的正则表达式后，就可以从文件中读数据了
import re# 读取文件with open("ping/ping_flkf_gz.txt","r") as input_file:str = input_file.read()print(re.findall(r'(\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}).*?mdev = (.*?) ms', str, re.S)) # 提取时间和延迟 包括换行input_file.close() # 关闭文件
输出比较多，截取一部分展示：
D:\python37\python.exe D:/test/data_process.py[('-03-11 15:21', '250.203/250.563/253.202/0.961'), ('-03-11 15:22', '250.181/250.256/250.434/0.636'), ('-03-11 15:23', '250.209/250.320/250.658/0.563'), ('-03-11 15:25', '250.183/250.240/250.275/0.225'), ('-03-11 15:26', '250.217/250.240/250.300/0.592'), ('-03-11 15:27', '250.166/250.362/250.956/0.683'), ('-03-11 15:28', '250.186/250.256/250.343/0.319'), ('-03-11 15:29', '250.181/250.435/252.077/0.776'), ('-03-11 15:30', '250.177/250.249/250.401/0.673'), ('-03-11 15:31', '250.210/250.436/251.498/0.376'), ('-03-11 15:32', '250.207/250.280/250.588/0.401'), ('-03-11 15:33', '250.237/250.336/250.747/0.568'), ('-03-11 15:34', '250.217/250.283/250.437/0.675'), ('-03-11 15:35', '250.254/250.456/251.092/0.623'), ('-03-11 15:36', '250.167/250.236/250.308/0.226'), ('-03-11 15:37', '250.162/250.399/251.032/0.667'), ('-03-11 15:38', '250.207/250.261/250.406/0.053'), ('-03-11 15:39', '250.219/250.657/252.056/0.878')]
这里其实是一个列表，里面的每个元组是我提取出来的时间和rtt。
3. 写入csv
能够正确读取输入文件并提取数据后，下一步就是要把结果写入csv文件，所以用到了csv模块。
for循环遍历列表，使用csv_writer.writerow一行行写入csv文件。
import reimport csv# 读取文件with open("ping/ping_flkf_gz.txt", "r") as input_file:str = input_file.read()# 用一个列表接收提取出来的数据list = re.findall(r'(\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}).*?mdev = (.*?) ms', str, re.S) # 1.创建文件对象output_file = open('res/ping/ping_flkf_gz.csv', 'w', encoding='utf-8', newline='')# 2.基于文件对象构建csv写入对象csv_writer = csv.writer(output_file)# 3. 写入表头csv_writer.writerow(["time", "latency"])# 4.遍历列表，写入csv文件for i in list:csv_writer.writerow([i[0], i[1]])input_file.close() # 关闭文件output_file.close() # 关闭文件
结果就写入到csv文件中了
time,latency-03-11 15:21,250.203/250.563/253.202/0.961-03-11 15:22,250.181/250.256/250.434/0.636-03-11 15:23,250.209/250.320/250.658/0.563-03-11 15:25,250.183/250.240/250.275/0.225-03-11 15:26,250.217/250.240/250.300/0.592-03-11 15:27,250.166/250.362/250.956/0.683-03-11 15:28,250.186/250.256/250.343/0.319-03-11 15:29,250.181/250.435/252.077/0.776-03-11 15:30,250.177/250.249/250.401/0.673-03-11 15:31,250.210/250.436/251.498/0.376
4. 还可以把每个数值分开存放
发现此时latency那一列是这样的250.203/250.563/253.202/0.961
为了后面方便处理，把每个数值单独作为一列，因此要修改正则表达式
import reimport csv# 读取文件with open("ping/ping_flkf_gz.txt", "r") as input_file:str = input_file.read()# 用一个列表接收提取出来的数据 这里修改了正则表达式，使得每个数值单独作为一列list = re.findall(r'(\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}).*?mdev = (.*?)/(.*?)/(.*?)/(.*?) ms', str, re.S)# 1.创建文件对象output_file = open('res/ping/ping_flkf_gz.csv', 'w', encoding='utf-8', newline='')# 2.基于文件对象构建csv写入对象csv_writer = csv.writer(output_file)# 3. 写入表头csv_writer.writerow(["time", "min", "avg", "max", "mdev"])# 4.遍历列表，写入csv文件for i in list:csv_writer.writerow([i[0], i[1], i[2], i[3], i[4]])input_file.close() # 关闭文件output_file.close() # 关闭文件
输出到csv文件的效果：
至此就完成了~
