MemoryError с использованием dpkt Python 2.7 - PullRequest
Новая
       23

MemoryError с использованием dpkt Python 2.7

1 голос
/ 23 апреля 2019

Я захватил данные с помощью wireshark. Сейчас я пытаюсь проанализировать данные в Python 2.7, используя dpkt, в словарь, который я затем выберу.

Я постоянно получаю MemoryError. Я предполагаю, что это может быть потому, что я имею дело с тысячами МБ и просто не хватает памяти, но я не уверен.

storepcap.py 

from socket import inet_ntoa
import traceback
import datetime
try:
    import cPickle as pickle
except:
    import pickle

myAddress = '10.0.0.3'

files = os.listdir(os.getcwd())

dataFileName = 'alldata.dat'
dataFile = open(dataFileName, 'wb')  
try:
    dpktData = pickle.load(dataFile)
except IOError:
    dpktData = {}            

def inet_to_str(inet):
    return socket.inet_ntop(socket.AF_INET, inet)
badFrames = 0
data = ""
for file in files:
    if file.endswith('pcap'):
        print "Bad Frames " + str(badFrames)
        print 'Parsing ' + str(file)        

        data = open(file, 'rb')        
        pcap = dpkt.pcap.Reader(data)

        try:
            for timestamp, buf in pcap:

                try:
                    eth = dpkt.ethernet.Ethernet(buf)
                except:
                    print "Error parsing frame"
                    continue

                if not isinstance(eth.data, dpkt.ip.IP):
                    continue

                ip = eth.data

                if myAddress != inet_ntoa(ip.src) and myAddress != inet_ntoa(ip.dst):
                    continue

                try:
                    dpktData[str(datetime.datetime.utcfromtimestamp(timestamp))] = ip
                except:
                    badFrames += 1
                    continue

        except Exception, e:
            traceback.print_exc()
        finally:
            print "Closing file %s"%str(file)
            data.close()
pickle.dump(dpktData, dataFile)
dataFile.close()

выход: * +1010 * 

Bad Frames 0
Parsing 2019-04-17-traffic.pcap
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Error parsing frame
Traceback (most recent call last):
  File "storepcap.py", line 34, in <module>
    for timestamp, buf in pcap:
  File "C:\Python27\lib\site-packages\dpkt\pcap.py", line 315, in __iter__
MemoryError
Closing file 2019-04-17-traffic.pcap
Bad Frames 93907
Parsing traffic-2019-4-20-3.pcap
Error parsing frame
Traceback (most recent call last):
  File "storepcap.py", line 34, in <module>
  File "C:\Python27\lib\site-packages\dpkt\pcap.py", line 315, in __iter__
MemoryError
Closing file traffic-2019-4-20-3.pcap
Bad Frames 93915
Parsing traffic_00001_20190418010523.pcap
Error parsing frame
Traceback (most recent call last):
  File "storepcap.py", line 34, in <module>
  File "C:\Python27\lib\site-packages\dpkt\pcap.py", line 315, in __iter__
MemoryError
Closing file traffic_00001_20190418010523.pcap
Bad Frames 93915
Parsing traffic_00001_20190418010610.pcap
Error parsing frame
Traceback (most recent call last):
  File "storepcap.py", line 34, in <module>
  File "C:\Python27\lib\site-packages\dpkt\pcap.py", line 315, in __iter__
MemoryError
Closing file traffic_00001_20190418010610.pcap
Bad Frames 93915
Parsing traffic_00002_20190418070000.pcap
Error parsing frame
Traceback (most recent call last):
  File "storepcap.py", line 34, in <module>
  File "C:\Python27\lib\site-packages\dpkt\pcap.py", line 315, in __iter__
MemoryError
Closing file traffic_00002_20190418070000.pcap
Bad Frames 93915
Parsing traffic_00003_20190418130000.pcap
Error parsing frame
Traceback (most recent call last):
  File "storepcap.py", line 34, in <module>
  File "C:\Python27\lib\site-packages\dpkt\pcap.py", line 315, in __iter__
MemoryError
Closing file traffic_00003_20190418130000.pcap
Bad Frames 93915
Parsing traffic_00004_20190418190000.pcap
Traceback (most recent call last):
  File "storepcap.py", line 31, in <module>
    pcap = dpkt.pcap.Reader(data)
  File "C:\Python27\lib\site-packages\dpkt\pcap.py", line 245, in __init__
    self.__fh = FileHdr(buf)
MemoryError

G:\AIdata>```
...