Python утечка на полке? - PullRequest
       32

Python утечка на полке?

2 голосов
/ 04 августа 2020

Похоже, что при перезаписи ключа на полке при определенных обстоятельствах размер полки неожиданно увеличивается. Это как если бы некоторые данные на полке не имели ссылки на них, как в случае утечки памяти. Похоже, это как-то связано с добавлением в списки. Кто-нибудь знает почему? Вот минимальный пример:

import shelve, os
import numpy as np

def run():
    list_array = []
    expected_size = 0
    for i in range(5):
        array_100mb = np.zeros(1024*1024*100//8)
        list_array.append(array_100mb)
        expected_size = 100*len(list_array)
        with shelve.open('shelve_test') as s:
            s['val'] = list_array
        size_mb = os.path.getsize('shelve_test.dat') // 1024 // 1024
        print(f'Iteration {i}: \t shelve size is {size_mb}Mb; \t expected size is     {expected_size}Mb')
        
for j in range(5):
    run()
    print()

Это выводит:

Iteration 0:     shelve size is 100Mb;   expected size is 100Mb
Iteration 1:     shelve size is 300Mb;   expected size is 200Mb
Iteration 2:     shelve size is 600Mb;   expected size is 300Mb
Iteration 3:     shelve size is 1000Mb;  expected size is 400Mb
Iteration 4:     shelve size is 1500Mb;  expected size is 500Mb

Iteration 0:     shelve size is 1500Mb;  expected size is 100Mb
Iteration 1:     shelve size is 1700Mb;  expected size is 200Mb
Iteration 2:     shelve size is 2000Mb;  expected size is 300Mb
Iteration 3:     shelve size is 2400Mb;  expected size is 400Mb
Iteration 4:     shelve size is 2900Mb;  expected size is 500Mb

Iteration 0:     shelve size is 2900Mb;  expected size is 100Mb
Iteration 1:     shelve size is 3100Mb;  expected size is 200Mb
Iteration 2:     shelve size is 3400Mb;  expected size is 300Mb
Iteration 3:     shelve size is 3800Mb;  expected size is 400Mb
Iteration 4:     shelve size is 4300Mb;  expected size is 500Mb

Iteration 0:     shelve size is 4300Mb;  expected size is 100Mb
Iteration 1:     shelve size is 4500Mb;  expected size is 200Mb
Iteration 2:     shelve size is 4800Mb;  expected size is 300Mb
Iteration 3:     shelve size is 5200Mb;  expected size is 400Mb
Iteration 4:     shelve size is 5700Mb;  expected size is 500Mb

Iteration 0:     shelve size is 5700Mb;  expected size is 100Mb
Iteration 1:     shelve size is 5900Mb;  expected size is 200Mb
Iteration 2:     shelve size is 6200Mb;  expected size is 300Mb
Iteration 3:     shelve size is 6600Mb;  expected size is 400Mb
Iteration 4:     shelve size is 7100Mb;  expected size is 500Mb

Python версия 3.6.6

Добро пожаловать на сайт PullRequest, где вы можете задавать вопросы и получать ответы от других членов сообщества.
...