У меня есть один экземпляр mongoDB 3.4, который поглощает всю оперативную память на компьютере EC2 и задается вопросом, как с этим лучше всего справиться.
Это единственный экземпляр, на котором запущено несколько приложений с очень низким трафиком, но накопили некоторые данные.По бюджетным причинам мне нужно остаться на этом экземпляре. Размер ОЗУ.
При других процессах на этой машине недавно произошел сбой mongo, я думаю, из-за OOM, и у меня на машине только несколько МБ оперативной памяти..
$ free -h
total used free shared buffers cached
Mem: 3.9G 3.6G 263M 416K 667M 446M
-/+ buffers/cache: 2.5G 1.3G
Swap: 0B 0B 0B
Какие подходы я могу использовать?Я рассмотрел следующее, но хотел бы, чтобы любые комментарии для этой ситуации были наилучшим.
ограничение размера кэша wiredTiger в параметре конфигурации db.conf ИЛИ --wiredTigerEngineConfigString="cache_size=200M" не уверен, что это хорошоразмер для этого ...
Я изменил некоторые коллекции на меньшие и ограниченные, но я думаю, что мне нужно было бы запустить db.repairDatabase(), чтобы увидеть какие-либо преимущества
просмотр размеров индексов для каждой коллекции и удаление индексов ... это кажется очень утомительным и должно быть сделано для каждой коллекции.Есть ли какая-нибудь утилита для проверки индексов по нескольким БД и коллекциям?
ограничивают память с помощью cgroups https://www.techpaste.com/2016/04/limit-mongodb-memory-usage/
Какие результаты были у людей для любого из этихПодходит?
Пока что немного диагностики
> db.serverStatus().wiredTiger.cache
{
"application threads page read from disk to cache count" : 414211,
"application threads page read from disk to cache time (usecs)" : 636568228,
"application threads page write from cache to disk count" : 458323,
"application threads page write from cache to disk time (usecs)" : 9120699,
"bytes belonging to page images in the cache" : 1107448266,
"bytes belonging to the lookaside table in the cache" : 182,
"bytes currently in the cache" : 1224100958,
"bytes not belonging to page images in the cache" : 116652691,
"bytes read into cache" : 12456110599,
"bytes written from cache" : 8781586219,
"checkpoint blocked page eviction" : 3,
"eviction calls to get a page" : 440255,
"eviction calls to get a page found queue empty" : 64661,
"eviction calls to get a page found queue empty after locking" : 3988,
"eviction currently operating in aggressive mode" : 0,
"eviction empty score" : 0,
"eviction passes of a file" : 39052,
"eviction server candidate queue empty when topping up" : 2703,
"eviction server candidate queue not empty when topping up" : 2067,
"eviction server evicting pages" : 0,
"eviction server slept, because we did not make progress with eviction" : 58773,
"eviction server unable to reach eviction goal" : 0,
"eviction state" : 32,
"eviction walk target pages histogram - 0-9" : 5172,
"eviction walk target pages histogram - 10-31" : 31097,
"eviction walk target pages histogram - 128 and higher" : 0,
"eviction walk target pages histogram - 32-63" : 2131,
"eviction walk target pages histogram - 64-128" : 652,
"eviction walks abandoned" : 19448,
"eviction walks gave up because they restarted their walk twice" : 9071,
"eviction walks gave up because they saw too many pages and found no candidates" : 58,
"eviction walks gave up because they saw too many pages and found too few candidates" : 4,
"eviction walks reached end of tree" : 37987,
"eviction walks started from root of tree" : 28845,
"eviction walks started from saved location in tree" : 10207,
"eviction worker thread active" : 4,
"eviction worker thread created" : 0,
"eviction worker thread evicting pages" : 371587,
"eviction worker thread removed" : 0,
"eviction worker thread stable number" : 0,
"failed eviction of pages that exceeded the in-memory maximum count" : 4,
"failed eviction of pages that exceeded the in-memory maximum time (usecs)" : 17,
"files with active eviction walks" : 0,
"files with new eviction walks started" : 28916,
"force re-tuning of eviction workers once in a while" : 0,
"hazard pointer blocked page eviction" : 606,
"hazard pointer check calls" : 372300,
"hazard pointer check entries walked" : 17002211,
"hazard pointer maximum array length" : 9,
"in-memory page passed criteria to be split" : 332,
"in-memory page splits" : 162,
"internal pages evicted" : 114,
"internal pages split during eviction" : 0,
"leaf pages split during eviction" : 434,
"lookaside score" : 0,
"lookaside table entries" : 0,
"lookaside table insert calls" : 0,
"lookaside table remove calls" : 0,
"maximum bytes configured" : 1535115264,
"maximum page size at eviction" : 0,
"modified pages evicted" : 10058,
"modified pages evicted by application threads" : 0,
"overflow pages read into cache" : 0,
"page split during eviction deepened the tree" : 0,
"page written requiring lookaside records" : 0,
"pages currently held in the cache" : 36933,
"pages evicted because they exceeded the in-memory maximum count" : 163,
"pages evicted because they exceeded the in-memory maximum time (usecs)" : 8083,
"pages evicted because they had chains of deleted items count" : 54,
"pages evicted because they had chains of deleted items time (usecs)" : 43123,
"pages evicted by application threads" : 0,
"pages queued for eviction" : 461533,
"pages queued for urgent eviction" : 802,
"pages queued for urgent eviction during walk" : 11,
"pages read into cache" : 415223,
"pages read into cache after truncate" : 53,
"pages read into cache after truncate in prepare state" : 0,
"pages read into cache requiring lookaside entries" : 0,
"pages read into cache requiring lookaside for checkpoint" : 0,
"pages read into cache skipping older lookaside entries" : 0,
"pages read into cache with skipped lookaside entries needed later" : 0,
"pages read into cache with skipped lookaside entries needed later by checkpoint" : 0,
"pages requested from the cache" : 80308407,
"pages seen by eviction walk" : 581486,
"pages selected for eviction unable to be evicted" : 625,
"pages walked for eviction" : 4385068,
"pages written from cache" : 481133,
"pages written requiring in-memory restoration" : 5,
"percentage overhead" : 8,
"tracked bytes belonging to internal pages in the cache" : 4786604,
"tracked bytes belonging to leaf pages in the cache" : 1219314354,
"tracked dirty bytes in the cache" : 0,
"tracked dirty pages in the cache" : 0,
"unmodified pages evicted" : 361519
}
Этот полезный поток дал несколько советов Есть ли возможность ограничить использование памяти mongodb?
> mem.tcmalloc.formattedString
------------------------------------------------
MALLOC: 1708093144 ( 1629.0 MiB) Bytes in use by application
MALLOC: + 36327424 ( 34.6 MiB) Bytes in page heap freelist
MALLOC: + 41100536 ( 39.2 MiB) Bytes in central cache freelist
MALLOC: + 3532640 ( 3.4 MiB) Bytes in transfer cache freelist
MALLOC: + 29304016 ( 27.9 MiB) Bytes in thread cache freelists
MALLOC: + 9396480 ( 9.0 MiB) Bytes in malloc metadata
MALLOC: ------------
MALLOC: = 1827754240 ( 1743.1 MiB) Actual memory used (physical + swap)
MALLOC: + 506347520 ( 482.9 MiB) Bytes released to OS (aka unmapped)
MALLOC: ------------
MALLOC: = 2334101760 ( 2226.0 MiB) Virtual address space used
MALLOC:
MALLOC: 70088 Spans in use
MALLOC: 50 Thread heaps in use
MALLOC: 4096 Tcmalloc page size
------------------------------------------------