Сбой MongoDB с транзакцией Slow WT в производственной среде - PullRequest
Новая
       15

Сбой MongoDB с транзакцией Slow WT в производственной среде

0 голосов
/ 21 декабря 2018

Поведение проблемы

  • MongoDB часто дает сбой в производственной среде.по крайней мере, один раз в день. Перед каждым сбоем в журнале появляется только то, что указано ниже: «Медленная транзакция WT».

Подозрение в журналах перед сбоем [grep]: 

2018-12-05T16:59:02.569+0000 D STORAGE  [conn1170] Slow WT transaction. Lifetime of SnapshotId 1356150 was 154ms
    2018-12-05T16:59:38.214+0000 D STORAGE  [conn4] Slow WT transaction. Lifetime of SnapshotId 1357284 was 154ms
    2018-12-05T16:59:38.390+0000 D STORAGE  [conn4] Slow WT transaction. Lifetime of SnapshotId 1357291 was 103ms
    2018-12-05T17:00:03.076+0000 D STORAGE  [conn1170] Slow WT transaction. Lifetime of SnapshotId 1357925 was 158ms
    2018-12-05T17:00:24.940+0000 D STORAGE  [conn4] Slow WT transaction. Lifetime of SnapshotId 1358710 was 115ms
    2018-12-05T17:00:46.096+0000 D STORAGE  [conn8] Slow WT transaction. Lifetime of SnapshotId 1359381 was 105ms
    2018-12-05T17:00:59.304+0000 D STORAGE  [conn4] Slow WT transaction. Lifetime of SnapshotId 1359903 was 142ms
    2018-12-05T17:01:13.309+0000 D STORAGE  [conn26] Slow WT transaction. Lifetime of SnapshotId 1360311 was 164ms
    2018-12-05T17:01:13.554+0000 D STORAGE  [conn26] Slow WT transaction. Lifetime of SnapshotId 1360362 was 143ms
    2018-12-05T17:01:23.541+0000 D STORAGE  [conn4] Slow WT transaction. Lifetime of SnapshotId 1360671 was 116ms
    2018-12-05T17:01:23.761+0000 D STORAGE  [conn4] Slow WT transaction. Lifetime of SnapshotId 1360679 was 110ms
    2018-12-05T17:01:29.340+0000 D STORAGE  [conn4] Slow WT transaction. Lifetime of SnapshotId 1360880 was 203ms

Среда: ОС + браузер / клиент + версия

Информация о версии MongoDB: 

db version v3.6.8
git version: 6bc9ed599c3fa164703346a22bad17e33fa913e4
OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
allocator: tcmalloc
modules: none
build environment:
    distmod: rhel70
    distarch: x86_64
    target_arch: x86_64

Информация об ОС: 

NAME=CentOS Linux
VERSION=7 (Core)
ID=centos
ID_LIKE: rhel fedora
VERSION_ID=7
PRETTY_NAME=CentOS Linux 7 (Core)

Нижефайл конфигурации -

mongod.conf 

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log

# Where and how to store data.
storage:
  dbPath: /apps/mongodb/data
  journal:
    enabled: true
#  engine:
#  mmapv1:
#  wiredTiger

# how the process runs
processManagement:
  fork: true  # fork and run in background
  pidFilePath: /var/run/mongodb/mongod.pid
  timeZoneInfo: /usr/share/zoneinfo

# network interfaces
net:
  port: 27017
  bindIp: 127.0.0.1  # Listen to local interface only, comment to listen on all interfaces.

#security:

#operationProfiling:

#replication:

#sharding:

## Enterprise-Only Options

#auditLog:

#snmp:

Возможное решение [Попробовал]

  • Попытка увеличения ограничений системы, но выдача продолжается через 2 дня.

Текущий предел: 

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 64045
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 128000
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 64045
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Пожалуйста, дайте мне знать, если вам нужна дополнительная информация.

...