Похоже на uwsgi, порождающее больше процессов, чем ожидалось , но так как этот процесс остается без ответа, и у меня не хватает репутации, чтобы спросить там, раскручиваю это снова. Это ( UWSGI слишком быстро убивает рабочих ) также может иметь значение.
Моя ситуация: у меня есть приложение для колб, uWSGI и nginx. Несистемное выполнение моей конфигурации uWSGI работает как шарм. Когда я запускаю uWSGI с помощью emperor, я вижу один запущенный процесс. Когда я добавляю файл .ini в расположение вассалов, эта штука взрывается, запускает и убивает процессы настолько быстро, что загрузка процессора увеличивается до 100%, а виртуальная машина, над которой я работаю, падает. Кажется, не имеет значения, что такое содержимое .ini файла (пробовал пустой).
/ некоторые / NFS / доля / рабочие / конф
PYTHONPATH=/some/nfs/share/production/repo
PATH=/some/nfs/share/production/bin:/sbin:/bin:/usr/sbin:/usr/bin
UWSGI_BIN=/some/nfs/share/production/bin/uwsgi
UWSGI_INI=/some/nfs/share/workers/emperor.ini
emperor.service
[Unit]
Description=uWSGI Emperor
After=syslog.target
[Service]
User=myuser
Group=mygroup
EnvironmentFile=/some/nfs/share/workers/conf
ExecStart=/bin/sh -c '${UWSGI_BIN} --ini ${UWSGI_INI}'
RuntimeDirectory=uwsgi
RestartSec=5
KillSignal=SIGQUIT
Type=notify
StandardError=syslog
NotifyAccess=all
[Install]
WantedBy=multi-user.target
emperor.ini
[uwsgi]
emperor = /some/nfs/share/uwsgi/
uid = myuser
gid = mygroup
фактический сервис .ini
[uwsgi]
wsgi-file = /some/nfs/share/production/repo/app/main.py
callable = create_app
# load some variables
for-readline = /some/nfs/share/workers/conf
env = %(_)
endfor =
plugins = python
wsgi-disable-file-wrapper = true
socket = /some/nfs/share/workers/uwsgi/%n.sock
pidfile = /some/nfs/share/workers/uwsgi/%n.pid
chmod-socket = 664
vacuum = true
die-on-term = true
close-on-exec = true
другой фиктивный файл .ini, который я пробовал, который вызывает ту же проблему (да, он пустой):
[uwsgi]
Если я добавлю запись в файл emepror.ini, я получу:
emperor.ini
[uwsgi]
emperor = /some/nfs/share/uwsgi/
uid = myuser
gid = evfold
logger = file:/some/nfs/share/uwsgi/emperor.log
В свою очередь, это вызовет два процесса вместо одного:
● emperor.service - uWSGI Emperor
Loaded: loaded (/etc/systemd/system/emperor.service; disabled; vendor preset: disabled)
Active: active (running) since Thu 2018-07-05 11:09:40 EDT; 2min 7s ago
Main PID: 19777 (uwsgi)
Status: "The Emperor is governing 0 vassals"
CGroup: /system.slice/emperor.service
├─19777 /some/nfs/share/production/bin/uwsgi --ini /some/nfs/share/workers/emperor.ini
└─19778 /some/nfs/share/production/bin/uwsgi --ini /some/nfs/share/workers/emperor.ini
Jul 05 11:09:40 backend systemd[1]: Starting uWSGI Emperor...
Jul 05 11:09:40 backend sh[19777]: [uWSGI] getting INI configuration from /some/nfs/share/workers/emperor.ini
Jul 05 11:09:40 backend sh[19777]: [uWSGI] getting INI configuration from /some/nfs/share/workers/emperor.ini
Jul 05 11:09:40 backend systemd[1]: Started uWSGI Emperor.
этот файл журнала до сих пор (обратите внимание, что все продублировано, скорее всего, потому что экземпляр регистратора является отдельным:
*** Starting uWSGI 2.0.16 (64bit) on [Thu Jul 5 11:09:40 2018] ***
*** Starting uWSGI 2.0.16 (64bit) on [Thu Jul 5 11:09:40 2018] ***
compiled with version: 4.8.2 20140120 (Red Hat 4.8.2-15) on 12 February 2018 22:16:17
compiled with version: 4.8.2 20140120 (Red Hat 4.8.2-15) on 12 February 2018 22:16:17
os: Linux-3.10.0-327.3.1.el7.x86_64 #1 SMP Wed Dec 9 14:09:15 UTC 2015
os: Linux-3.10.0-327.3.1.el7.x86_64 #1 SMP Wed Dec 9 14:09:15 UTC 2015
nodename: backend
nodename: backend
machine: x86_64
machine: x86_64
clock source: unix
clock source: unix
pcre jit disabled
pcre jit disabled
detected number of CPU cores: 4
detected number of CPU cores: 4
current working directory: /
current working directory: /
detected binary path: /some/nfs/share/production/bin/uwsgi
detected binary path: /some/nfs/share/production/bin/uwsgi
dropping root privileges as early as possible
dropping root privileges as early as possible
your processes number limit is 31212
your processes number limit is 31212
your memory page size is 4096 bytes
your memory page size is 4096 bytes
detected max file descriptor number: 1024
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
thunder lock: disabled (you can enable it with --thunder-lock)
*** starting uWSGI Emperor ***
*** starting uWSGI Emperor ***
dropping root privileges after socket binding
dropping root privileges after socket binding
Python version: 3.5.4 |Continuum Analytics, Inc.| (default, Aug 14 2017, 13:26:58) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
Python version: 3.5.4 |Continuum Analytics, Inc.| (default, Aug 14 2017, 13:26:58) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
*** Python threads support is disabled. You can enable it with --enable-threads ***
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x1cf2180
Python main interpreter initialized at 0x1cf2180
dropping root privileges after plugin initialization
dropping root privileges after plugin initialization
your mercy for graceful operations on workers is 60 seconds
your mercy for graceful operations on workers is 60 seconds
*** Operational MODE: no-workers ***
*** Operational MODE: no-workers ***
dropping root privileges after application loading
dropping root privileges after application loading
spawned uWSGI master process (pid: 19777)
spawned uWSGI master process (pid: 19777)
если я затем добавлю production.ini в папку vassals, я получу следующий журнал:
*** has_emperor mode detected (fd: 10) ***
*** has_emperor mode detected (fd: 10) ***
[uWSGI] getting INI configuration from /some/nfs/share/workers/emperor.ini
[uWSGI] getting INI configuration from /some/nfs/share/workers/emperor.ini
[uWSGI] getting INI configuration from production.ini
[uWSGI] getting INI configuration from production.ini
open("./python_plugin.so"): No such file or directory [core/utils.c line 3713]
open("./python_plugin.so"): No such file or directory [core/utils.c line 3713]
!!! UNABLE to load uWSGI plugin: ./python_plugin.so: cannot open shared object file: No such file or directory !!!
!!! UNABLE to load uWSGI plugin: ./python_plugin.so: cannot open shared object file: No such file or directory !!!
Thu Jul 5 11:22:29 2018 - [emperor] vassal production.ini is now loyal
Thu Jul 5 11:22:29 2018 - [emperor] vassal production.ini is now loyal
ошибка python_plugin, похоже, никак не влияет на выполнение, я могу удалить plugins = python из файла .ini, и это не сильно изменится.
На этом этапе более 50 работников, связанных с работниками uWSGI выше, и машина перестает отвечать на запросы.
Любая помощь приветствуется:)
РЕДАКТИРОВАТЬ : я смог изолировать emperor.service один раз, без регистрации и с файлом production.ini в вассалах, это выглядит так (вместо этого я предполагаю, что сосуды должны быть вложенными внутри основного PID там?)
● emperor.service - uWSGI Emperor
Loaded: loaded (/etc/systemd/system/emperor.service; disabled; vendor preset: disabled)
Active: active (running) since Thu 2018-07-05 12:01:57 EDT; 12min ago
Main PID: 25432 (uwsgi)
Status: "The Emperor is governing 1 vassals"
CGroup: /system.slice/emperor.service
├─25432 /some/nfs/share/production/bin/uwsgi --ini /some/nfs/share/workers/emperor.ini
├─25960 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25961 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25962 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25963 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25964 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25965 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25966 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25967 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25968 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25969 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25970 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25971 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25972 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25973 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25974 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25975 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25976 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25977 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25978 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25979 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25980 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25981 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25982 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25983 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25984 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25985 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25986 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25987 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25988 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25989 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25990 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25991 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25992 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25993 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25994 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25995 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25996 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25997 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25998 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─25999 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26000 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26001 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26002 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26003 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26004 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26005 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26006 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26007 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26008 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26014 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26015 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26030 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26035 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26037 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26038 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26039 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26040 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26044 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26045 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26048 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26049 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26051 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26052 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26054 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26055 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26056 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26057 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26059 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26062 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26063 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26064 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26065 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26066 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26068 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26069 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26070 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26071 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26074 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26075 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26076 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26078 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26080 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26081 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26083 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26084 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26086 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26088 /some/nfs/share/production/bin/uwsgi --ini production.ini
├─26090 /some/nfs/share/production/bin/uwsgi --ini production.ini
└─26091 /some/nfs/share/production/bin/uwsgi --ini production.ini
Jul 05 12:14:29 backend uwsgi[25432]: *** Starting uWSGI 2.0.16 (64bit) on [Thu Jul 5 12:14:29 2018] ***
Jul 05 12:14:29 backend uwsgi[25432]: compiled with version: 4.8.2 20140120 (Red Hat 4.8.2-15) on 12 February 2018 22:16:17
Jul 05 12:14:29 backend uwsgi[25432]: os: Linux-3.10.0-327.3.1.el7.x86_64 #1 SMP Wed Dec 9 14:09:15 UTC 2015
Jul 05 12:14:29 backend uwsgi[25432]: nodename: backend
Jul 05 12:14:29 backend uwsgi[25432]: machine: x86_64
Jul 05 12:14:29 backend uwsgi[25432]: clock source: unix
Jul 05 12:14:29 backend uwsgi[25432]: pcre jit disabled
Jul 05 12:14:29 backend uwsgi[25432]: detected number of CPU cores: 4
Jul 05 12:14:29 backend uwsgi[25432]: current working directory: /some/nfs/share/production/bin/workers/uwsgi
Jul 05 12:14:29 backend uwsgi[25432]: writing pidfile to /some/nfs/share/production/bin/workers/uwsgi/production.pid