Мне нужно проверить установку и развертывание службы, которая работает на CentOS7 и подключается к базе данных MongoDB 3.4. В производственном процессе служба и база данных развернуты на отдельных серверах. AFAIK kitchen-dokken не поддерживает многоузловую поддержку, поэтому планируется настроить тестовый узел CentOS как со службой, так и с базой данных.
s c -mongodb по умолчанию устанавливает экземпляр MongoDB 3.2, все работает нормально с минимальной конфигурацией. Однако после переключения на MongoDB 3.4 запуск экземпляра базы данных всегда завершается неудачно с таймаутом.
metadata.rb
name 'mongo34-centos7'
version '0.1.0'
chef_version '>= 13.0'
depends 'sc-mongodb'
Berksfile
source 'https://supermarket.chef.io'
metadata
** .kitchen.yml **
---
driver:
name: dokken
chef_version: latest
transport:
name: dokken
provisioner:
name: dokken
always_update_cookbooks: true
client_rb:
chef_license: accept-no-persist
platforms:
- name: centos-7
driver:
image: dokken/centos-7
privileged: true
pid_one_command: /usr/lib/systemd/systemd
suites:
- name: default
run_list:
- recipe[sc-mongodb::default]
- name: troisquatre
run_list:
- recipe[sc-mongodb::default]
attributes:
mongodb:
package_version: 3.4.11
Набор по умолчанию устанавливает и запускает экземпляр MongoDB 3.2, все в порядке.
$ kitchen converge default
-----> Starting Kitchen (v1.24.0)
-----> Converging <default-centos-7>...
Creating kitchen sandbox in /home/mtarin/.dokken/kitchen_sandbox/9df3916bce-default-centos-7
Preparing dna.json
Resolving cookbook dependencies with Berkshelf 7.0.8...
Removing non-cookbook files before transfer
Preparing validation.pem
Preparing client.rb
Starting Chef Infra Client, version 15.7.32
Creating a new client identity for default-centos-7 using the validator key.
resolving cookbooks for run list: ["sc-mongodb::default"]
Synchronizing Cookbooks:
- yum (5.1.0)
- build-essential (8.2.1)
- sc-mongodb (1.2.0)
- seven_zip (3.1.2)
- apt (7.2.0)
- mingw (2.1.0)
- windows (6.0.1)
Installing Cookbook Gems:
Compiling Cookbooks...
Converging 14 resources
Recipe: sc-mongodb::mongodb_org_repo
...
SKIPPING A LOT OF VERBOSE STEPS
...
* execute[mongodb-systemctl-daemon-reload-mongod] action run
- execute systemctl daemon-reload
* service[mongod] action enable (up to date)
* service[mongod] action start
- start service service[mongod]
* service[mongod] action restart
- restart service service[mongod]
Running handlers:
Running handlers complete
Chef Infra Client finished, 5/22 resources updated in 07 seconds
Finished converging <default-centos-7> (0m18.04s).
Однако, после переключения на MongoDB 3.4 запуск экземпляра базы данных всегда завершается неудачно с таймаутом:
$ kitchen converge troisquatre -----> Starting Kitchen (v1.24.0)
-----> Creating <troisquatre-centos-7>...
Creating kitchen sandbox at /home/mtarin/.dokken/kitchen_sandbox/9df3916bce-troisquatre-centos-7
Creating verifier sandbox at /home/mtarin/.dokken/verifier_sandbox/9df3916bce-troisquatre-centos-7
Building work image..
Creating container 9df3916bce-troisquatre-centos-7
Finished creating <troisquatre-centos-7> (0m2.70s).
-----> Converging <troisquatre-centos-7>...
Creating kitchen sandbox in /home/mtarin/.dokken/kitchen_sandbox/9df3916bce-troisquatre-centos-7
Preparing dna.json
Resolving cookbook dependencies with Berkshelf 7.0.8...
/etc/host.conf: line 6: bad command `nospoof on'
/etc/host.conf: line 7: bad command `spoofalert on'
Removing non-cookbook files before transfer
Preparing validation.pem
Preparing client.rb
Starting Chef Infra Client, version 15.7.32
Creating a new client identity for troisquatre-centos-7 using the validator key.
resolving cookbooks for run list: ["sc-mongodb::default"]
Synchronizing Cookbooks:
- yum (5.1.0)
- build-essential (8.2.1)
- sc-mongodb (1.2.0)
- apt (7.2.0)
- seven_zip (3.1.2)
- mingw (2.1.0)
- windows (6.0.1)
Installing Cookbook Gems:
Compiling Cookbooks...
Converging 14 resources
Recipe: sc-mongodb::mongodb_org_repo
...
SKIPPING A LOT OF VERBOSE STEPS
...
* execute[mongodb-systemctl-daemon-reload] action run
- execute systemctl daemon-reload
* yum_package[mongodb-org] action install
- install version 3.4.11-1.el7 of package mongodb-org
Recipe: sc-mongodb::default
* template[/etc/sysconfig/mongodb] action create
- update content in file /etc/sysconfig/mongodb from a35762 to 803da0
--- /etc/sysconfig/mongodb 2020-02-06 15:18:11.171032500 +0000
+++ /etc/sysconfig/.chef-mongodb20200206-108-1q2ajqe 2020-02-06 15:18:22.216211217 +0000
@@ -1,2 +1,10 @@
-ENABLE_MONGODB=no
+#
+# Automatically Generated by Chef, do not edit directly!
+#
+
+CONFIGFILE="/etc/mongod.conf"
+DAEMON="/usr/bin/$NAME"
+DAEMON_OPTS="--config /etc/mongod.conf"
+DAEMON_USER="mongod"
+ENABLE_MONGODB="yes"
* template[/etc/mongod.conf] action create (up to date)
* directory[/var/log/mongodb] action create (up to date)
* directory[/var/lib/mongo] action create (up to date)
* execute[mongodb-systemctl-daemon-reload-mongod] action nothing (skipped due to action :nothing)
* template[/etc/init.d/mongod] action create (up to date)
* service[mongod] action enable (up to date)
* service[mongod] action start
================================================================================
Error executing action `start` on resource 'service[mongod]'
================================================================================
Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '1'
---- Begin output of /usr/bin/systemctl --system start mongod ----
STDOUT:
STDERR: Job for mongod.service failed because a timeout was exceeded. See "systemctl status mongod.service" and "journalctl -xe" for details.
---- End output of /usr/bin/systemctl --system start mongod ----
Ran /usr/bin/systemctl --system start mongod returned 1
Resource Declaration:
---------------------
# In /opt/kitchen/cache/cookbooks/sc-mongodb/definitions/mongodb.rb
222: service new_resource.name do
223: supports status: true, restart: true
224: action new_resource.service_action
225: new_resource.service_notifies.each do |service_notify|
226: notifies :run, service_notify
227: end
228: notifies :run, 'ruby_block[config_replicaset]', :immediately if new_resource.is_replicaset && new_resource.auto_configure_replicaset
229: notifies :run, 'ruby_block[config_sharding]', :immediately if new_resource.is_mongos && new_resource.auto_configure_sharding
230: # we don't care about a running mongodb service in these cases, all we need is stopping it
231: ignore_failure true if new_resource.name == 'mongodb'
232: end
233:
234: # replicaset
235: if new_resource.is_replicaset && new_resource.auto_configure_replicaset
236: rs_nodes = search(
237: :node,
238: "mongodb_cluster_name:#{new_resource.cluster_name} AND "\
239: 'mongodb_is_replicaset:true AND '\
240: "mongodb_config_mongod_replication_replSetName:#{new_resource.replicaset_name} AND "\
241: "chef_environment:#{node.chef_environment}"
242: )
243:
244: ruby_block 'config_replicaset' do
245: block do
246: MongoDB.configure_replicaset(node, replicaset_name, rs_nodes) unless new_resource.replicaset.nil?
247: end
248: action :nothing
249: end
250:
251: ruby_block 'run_config_replicaset' do
252: block {}
253: notifies :run, 'ruby_block[config_replicaset]'
254: end
255: end
256:
Compiled Resource:
------------------
# Declared in /opt/kitchen/cache/cookbooks/sc-mongodb/definitions/mongodb.rb:222:in `block in from_file'
service("mongod") do
params {:mongodb_type=>"mongod", :action=>[:enable, :start], :logpath=>"/var/log/mongodb/mongod.log", :configservers=>[], :replicaset=>nil, :notifies=>[], :only_if=>[], :name=>"mongod"}
action [:enable, :start]
default_guard_interpreter :default
declared_type :service
cookbook_name "sc-mongodb"
recipe_name "default"
supports {:status=>true, :restart=>true}
running false
enabled true
masked false
end
System Info:
------------
chef_version=15.7.32
platform=centos
platform_version=7.7.1908
ruby=ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-linux]
program_name=/opt/chef/embedded/bin/chef-client
executable=/opt/chef/embedded/bin/chef-client
...
MONGODB IS THEN RESTARTED, AND OF COURSE THAT ALSO FAILS
...
Running handlers:
[2020-02-06T15:21:23+00:00] ERROR: Running exception handlers
Running handlers complete
[2020-02-06T15:21:23+00:00] ERROR: Exception handlers complete
Chef Infra Client failed. 13 resources updated in 04 minutes 22 seconds
[2020-02-06T15:21:23+00:00] FATAL: Stacktrace dumped to /opt/kitchen/cache/chef-stacktrace.out
[2020-02-06T15:21:23+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2020-02-06T15:21:23+00:00] FATAL: Chef::Exceptions::MultipleFailures: Multiple failures occurred:
* Mixlib::ShellOut::ShellCommandFailed occurred in Chef Infra Client run: service[mongod] (sc-mongodb::default line 222) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of /usr/bin/systemctl --system start mongod ----
STDOUT:
STDERR: Job for mongod.service failed because a timeout was exceeded. See "systemctl status mongod.service" and "journalctl -xe" for details.
---- End output of /usr/bin/systemctl --system start mongod ----
Ran /usr/bin/systemctl --system start mongod returned 1
* Mixlib::ShellOut::ShellCommandFailed occurred in delayed notification: service[mongod] (sc-mongodb::default line 222) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of /usr/bin/systemctl --system restart mongod ----
STDOUT:
STDERR: Job for mongod.service failed because a timeout was exceeded. See "systemctl status mongod.service" and "journalctl -xe" for details.
---- End output of /usr/bin/systemctl --system restart mongod ----
Ran /usr/bin/systemctl --system restart mongod returned 1
При входе в контейнер я получаю следующую информацию:
[root@dokken /]# systemctl status mongod
● mongod.service - High-performance, schema-free document-oriented database
Loaded: loaded (/usr/lib/systemd/system/mongod.service; enabled; vendor preset: disabled)
Active: failed (Result: timeout) since Thu 2020-02-06 15:21:23 UTC; 58min ago
Docs: https://docs.mongodb.org/manual
Process: 12329 ExecStart=/usr/bin/mongod $OPTIONS (code=exited, status=0/SUCCESS)
Process: 12328 ExecStartPre=/usr/bin/chmod 0755 /var/run/mongodb (code=exited, status=0/SUCCESS)
Process: 12327 ExecStartPre=/usr/bin/chown mongod:mongod /var/run/mongodb (code=exited, status=0/SUCCESS)
Process: 12326 ExecStartPre=/usr/bin/mkdir -p /var/run/mongodb (code=exited, status=0/SUCCESS)
Feb 06 15:19:53 dokken systemd[1]: Starting High-performance, schema-free document-oriented database...
Feb 06 15:19:53 dokken mongod[12329]: about to fork child process, waiting until server is ready for connections.
Feb 06 15:19:53 dokken mongod[12329]: forked process: 12331
Feb 06 15:19:53 dokken systemd[1]: New main PID 12331 does not belong to service, and PID file is not owned by root. Refusing.
Feb 06 15:19:53 dokken systemd[1]: New main PID 12331 does not belong to service, and PID file is not owned by root. Refusing.
Feb 06 15:21:23 dokken systemd[1]: mongod.service start operation timed out. Terminating.
Feb 06 15:21:23 dokken systemd[1]: Failed to start High-performance, schema-free document-oriented database.
Feb 06 15:21:23 dokken systemd[1]: Unit mongod.service entered failed state.
Feb 06 15:21:23 dokken systemd[1]: mongod.service failed.
Настройка MongoDB и systemd с сервисным файлом все в порядке:
[root@dokken /]# cat /etc/mongod.conf
#
# Automatically Generated by Chef, do not edit directly!
#
---
net:
port: 27017
bindIp: 0.0.0.0
systemLog:
destination: file
logAppend: true
path: "/var/log/mongodb/mongod.log"
processManagement:
fork: true
pidFilePath: "/var/run/mongodb/mongod.pid"
storage:
journal:
enabled: true
dbPath: "/var/lib/mongo"
engine: wiredTiger
[root@dokken /]# cat /etc/systemd/system/multi-user.target.wants/mongod.service
[Unit]
Description=High-performance, schema-free document-oriented database
After=network.target
Documentation=https://docs.mongodb.org/manual
[Service]
User=mongod
Group=mongod
Environment="OPTIONS=-f /etc/mongod.conf"
ExecStart=/usr/bin/mongod $OPTIONS
ExecStartPre=/usr/bin/mkdir -p /var/run/mongodb
ExecStartPre=/usr/bin/chown mongod:mongod /var/run/mongodb
ExecStartPre=/usr/bin/chmod 0755 /var/run/mongodb
PermissionsStartOnly=true
PIDFile=/var/run/mongodb/mongod.pid
Type=forking
# file size
LimitFSIZE=infinity
# cpu time
LimitCPU=infinity
# virtual memory size
LimitAS=infinity
# open files
LimitNOFILE=64000
# processes/threads
LimitNPROC=64000
# locked memory
LimitMEMLOCK=infinity
# total threads (user+kernel)
TasksMax=infinity
TasksAccounting=false
# Recommended limits for for mongod as specified in
# http://docs.mongodb.org/manual/reference/ulimit/#recommended-settings
[Install]
WantedBy=multi-user.target
Факт «забавный», я могу запустить MongoDB напрямую:
root@dokken /]# mongod -f /etc/mongod.conf run
about to fork child process, waiting until server is ready for connections.
forked process: 12433
child process started successfully, parent exiting
[root@dokken /]# ps aux | grep mongod
root 12433 4.6 1.8 972380 35576 ? Sl 16:28 0:00 mongod -f /etc/mongod.conf run
root 12458 0.0 0.0 12524 972 pts/0 S+ 16:28 0:00 grep --color=auto mongod
Но если я попытаюсь запустить его с помощью systemd, снова произойдет сбой:
[root@dokken /]# service mongod start
Starting mongod (via systemctl): Job for mongod.service failed because the control process exited with error code. See "systemctl status mongod.service" and "journalctl -xe" for details.
[FAILED]
Я не могу обернуть это вокруг головы ... Любая идея, почему MongoDB не будет работать как сервис, и как решить эту проблему?
Примечание: Поскольку я уже знаю, что могу запустить mongod вручную, я не ищу обходной путь, но я действительно заинтересован в понимании выясняю, почему это не работает с systemd.