Я использую MySQL Кластер Percona XtraDB с 3 узлами и служба MySQL на одном узле не работает из-за переполнения диска. после устранения проблемы мы пытаемся запустить MySQL на этом узле, но возникает следующая ошибка: (до этой проблемы все узлы работали нормально)
$ sudo /etc/init.d/mysql start
[....] Starting mysql (via systemctl): mysql.serviceJob for mysql.service failed because the control process exited with error code.
See "systemctl status mysql.service" and "journalctl -xe" for details.
failed!
$ sudo systemctl status mysql.service
● mysql.service - LSB: Start and stop the mysql (Percona XtraDB Cluster) daemon
Loaded: loaded (/etc/init.d/mysql; generated)
Drop-In: /etc/systemd/system/mysql.service.d
└─override.conf
Active: failed (Result: exit-code) since Tue 2020-03-24 11:57:55 +0430; 1h 2min ago
Docs: man:systemd-sysv-generator(8)
Process: 19397 ExecStart=/etc/init.d/mysql start (code=exited, status=1/FAILURE)
Mar 24 11:57:47 server-3 systemd[1]: Starting LSB: Start and stop the mysql (Percona XtraDB Cluster) daemon...
Mar 24 11:57:47 server-3 mysql[19397]: * Starting MySQL (Percona XtraDB Cluster) database server mysqld
Mar 24 11:57:55 server-3 mysql[19397]: * The server quit without updating PID file (/var/run/mysqld/mysqld.pid).
Mar 24 11:57:55 server-3 mysql[19397]: ...fail!
Mar 24 11:57:55 server-3 systemd[1]: mysql.service: Control process exited, code=exited status=1
Mar 24 11:57:55 server-3 systemd[1]: mysql.service: Failed with result 'exit-code'.
Mar 24 11:57:55 server-3 systemd[1]: Failed to start LSB: Start and stop the mysql (Percona XtraDB Cluster) daemon.
Операционная система: Ubuntu 18.04.2 LTS mysql Ver 14.14 Distrib 5.7.25-28, для debian- linux -gnu (x86_64) с использованием 7.0
также, пожалуйста, найдите журнал journalctl ниже:
$ sudo journalctl -xe
Journal file /var/log/journal/7e072ef87bdf452f8a9684d905f87daf/user-1863206337.journal is truncated, ignoring file.
Mar 24 19:05:22 server-3 pmm-agent[30366]: INFO[2020-03-24T19:05:22.235+04:30] time="2020-03-24T19:05:22+04:30" level=error msg="Error pinging mysqld: dial tcp 1
Mar 24 19:05:22 server-3 mysql[6820]: * The server quit without updating PID file (/var/run/mysqld/mysqld.pid).
Mar 24 19:05:22 server-3 mysql[6820]: ...fail!
Mar 24 19:05:22 server-3 systemd[1]: mysql.service: Control process exited, code=exited status=1
Mar 24 19:05:22 server-3 systemd[1]: mysql.service: Failed with result 'exit-code'.
Mar 24 19:05:22 server-3 systemd[1]: mysql.service: Failed with result 'exit-code'.
Mar 24 19:05:22 server-3 systemd[1]: mysql.service: Failed with result 'exit-code'.
Mar 24 19:05:22 server-3 systemd[1]: Failed to start LSB: Start and stop the mysql (Percona XtraDB Cluster) daemon.
-- Subject: Unit mysql.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit mysql.service has failed.
--
-- The result is RESULT.
Mar 24 19:05:22 server-3 sudo[6779]: pam_unix(sudo:session): session closed for user root
Mar 24 19:05:22 server-3 kernel: ACPI Error: SMBus/IPMI/GenericSerialBus write requires Buffer of length 66, found length 32 (20170831/exfield-427)
Mar 24 19:05:22 server-3 kernel: No Local Variables are initialized for Method [_PMM]
Mar 24 19:05:22 server-3 kernel: No Arguments are initialized for method [_PMM]
Mar 24 19:05:22 server-3 kernel: ACPI Error: Method parse/execution failed \_SB.PMI0._PMM, AE_AML_BUFFER_LIMIT (20170831/psparse-550)
Mar 24 19:05:22 server-3 kernel: ACPI Exception: AE_AML_BUFFER_LIMIT, Evaluating _PMM (20170831/power_meter-338)
Mar 24 19:05:23 server-3 snmpd[1805]: Connection from UDP: [192.168.2.10]:54620->[192.168.8.3]:161
Mar 24 19:05:23 server-3 snmpd[1805]: Connection from UDP: [192.168.2.10]:54647->[192.168.8.3]:161
Mar 24 19:05:25 server-3 pmm-agent[30366]: INFO[2020-03-24T19:05:25.550+04:30] time="2020-03-24T19:05:25+04:30" level=error msg="Error pinging mysqld: dial tcp 1
Mar 24 19:05:26 server-3 snmpd[1805]: Connection from UDP: [192.168.2.10]:55485->[192.168.8.3]:161
Mar 24 19:05:26 server-3 snmpd[1805]: Connection from UDP: [192.168.2.10]:55500->[192.168.8.3]:161
Mar 24 19:05:27 server-3 pmm-agent[30366]: INFO[2020-03-24T19:05:27.235+04:30] time="2020-03-24T19:05:27+04:30" level=error msg="Error pinging mysqld: dial tcp 1
Mar 24 19:05:27 server-3 kernel: ACPI Error: SMBus/IPMI/GenericSerialBus write requires Buffer of length 66, found length 32 (20170831/exfield-427)
Mar 24 19:05:27 server-3 kernel: No Local Variables are initialized for Method [_PMM]
Mar 24 19:05:27 server-3 kernel: No Arguments are initialized for method [_PMM]
Mar 24 19:05:27 server-3 kernel: ACPI Error: Method parse/execution failed \_SB.PMI0._PMM, AE_AML_BUFFER_LIMIT (20170831/psparse-550)
Mar 24 19:05:27 server-3 kernel: ACPI Exception: AE_AML_BUFFER_LIMIT, Evaluating _PMM (20170831/power_meter-338)
Mar 24 19:05:30 server-3 pmm-agent[30366]: INFO[2020-03-24T19:05:30.550+04:30] time="2020-03-24T19:05:30+04:30" level=error msg="Error pinging mysqld: dial tcp 1
Mar 24 19:05:31 server-3 pmm-agent[30366]: ERRO[2020-03-24T19:05:31.438+04:30] cannot select @@slow_query_log_file: dial tcp 127.0.0.1:3306: connect: connection
Mar 24 19:05:32 server-3 pmm-agent[30366]: INFO[2020-03-24T19:05:32.234+04:30] time="2020-03-24T19:05:32+04:30" level=error msg="Error pinging mysqld: dial tcp 1
Mar 24 19:05:32 server-3 kernel: ACPI Error: SMBus/IPMI/GenericSerialBus write requires Buffer of length 66, found length 32 (20170831/exfield-427)
Mar 24 19:05:32 server-3 kernel: No Local Variables are initialized for Method [_PMM]
Mar 24 19:05:32 server-3 kernel: No Arguments are initialized for method [_PMM]
Mar 24 19:05:32 server-3 kernel: ACPI Error: Method parse/execution failed \_SB.PMI0._PMM, AE_AML_BUFFER_LIMIT (20170831/psparse-550)
Mar 24 19:05:32 server-3 kernel: ACPI Exception: AE_AML_BUFFER_LIMIT, Evaluating _PMM (20170831/power_meter-338)
Mar 24 19:05:35 server-3 pmm-agent[30366]: INFO[2020-03-24T19:05:35.550+04:30] time="2020-03-24T19:05:35+04:30" level=error msg="Error pinging mysqld: dial tcp 1
Mar 24 19:05:36 server-3 sudo[7805]: admin_user : problem with defaults entries ; TTY=pts/0 ; PWD=/etc/systemd/system/mysql.service.d ; USER=root ;
Mar 24 19:05:36 server-3 sudo[7805]: admin_user : TTY=pts/0 ; PWD=/etc/systemd/system/mysql.service.d ; USER=root ; COMMAND=/bin/journalctl -xe
Mar 24 19:05:36 server-3 sudo[7805]: pam_unix(sudo:session): session opened for user root by admin_user(uid=0)
и найдите mysqld .log ниже:
/var/log$ sudo tail -n50 mysqld.log
2020-03-23T20:44:32.844540Z 36 [ERROR] WSREP: Node consistency compromised, aborting...
2020-03-23T20:44:32.844642Z 13 [ERROR] WSREP: Failed to apply trx: source: 89d92e92-6638-11e9-bdcb-7f2c26717188 version: 4 local: 0 state: APPLYING flags: 1 conn_id: 26113114 trx_id: 5580550846 seqnos (l: 151018291, g: 793430465, s: 793430462, d: 793430355, ts: 33700460567224654)
2020-03-23T20:44:32.844671Z 13 [ERROR] WSREP: Failed to apply trx 793430465 4 times
2020-03-23T20:44:32.844683Z 13 [ERROR] WSREP: Node consistency compromised, aborting...
2020-03-23T20:44:33.255602Z 21 [Note] WSREP: (c0cb4c75, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://192.168.8.1:4567 tcp://192.168.8.2:4567
2020-03-23T20:44:37.817120Z 21 [Note] WSREP: declaring node with index 0 suspected, timeout PT5S (evs.suspect_timeout)
2020-03-23T20:44:37.817139Z 21 [Note] WSREP: declaring node with index 1 suspected, timeout PT5S (evs.suspect_timeout)
2020-03-23T20:44:37.817147Z 21 [Note] WSREP: evs::proto(c0cb4c75, LEAVING, view_id(REG,82cab7cb,31)) suspecting node: 82cab7cb
2020-03-23T20:44:37.817152Z 21 [Note] WSREP: evs::proto(c0cb4c75, LEAVING, view_id(REG,82cab7cb,31)) suspected node without join message, declaring inactive
2020-03-23T20:44:37.817158Z 21 [Note] WSREP: evs::proto(c0cb4c75, LEAVING, view_id(REG,82cab7cb,31)) suspecting node: 89d92e92
2020-03-23T20:44:37.817161Z 21 [Note] WSREP: evs::proto(c0cb4c75, LEAVING, view_id(REG,82cab7cb,31)) suspected node without join message, declaring inactive
2020-03-23T20:44:37.817182Z 21 [Note] WSREP: Current view of cluster as seen by this node
view (view_id(NON_PRIM,82cab7cb,31)
memb {
c0cb4c75,0
}
joined {
}
left {
}
partitioned {
82cab7cb,0
89d92e92,0
}
)
2020-03-23T20:44:37.817203Z 21 [Note] WSREP: Current view of cluster as seen by this node
view ((empty))
2020-03-23T20:44:37.817358Z 21 [Note] WSREP: gcomm: closed
2020-03-23T20:44:37.817463Z 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
2020-03-23T20:44:37.817504Z 0 [Note] WSREP: Flow-control interval: [10000, 10000]
2020-03-23T20:44:37.817515Z 0 [Note] WSREP: Trying to continue unpaused monitor
2020-03-23T20:44:37.817522Z 0 [Note] WSREP: Received NON-PRIMARY.
2020-03-23T20:44:37.817524Z 13 [ERROR] WSREP: non-standard exception
2020-03-23T20:44:37.817530Z 0 [Note] WSREP: Shifting DONOR/DESYNCED -> OPEN (TO: 793430465)
2020-03-23T20:44:37.817568Z 13 [Note] WSREP: applier thread exiting (code:8)
2020-03-23T20:44:37.817584Z 13 [Note] WSREP: Starting Shutdown
2020-03-23T20:44:37.817587Z 0 [Note] WSREP: Received self-leave message.
2020-03-23T20:44:37.817529Z 36 [ERROR] WSREP: non-standard exception
2020-03-23T20:44:37.817643Z 36 [Note] WSREP: applier thread exiting (code:8)
2020-03-23T20:44:37.817656Z 36 [Note] WSREP: Starting Shutdown
2020-03-23T20:44:37.817759Z 0 [Note] WSREP: Received shutdown signal. Will sleep for 10 secs before initiating shutdown. pxc_maint_mode switched to SHUTDOWN
2020-03-23T20:44:37.817813Z 0 [Note] WSREP: Flow-control interval: [10000, 10000]
2020-03-23T20:44:37.817841Z 0 [Note] WSREP: Trying to continue unpaused monitor
2020-03-23T20:44:37.817852Z 0 [Note] WSREP: Received SELF-LEAVE. Closing connection.
2020-03-23T20:44:37.817860Z 0 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 793430465)
2020-03-23T20:44:37.817870Z 0 [Note] WSREP: RECV thread exiting 0: Success
2020-03-23T20:44:37.818185Z 21 [Note] WSREP: recv_thread() joined.
2020-03-23T20:44:37.818192Z 21 [Note] WSREP: Closing replication queue.
2020-03-23T20:44:37.818195Z 21 [Note] WSREP: Closing slave action queue.
2020-03-23T20:44:37.818247Z 21 [Note] WSREP: /usr/sbin/mysqld: Terminated.
Как запустить MySQL в этом узле?
заранее спасибо.