Я запускаю postgresql -10 в контейнере docker и подключаю его из другого установленного контейнера python 3.6 с помощью psycopg2.
Чтобы вставить 10 миллионов записей, я выполнил python script.
Во время вставки записей я получил эту ошибку, и скрипт завершился ненормально.
Traceback (most recent call last):
File "/usr/data_processing/data_processing_001.py", line 168, in <module>
extras.execute_values(c, insert_statement, np_cti[current_slice])
File "/home/pyuser/miniconda3/lib/python3.7/site-packages/psycopg2/extras.py", line 1278, in execute_values
cur.execute(b''.join(parts))
psycopg2.OperationalError: SSL SYSCALL error: EOF detected
Затем я подтвердил, что postgres был убит SIGABRT. Я проверял использование памяти с помощью команды free -h во время вставки записей, и было доступно 4,2 ГБ доступной памяти. Итак, я думаю, что причина не является недостатком памяти.
Почему postgres убит SIGABRT?
[root@docker-test ~]# abrt-cli list
id d84cad868c8dd05fbe92817ffaf798b1cd3194c5
reason: postgres killed by SIGABRT
time: 2020年02月21日 23時21分59秒
cmdline: 'postgres: 10/main: checkpointer process ' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' ''
uid: 1000 (lab)
count: 2
Directory: /var/spool/abrt/ccpp-2020-02-21-23:21:59-11443
<postgresql.log>
2020-02-21 22:33:49.418 JST,,,13,,5e4fdc2f.d,2,,2020-02-21 22:33:35 JST,,0,LOG,00000,"database system is ready to accept connections",,,,,,,,,""
2020-02-21 23:21:59.077 JST,,,16,,5e4fdc3d.10,1,,2020-02-21 22:33:49 JST,,0,PANIC,58030,"could not fsync file ""base/16385/16400"": Input/output error",,,,,,,,,""
2020-02-21 23:22:00.530 JST,,,13,,5e4fdc2f.d,3,,2020-02-21 22:33:35 JST,,0,LOG,00000,"checkpointer process (PID 16) was terminated by signal 6: Aborted",,,,,,,,,""
2020-02-21 23:22:00.532 JST,,,13,,5e4fdc2f.d,4,,2020-02-21 22:33:35 JST,,0,LOG,00000,"terminating any other active server processes",,,,,,,,,""
2020-02-21 23:22:00.532 JST,"pyuser","db_01",11060,"172.17.0.3:43316",5e4fe738.2b34,1,"INSERT",2020-02-21 23:20:40 JST,3/32339,6804,WARNING,57P02,"terminating connection because of crash of another server process","The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.","In a moment you should be able to reconnect to the database and repeat your command.",,,,,,,""
2020-02-21 23:22:00.532 JST,,,19,,5e4fdc3d.13,1,,2020-02-21 22:33:49 JST,1/0,0,WARNING,57P02,"terminating connection because of crash of another server process","The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.","In a moment you should be able to reconnect to the database and repeat your command.",,,,,,,""
2020-02-21 23:22:00.542 JST,,,13,,5e4fdc2f.d,5,,2020-02-21 22:33:35 JST,,0,LOG,00000,"all server processes terminated; reinitializing",,,,,,,,,""
2020-02-21 23:22:00.594 JST,,,11071,,5e4fe788.2b3f,1,,2020-02-21 23:22:00 JST,,0,LOG,00000,"database system was interrupted; last known up at 2020-02-21 23:21:03 JST",,,,,,,,,""
2020-02-21 23:22:00.752 JST,,,11071,,5e4fe788.2b3f,2,,2020-02-21 23:22:00 JST,,0,LOG,58030,"could not fsync file ""./base/16385/16400"": Input/output error",,,,,,,,,""
2020-02-21 23:22:00.759 JST,,,11071,,5e4fe788.2b3f,3,,2020-02-21 23:22:00 JST,,0,LOG,00000,"database system was not properly shut down; automatic recovery in progress",,,,,,,,,""
2020-02-21 23:22:00.761 JST,,,11071,,5e4fe788.2b3f,4,,2020-02-21 23:22:00 JST,,0,LOG,00000,"redo starts at 0/C303AF88",,,,,,,,,""
2020-02-21 23:22:38.996 JST,,,11071,,5e4fe788.2b3f,5,,2020-02-21 23:22:00 JST,,0,LOG,00000,"redo done at 0/E8FB78F8",,,,,,,,,""
2020-02-21 23:22:38.996 JST,,,11071,,5e4fe788.2b3f,6,,2020-02-21 23:22:00 JST,,0,LOG,00000,"last completed transaction was at log time 2020-02-21 14:21:55.578238+00",,,,,,,,,""
2020-02-21 23:22:39.455 JST,,,11071,,5e4fe788.2b3f,7,,2020-02-21 23:22:00 JST,,0,PANIC,58030,"could not fsync file ""base/16385/16400"": Input/output error",,,,,,,,,""
2020-02-21 23:22:40.666 JST,,,13,,5e4fdc2f.d,6,,2020-02-21 22:33:35 JST,,0,LOG,00000,"startup process (PID 11071) was terminated by signal 6: Aborted",,,,,,,,,""
2020-02-21 23:22:40.666 JST,,,13,,5e4fdc2f.d,7,,2020-02-21 22:33:35 JST,,0,LOG,00000,"aborting startup due to startup process failure",,,,,,,,,""
2020-02-21 23:22:40.695 JST,,,13,,5e4fdc2f.d,8,,2020-02-21 22:33:35 JST,,0,LOG,00000,"database system is shut down",,,,,,,,,""
Информация о моей среде
ОС хоста:
- Red Hat Enterprise Linux Сервер 7,6 (Maipo)
- Docker версия 19.03.2, сборка 6a30df c
- Память 8 ГБ
Docker Контейнер:
- Ubuntu 16.04.6 LTS
- psql (PostgreSQL) 10.7 (Ubuntu 10.7-1.pgdg16.04 + 1)
- python 3,6