Мой rsyslog, работающий с модулем imfile, вышел из строя через некоторое время, через несколько минут или спустя более часа.Я попытался без каких-либо сбоев повернуть отслеживаемые журналы или попытаться повернуть файл журнала, который записывал rsyslogd.Что может быть ядром проблемы?
Внизу выводится после сбоя «rsyslogd -dn».
Первый пример:
5581.597888316:imfile.c : strm 0x7f9e2dddcc40: file 8(/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log) closing
5581.597904637:imfile.c : file '/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log' opened as #8 with mode 384
5581.597910566:imfile.c : strm 0x7f9e2dddcc40: opened file '/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log' for READ as 8
5581.597915562:imfile.c : strm 0x7f9e2dddcc40: file 8 read 4096 bytes
5581.597962407:imfile.c : strm 0x7f9e2dddcc40: file 8 read 2494 bytes
5581.597991448:imfile.c : strm 0x7f9e2dddcc40: file 8 read 0 bytes
5581.598003135:imfile.c : stream checking for file change on '/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log', inode 5388691/5388691
5581.598009776:imfile.c : main Q: qqueueAdd: entry added, size now log 1, phys 1 entries
5581.598015138:imfile.c : main Q: qqueueAdd: entry added, size now log 2, phys 2 entries
5581.598017779:imfile.c : main Q: qqueueAdd: entry added, size now log 3, phys 3 entries
5581.598022251:imfile.c : main Q: MultiEnqObj advised worker start
5581.598025367:imfile.c : imfile: DELETING listener data for 'server.log' - '/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log'
5581.598028487:imfile.c : imfile: lstnDel called for /u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log
5581.598031821:imfile.c : imfile: persisting state for '/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log' to file 'imfile-state:-u01-app-glassfish4_cluster-glassfish-domains-fss_domain-logs-server.log'
5581.598038342:imfile.c : file stream imfile-state:-u01-app-glassfish4_cluster-glassfish-domains-fss_domain-logs-server.log params: flush interval 0, async write 0
5581.598041571:imfile.c : strm 0x7f9e2dddcc40: file 8(/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log) flush, buflen 2494
5581.598055825:imfile.c : strm 0x7f9e24001770: file -1(imfile-state:-u01-app-glassfish4_cluster-glassfish-domains-fss_domain-logs-server.log) flush, buflen 4096
5581.598058669:imfile.c : strmPhysWrite, stream 0x7f9e24001770, len 4096
5581.598137252:imfile.c : file '/tmp/rsyslog/work/imfile-state:-u01-app-glassfish4_cluster-glassfish-domains-fss_domain-logs-server.log' opened as #12 with mode 384
5581.598142458:imfile.c : strm 0x7f9e24001770: opened file '/tmp/rsyslog/work/imfile-state:-u01-app-glassfish4_cluster-glassfish-domains-fss_domain-logs-server.log' for WRITE as 12
5581.598158810:imfile.c : strm 0x7f9e24001770: file 12 write wrote 4096 bytes
5581.598163709:imfile.c : strm 0x7f9e24001770: max file size 0 reached for 12, now 4096 - starting new file
5581.598166311:imfile.c : strm 0x7f9e24001770: file 12(imfile-state:-u01-app-glassfish4_cluster-glassfish-domains-fss_domain-logs-server.log) closing
5581.598168967:imfile.c : strm 0x7f9e24001770: file 12(imfile-state:-u01-app-glassfish4_cluster-glassfish-domains-fss_domain-logs-server.log) flush, buflen 2101
5581.598171869:imfile.c : strmPhysWrite, stream 0x7f9e24001770, len 2101
5581.598180848:imfile.c : strm 0x7f9e24001770: file 12 write wrote 2101 bytes
Floating point exception
Второй пример:
7854.095095708:imfile.c : watch was MODIFID
7854.095105246:imfile.c : strm 0x7f2aec893270: file 7 read 3904 bytes
7854.095146217:imfile.c : strm 0x7f2aec893270: file 7 read 0 bytes
7854.095152316:imfile.c : stream checking for file change on '/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log', inode 5388309/5388309
7854.095155839:imfile.c : main Q: qqueueAdd: entry added, size now log 4, phys 7 entries
7854.095158396:imfile.c : main Q: MultiEnqObj advised worker start
7854.095164464:main Q:Reg/w0 : 7F 7F 7F 7F 7F 7F X X 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F
7854.095280434:imfile.c : watch IN_MOVED_FROM
7854.095285340:imfile.c : imfile: fileTableSearchNoWildcard file 'server.log' - 'server.log', found:0
7854.095287791:imfile.c : imfile: IN_MOVED_FROM Event (ftIdx=0, name=server.log)
7854.095302483:imfile.c : imfile: inotify_rm_watch successfully removed file from watch (ftIdx=0, wd=2, name=server.log)
7854.095306422:imfile.c : imfile: remove listener '/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log', dirIdx 0
7854.095315612:imfile.c : strm 0x7f2aec893270: file 7 read 0 bytes
7854.095321293:imfile.c : stream checking for file change on '/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log', inode 5388309/5388704
7854.095323900:imfile.c : we had a file change on '/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log'
7854.095336181:imfile.c : strm 0x7f2aec893270: file 7(/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log) closing
7854.095348406:imfile.c : file '/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log' opened as #7 with mode 384
7854.095353956:imfile.c : strm 0x7f2aec893270: opened file '/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log' for READ as 7
7854.095361401:imfile.c : strm 0x7f2aec893270: file 7 read 0 bytes
7854.095365605:imfile.c : stream checking for file change on '/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log', inode 5388704/5388704
7854.095368787:imfile.c : imfile: DELETING listener data for 'server.log' - '/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log'
7854.095371982:imfile.c : imfile: lstnDel called for /u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log
7854.095375163:imfile.c : imfile: persisting state for '/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log' to file 'imfile-state:-u01-app-glassfish4_cluster-glassfish-domains-fss_domain-logs-server.log'
7854.095382028:imfile.c : file stream imfile-state:-u01-app-glassfish4_cluster-glassfish-domains-fss_domain-logs-server.log params: flush interval 0, async write 0
7854.095385518:imfile.c : strm 0x7f2aec893270: file 7(/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log) flush, buflen 3904
7854.095398134:imfile.c : strm 0x7f2aec89a6d0: file -1(imfile-state:-u01-app-glassfish4_cluster-glassfish-domains-fss_domain-logs-server.log) flush, buflen 4096
7854.095401486:imfile.c : strmPhysWrite, stream 0x7f2aec89a6d0, len 4096
7854.095448904:imfile.c : file '/tmp/rsyslog/work/imfile-state:-u01-app-glassfish4_cluster-glassfish-domains-fss_domain-logs-server.log' opened as #12 with mode 384
7854.095453475:imfile.c : strm 0x7f2aec89a6d0: opened file '/tmp/rsyslog/work/imfile-state:-u01-app-glassfish4_cluster-glassfish-domains-fss_domain-logs-server.log' for WRITE as 12
7854.095475589:imfile.c : strm 0x7f2aec89a6d0: file 12 write wrote 4096 bytes
7854.095480449:imfile.c : strm 0x7f2aec89a6d0: max file size 0 reached for 12, now 4096 - starting new file
7854.095483161:imfile.c : strm 0x7f2aec89a6d0: file 12(imfile-state:-u01-app-glassfish4_cluster-glassfish-domains-fss_domain-logs-server.log) closing
7854.095485992:imfile.c : strm 0x7f2aec89a6d0: file 12(imfile-state:-u01-app-glassfish4_cluster-glassfish-domains-fss_domain-logs-server.log) flush, buflen 119
7854.095489070:imfile.c : strmPhysWrite, stream 0x7f2aec89a6d0, len 119
7854.095499354:imfile.c : strm 0x7f2aec89a6d0: file 12 write wrote 119 bytes
Floating point exception
Третий пример ("strace rsyslogd -dn"):
1978.080029439:imfile.c : stream checking for file change on '/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log', inode 5387206/5387206
1978.080036053:imfile.c : main Q: qqueueAdd: entry added, size now log 1, phys 1 entries
1978.080044909:imfile.c : main Q: MultiEnqObj advised worker start
1978.080048091:imfile.c : watch IN_MOVED_FROM
1978.080051410:imfile.c : imfile: fileTableSearchNoWildcard file 'server.log' - 'server.log', found:0
1978.080058108:imfile.c : imfile: IN_MOVED_FROM Event (ftIdx=0, name=server.log)
1978.080066602:main Q:Reg/w0 : wti 0x7fdc14a6af00: worker awoke from idle processing
1978.080080646:main Q:Reg/w0 : DeleteProcessedBatch: we deleted 0 objects and enqueued 0 objects
1978.080086871:main Q:Reg/w0 : doDeleteBatch: delete batch from store, new sizes: log 1, phys 1
1978.080090282:imfile.c : imfile: inotify_rm_watch successfully removed file from watch (ftIdx=0, wd=6, name=server.log)
1978.080094448:main Q:Reg/w0 : processBATCH: batch of 1 elements must be processed
1978.080101566:main Q:Reg/w0 : processBATCH: next msg 0: [2018-10-24T12:06:18.065+0300] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=33 _ThreadName=Thread-9] [timeMillis: 1540371978065]
1978.080109475:imfile.c : imfile: remove listener '/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log', dirIdx 0
1978.080119383:main Q:Reg/w0 : PRIFILT 'kern.*'
1978.080129680:main Q:Reg/w0 : pmask:
1978.080138231:imfile.c : strm 0x7fdc14a81170: file 7 read 0 bytes
1978.080147943:main Q:Reg/w0 : FF X X X X X
1978.080160158:imfile.c : stream checking for file change on '/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log', inode 5387206/5387206
1978.080168410:imfile.c : imfile: DELETING listener data for 'server.log' - '/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log'
1978.080174527:imfile.c : imfile: lstnDel called for /u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log
1978.080180731:imfile.c : imfile: persisting state for '/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log' to file 'imfile-state:-u01-app-glassfish4_cluster-glassfish-domains-fss_domain-logs-server.log'
1978.080189037:main Q:Reg/w0 : X X X
1978.080202374:imfile.c : file stream imfile-state:-u01-app-glassfish4_cluster-glassfish-domains-fss_domain-logs-server.log params: flush interval 0, async write 0
1978.080211869:main Q:Reg/w0 : X X X
1978.080224607:imfile.c : strm 0x7fdc14a81170: file 7(/u01/app/glassfish4_cluster/glassfish/domains/fss_domain/logs/server.log) flush, buflen 757
1978.080233997:main Q:Reg/w0 : X X X X X X X X X X X X X X
1978.080266861:imfile.c : strm 0x7fdc0c002650: file -1(imfile-state:-u01-app-glassfish4_cluster-glassfish-domains-fss_domain-logs-server.log) flush, buflen 4096
1978.080278167:imfile.c : strmPhysWrite, stream 0x7fdc0c002650, len 4096
1978.080285954:main Q:Reg/w0 :
1978.080290690:main Q:Reg/w0 : PRIFILT condition result is 0
1978.080293027:main Q:Reg/w0 : PRIFILT '*.info;mail.none;authpriv.none;cron.none'
1978.080297291:main Q:Reg/w0 : pmask: 7F 7F X 7F 7F 7F 7F 7F 7F X X 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F
1978.080327131:main Q:Reg/w0 : PRIFILT condition result is 1
1978.080329560:main Q:Reg/w0 : ACTION 1 [builtin:omfile:/var/log/messages]
1978.080334740:main Q:Reg/w0 : executing action 1
1978.080337852:main Q:Reg/w0 : action 'action 1': called, logging to builtin:omfile (susp 0/0, direct q 1)
1978.080344633:imfile.c : file '/tmp/rsyslog/work/imfile-state:-u01-app-glassfish4_cluster-glassfish-domains-fss_domain-logs-server.log' opened as #10 with mode 384
1978.080354006:main Q:Reg/w0 : action 'action 1': is transactional - executing in commit phase
1978.080364174:imfile.c : strm 0x7fdc0c002650: opened file '/tmp/rsyslog/work/imfile-state:-u01-app-glassfish4_cluster-glassfish-domains-fss_domain-logs-server.log' for WRITE as 10
1978.080373668:main Q:Reg/w0 : Action 1 transitioned to state: itx
1978.080377573:main Q:Reg/w0 : action 'action 1': set suspended state to 0
1978.080380022:main Q:Reg/w0 : PRIFILT 'authpriv.*'
1978.080384000:main Q:Reg/w0 : pmask: X X X X X X X X X X FF X X X X X X X X X X X X X X X
1978.080445223:main Q:Reg/w0 : PRIFILT condition result is 0
1978.080450125:main Q:Reg/w0 :
1978.080457360:imfile.c : strm 0x7fdc0c002650: file 10 write wrote 4096 bytes
1978.080468797:main Q:Reg/w0 : PRIFILT 'mail.*'
1978.080476591:main Q:Reg/w0 :
1978.080483195:imfile.c : strm 0x7fdc0c002650: max file size 0 reached for 10, now 4096 - starting new file
1978.080491087:imfile.c : strm 0x7fdc0c002650: file 10(imfile-state:-u01-app-glassfish4_cluster-glassfish-domains-fss_domain-logs-server.log) closing
1978.080497045:imfile.c : strm 0x7fdc0c002650: file 10(imfile-state:-u01-app-glassfish4_cluster-glassfish-domains-fss_domain-logs-server.log) flush, buflen 1068
1978.080502885:imfile.c : strmPhysWrite, stream 0x7fdc0c002650, len 1068
1978.080510152:main Q:Reg/w0 : pmask: X X FF X X X X
1978.080525921:imfile.c : strm 0x7fdc0c002650: file 10 write wrote 1068 bytes
1978.080535192:main Q:Reg/w0 : X X X X X X X X X X X X X X X X X X X
1978.080555708:main Q:Reg/w0 : PRIFILT condition result is 0
1978.080557857:main Q:Reg/w0 : PRIFILT 'cron.*'
1978.080561557:main Q:Reg/w0 : pmask: X X <unfinished ...>
+++ killed by SIGFPE +++