Многострочный шаблон для logstash rsyslog - PullRequest
Новая
       22

Многострочный шаблон для logstash rsyslog

0 голосов
/ 01 октября 2018

Я анализирую сообщение из rsyslog (systemd) с помощью многострочного кодека, но он не работает должным образом.

Я использую такой шаблон 

pattern => "^%{SYSLOGBASE} %{TIMESTAMP_ISO8601}"

для анализа началасообщение: 

Sep 28 14:42:10 ibcore9t tarantool[148700]: 2018-09-28 14:42:10.805

Полное сообщение, подобное этому: 

Oct 01 10:27:21 ibcore9t tarantool[148700]: 2018-10-01 10:27:21.533 [148700] main/14364/http/172.25.8.18:64209 init.lua:179 E> Pipeline execution failed: Function "focus_spotforward_couponpayment_map" in pipeline "focus_spotforward_couponpayment_handle": ./ib-common/sandbox.lua:130: Trade not found
Oct 01 10:27:21 ibcore9t tarantool[148700]: stack traceback:
Oct 01 10:27:21 ibcore9t tarantool[148700]: ./ib-common/pipeline/init.lua:37: in function 'call_function'
Oct 01 10:27:21 ibcore9t tarantool[148700]: ./ib-common/pipeline/init.lua:75: in function 'call_pipeline'
Oct 01 10:27:21 ibcore9t tarantool[148700]: ./ib-common/pipeline/init.lua:111: in function 
Oct 01 10:27:21 ibcore9t tarantool[148700]: [C]: in function 'pcall'
Oct 01 10:27:21 ibcore9t tarantool[148700]: ./ib-common/pipeline/init.lua:109: in function 'call'
Oct 01 10:27:21 ibcore9t tarantool[148700]: ./ib-core/server.lua:119: in function 
Oct 01 10:27:21 ibcore9t tarantool[148700]: [C]: in function 'xpcall'
Oct 01 10:27:21 ibcore9t tarantool[148700]: ./ib-common/errors.lua:93: in function 'pcall'
Oct 01 10:27:21 ibcore9t tarantool[148700]: ./ib-core/server.lua:162: in function 
Oct 01 10:27:21 ibcore9t tarantool[148700]: [C]: in function 'xpcall'
Oct 01 10:27:21 ibcore9t tarantool[148700]: ...
Oct 01 10:27:21 ibcore9t tarantool[148700]: ./ib-common/errors.lua:93: in function 'pcall'
Oct 01 10:27:21 ibcore9t tarantool[148700]: ./ib-common/funcall.lua:41: in function 'call'
Oct 01 10:27:21 ibcore9t tarantool[148700]: ./t-connect/soapserver.lua:139: in function 'callback'
Oct 01 10:27:21 ibcore9t tarantool[148700]: ./ib-common/http.lua:201: in function 'sub'
Oct 01 10:27:21 ibcore9t tarantool[148700]: ...e/tarantool/core0/.rocks/share/tarantool/http/server.lua:610: in function 
Oct 01 10:27:21 ibcore9t tarantool[148700]: [C]: in function 'pcall'
Oct 01 10:27:21 ibcore9t tarantool[148700]: ...e/tarantool/core0/.rocks/share/tarantool/http/server.lua:687: in function 'process_client'
Oct 01 10:27:21 ibcore9t tarantool[148700]: ...e/tarantool/core0/.rocks/share/tarantool/http/server.lua:1132: in function 
Oct 01 10:27:21 ibcore9t tarantool[148700]: [C]: in function 'pcall'
Oct 01 10:27:21 ibcore9t tarantool[148700]: builtin/socket.lua:1059: in function

Но я вижу только первую строку в кибане

enter image description here

Pipeline execution failed: Function "focus_spotforward_couponpayment_map" in pipeline "focus_spotforward_couponpayment_handle": ./ib-common/sandbox.lua:130: Trade not found

Мой logstash.conf: 

input {
    udp {
    port => 5000
    type => syslog
    codec => multiline {
    pattern => "^%{SYSLOGBASE} %{TIMESTAMP_ISO8601}"
    negate => true
    what => "next"
     }
    }
}

filter {
    grok {
      match => [ "message", "%{SYSLOGBASE} %{TIMESTAMP_ISO8601:timestamp_tarantool} \[%{NUMBER:id}\] %{WORD}/%{NUMBER}/%{WORD:method} %{WORD:log_level}> %{GREEDYDATA:msg}",
                 "message", "%{SYSLOGBASE} %{TIMESTAMP_ISO8601:timestamp_tarantool} \[%{NUMBER:id}\] %{WORD}/%{NUMBER}/%{WORD:method}/%{IP}:%{NUMBER} %{WORD:log_level}> %{GREEDYDATA:msg}",
                 "message", "%{SYSLOGBASE} %{TIMESTAMP_ISO8601:timestamp_tarantool} \[%{NUMBER:id}\] %{WORD}/%{NUMBER}/%{WORD:method}/%{IP}:%{NUMBER} %{WORD}.%{WORD}:%{NUMBER} %{WORD:log_level}> %{GREEDYDATA:msg}"
               ]
       }

     mutate {
      replace => { "message" => "%{msg}" }
      remove_field => [ "msg" ]
     }

     if "_grokparsefailure" in [tags] { drop {} }

     if [message] =~ "queue" {
      mutate {
       add_field => { "repair_queue" => "%{message}"}
        }
       }

}

output {
    elasticsearch {
    action => "index"
    index => "logstash-%{+YYYY.MM.dd}"
    hosts  => "ibmetrics2:9200"
 }
}

Пожалуйста, помогите

...