Logsta sh Output dosent публикует журналы при использовании systemctl - PullRequest
Новая
       22

Logsta sh Output dosent публикует журналы при использовании systemctl

0 голосов
/ 09 июля 2020
  1. Если я запускаю службу logsta sh с помощью service или systemctl sudo systemctl start logstash.service, она запускается без ошибок, запускается успешно, но журналы не публикуются там в кибане.

Логста sh Журналы выполнения: 

● logstash.service - logstash
   Loaded: loaded (/etc/systemd/system/logstash.service; disabled; vendor preset: enabled)
   Active: active (running) since Thu 2020-07-09 11:51:32 UTC; 51min ago
 Main PID: 20795 (java)
    Tasks: 49 (limit: 4915)
   CGroup: /system.slice/logstash.service
           └─20795 /usr/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djruby.compile.invokedynamic=true -Djruby.jit

Jul 09 11:51:56 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:56,406][WARN ][logstash.outputs.amazonelasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_versi
Jul 09 11:51:56 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:56,431][INFO ][logstash.outputs.amazonelasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::AmazonElasticSearch", :hosts=>["https://vpc-logerror-vilj5v
Jul 09 11:51:56 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:56,457][INFO ][logstash.outputs.amazonelasticsearch][main] Using mapping template from {:path=>nil}
Jul 09 11:51:56 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:56,497][INFO ][logstash.outputs.amazonelasticsearch][main] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60002, "settings"=>{"inde
Jul 09 11:51:56 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:56,548][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pip
Jul 09 11:51:58 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:58,041][INFO ][logstash.inputs.file     ][main] No sincedb_path set, generating one based on the "path" setting {:sincedb_path=>"/var/lib/logstash/plugins/inputs/file/.sinc
Jul 09 11:51:58 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:58,086][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
Jul 09 11:51:58 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:58,244][INFO ][filewatch.observingtail  ][main][08f9a378b558a96db1555616c42d9ed88ef9b26655c2be176bc35c226b2e5572] START, creating Discoverer, Watch with file and sincedb co
Jul 09 11:51:58 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:58,248][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
Jul 09 11:51:58 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:58,718][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}

Если я использую bin/logstash -f /etc/logstash/conf.d/logstash.conf, то журналы публикуются в кибане.

Я пробовал использовать решение, предоставленное здесь https://discuss.elastic.co/t/logstash-service-doesnt-output-logs-command-line-does/90294/2 Тогда тоже нет удачи

Для справки вот мой конфиг: logsta sh .service 

[Unit]
Description=logstash
[Service]
Type=simple
User=logstash
Group=logstash
EnvironmentFile=-/etc/default/logstash
EnvironmentFile=-/etc/sysconfig/logstash
ExecStart=/usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash"
Restart=always
WorkingDirectory=/
Nice=19
LimitNOFILE=16384
TimeoutStopSec=infinity
[Install]
WantedBy=multi-user.target

logsta sh .yml 

path.data: /var/lib/logstash
pipeline.ordered: auto
path.logs: /var/log/logstash

pipeline.yml 

- pipeline.id: main
  path.config: "/etc/logstash/conf.d/*.conf"

logsta sh .conf 

input {
  file {
    path  => "/home/ubuntu/log/*"
    start_position => "beginning"
  }
}
filter {

   grok {                                                                                                   
    match => { "message" => "(?<jsonf>({.*}))"}
  }
  json {
            source => "jsonf"
  }
  mutate {
        remove_field => [ "message","jsonf" ]
      }
}
output {
  amazon_es {
    hosts => ["https://*****************.es.amazonaws.com"]
    region => "us-east-1"
    index => "errorlogs-%{+YYYY.MM.dd}"
    #user => "elastic"
    #password => "changeme"
  }
}

Любая помощь приветствуется

...