Как получить поле во вложенном документе ES - PullRequest
Новая
       28

Как получить поле во вложенном документе ES

0 голосов
/ 16 апреля 2019

У меня есть команда curl для извлечения данных из вложенных документов ES по дате. В настоящее время это не работает.

См. Следующее для сопоставления: 

{
  "test" : {
    "mappings" : {
      "doc" : {
        "properties" : {
          "@timestamp" : {
            "type" : "date"
          },
          "@version" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "_APIName" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "_parameters" : {
            "properties" : {
              "event" : {
                "properties" : {
                  "body_json" : {
                    "properties" : {
                      "apps" : {
                        "properties" : {
                          "bundle" : {
                            "type" : "text",
                            "fields" : {
                              "keyword" : {
                                "type" : "keyword",
                                "ignore_above" : 256
                              }
                            }
                          },
                          "version" : {
                            "type" : "text",
                            "fields" : {
                              "keyword" : {
                                "type" : "keyword",
                                "ignore_above" : 256
                              }
                            }
                          }
                        }
                      },
                      "model_name" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      },
                      "serial_number" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      },
                      "version" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      }
                    }
                  }
                }
              }
            }
          },
          "_stackName" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "beat" : {
            "type" : "object"
          },
          "category" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "log" : {
            "properties" : {
              "name" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              }
            }
          },
          "log_name" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "message" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "offset" : {
            "type" : "long"
          },
          "prospector" : {
            "properties" : {
              "type" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              }
            }
          },
          "source" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "stack" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          }
        }
      }
    }
  }
}

Ниже приведен пример документа в ES. 

{
  "_index": "test",
  "_type": "doc",
  "_id": "odUvZFjNxoBJGtXhSoBA",
  "_version": 1,
  "_score": null,
  "_source": {
    "log.name": "information",
    "offset": 8106321,
    "prospector": {
      "type": "log"
    },
    "@version": "1",
    "beat": {},
    "_stackName": "test",
    "_APIName": "Information",
    "category": "lambda",
    "@timestamp": "2019-04-16T02:22:32.000Z",
    "_parameters": {
      "event": {
        "body_json": {
          "model_name": "model-01",
          "serial_number": "1234567890",
          "version": "1.2",
          "apps": [
            {
              "name": "app1",
              "version": "1.0.14"
            },
            {
              "name": "app2",
              "version": "1.0.15"
            }
          ]
        }
      }
    },
    "stack": "test"
  },
  "fields": {
    "@timestamp": [
      "2019-04-16T02:22:32.000Z"
    ]
  }
}

Это моя команда curl: 

#!/bin/bash

curl -XGET "http://localhost:9200/test*/_search?pretty" -H 'Content-Type: application/json' -d' {
  "query": {
    "bool":{
      "must":[
        {
          "range": {
            "@timestamp": {
              "gte": 1546837215000,
              "lte": 1552712415000,
              "format": "epoch_millis"
            }
          }
        }
      ]
    }
  },
  "aggs": {
    "source_bucket": {
      "nested": {
        "path": "_source._parameters.event.body_json"
      },
      "aggs": {
        "model_name": {
          "terms": {
            "script": {
              "inline": "def model = doc['_source._parameters.event.body_json.model_name'].value;\n def serial = doc['_source._parameters.event.body_json.serial_number'].value;\nreturn \"model + serial\";",
              "lang": "painless"
            }
          }
        }
      }
    }
  }
}'

На данный момент возвращает эту ошибку: 

{
  "error" : {
    "root_cause" : [
      {
        "type" : "script_exception",
        "reason" : "compile error",
        "script_stack" : [
          "def model = doc[_parameters.event.body_js ...",
          "                ^---- HERE"
        ],
        "script" : "def model = doc[_parameters.event.body_json.model_name.keyword].value;\n def serial = doc[_parameters.event.body_json.serial_number.keyword].value;\nreturn model + serial;",
        "lang" : "painless"
      }
    ],
    "type" : "search_phase_execution_exception",
    "reason" : "all shards failed",
    "phase" : "query",
    "grouped" : true,
    "failed_shards" : [
      {
        "shard" : 0,
        "index" : "test",
        "node" : "-OHA7hfMTBGqlTNwjOOngg",
        "reason" : {
          "type" : "script_exception",
          "reason" : "compile error",
          "script_stack" : [
            "def model = doc[_parameters.event.body_js ...",
            "                ^---- HERE"
          ],
          "script" : "def model = doc[_parameters.event.body_json.model_name.keyword].value;\n def serial = doc[_parameters.event.body_json.serial_number.keyword].value;\nreturn model + serial;",
          "lang" : "painless",
          "caused_by" : {
            "type" : "illegal_argument_exception",
            "reason" : "Variable [_parameters] is not defined."
          }
        }
      }
    ]
  },
  "status" : 500
}

Как я могу эффективно получить имя_модели и серийный номер, объединить их и вернуть?

1 Ответ

2 голосов
/ 16 апреля 2019

Хорошо, в вашем отображении нет вложенных полей, поэтому ваш запрос должен выглядеть следующим образом: 

#!/bin/bash

curl -XGET "http://localhost:9200/test*/_search?pretty" -H 'Content-Type: application/json' -d'{
  "query": {
    "bool": {
      "filter": [
        {
          "range": {
            "@timestamp": {
              "gte": 1546837215000,
              "lte": 1552712415000,
              "format": "epoch_millis"
            }
          }
        },
        {
          "exists": {
            "field": "_parameters.event.body_json"
          }
        }
      ]
    }
  },
  "aggs": {
    "model_name": {
      "terms": {
        "script": {
          "source": "def model = doc['_parameters.event.body_json.model_name.keyword'].value;\n def serial = doc['_parameters.event.body_json.serial_number.keyword'].value;\nreturn model + serial;",
          "lang": "painless"
        }
      }
    }
  }
}'
...