Loading

Elasticsearch ingest_pipeline metricset

Stack Beta

Warning

This functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.

This is the ingest_pipeline metricset of the module elasticsearch.

Collects metrics on ingest pipeline executions, with processor-level granularity.

Processor-level metrics sampling

Processor-level metrics can produce a high volume of data, so the default behavior is to collect those metrics less frequently than the period for pipeline-level metrics, by applying a sampling strategy. By default, the processor-level metrics will be collected during 25% of the time. This can be configured with the ingest.processor_sample_rate setting:

Configuration example

- module: elasticsearch
  period: 10s
  metricsets:
    - ingest_pipeline
  ingest.processor_sample_rate: 0.1
  1. decrease to 10% of fetches

This is a default metricset. If the host module is unconfigured, this metricset is enabled by default.

Fields

For a description of each field in the metricset, see the exported fields section.

Here is an example document generated by this metricset:

{
  "@timestamp": "2023-01-19T19:08:28.770Z",
  "elasticsearch": {
    "cluster": {
      "id": "WocBBA0QRma0sGpdQ7vLfQ",
      "name": "my-cluster"
    },
    "node": {
      "name": "27fb1c2fd783",
      "roles": [
        "ingest",
        "remote_cluster_client"
      ],
      "id": "f5i3v9hMT_q__q6B9WOo5A"
    },
    "ingest_pipeline": {
      "name": "my-pipeline",
      "total": {
        "count": 64,
        "failed": 0,
        "time": {
          "total": {
            "ms": 39
          },
          "self": {
            "ms": 39
          }
        }
      }
    }
  },
  "ecs": {
    "version": "8.0.0"
  },
  "host": {
    "name": "myhost.local"
  },
  "agent": {
    "type": "metricbeat",
    "version": "8.7.0",
    "ephemeral_id": "80fe51b6-57df-46d1-9240-20424df6d675",
    "id": "faab8154-7ceb-470f-b2ca-0aee636951c3",
    "name": "myhost.local"
  },
  "event": {
    "dataset": "elasticsearch.ingest_pipeline",
    "module": "elasticsearch",
    "duration": 558169708
  },
  "metricset": {
    "period": 10000,
    "name": "ingest_pipeline"
  },
  "service": {
    "address": "https://localhost:9200",
    "type": "elasticsearch"
  }
}