MetricPipeline

The metricpipeline.telemetry.kyma-project.io CustomResourceDefinition (CRD) is a detailed description of the kind of data and the format used to filter and ship metric data in Kyma. To get the current CRD and show the output in the YAML format, run this command:

bash

kubectl get crd metricpipeline.telemetry.kyma-project.io -o yaml

Sample Custom Resource

The following MetricPipeline object defines a pipeline that integrates into an OTLP backend:

yaml

apiVersion: telemetry.kyma-project.io/v1alpha1
kind: MetricPipeline
metadata:
  name: otlp
  generation: 1
spec:
  input:
    prometheus:
      enabled: false
    istio:
      enabled: false
    runtime:
      enabled: false
  output:
    otlp:
      endpoint:
        value: https://myBackend:4317
status:
  conditions:
  - lastTransitionTime: "2024-01-09T07:02:16Z"
    message: "Metric agent DaemonSet is ready"
    observedGeneration: 1
    reason: AgentReady
    status: "True"
    type: AgentHealthy
  - lastTransitionTime: "2024-01-08T10:40:18Z"
    message: "Metric gateway Deployment is ready"
    observedGeneration: 1
    reason: GatewayReady
    status: "True"
    type: GatewayHealthy
  - lastTransitionTime: "2023-12-28T11:27:04Z"
    message: ""
    observedGeneration: 1
    reason: ConfigurationGenerated
    status: "True"
    type: ConfigurationGenerated

For further examples, see the samples directory.

Custom Resource Parameters

For details, see the MetricPipeline specification file.

MetricPipeline.telemetry.kyma-project.io/v1alpha1

Spec:

Parameter	Type	Description
input	object	Input configures additional inputs for metric collection.
input.istio	object	Istio input configures collection of Istio metrics from applications running in the Istio service mesh.
input.istio.diagnosticMetrics	object	DiagnosticMetrics configures collection of additional diagnostic metrics. The default is `false`.
input.istio.diagnosticMetrics.enabled	boolean	If enabled, diagnostic metrics are collected. The default is `false`.
input.istio.enabled	boolean	Enabled specifies if the 'istio' input is enabled. If enabled, istio-proxy metrics are scraped from Pods that have the istio-proxy sidecar injected. The default is `false`.
input.istio.envoyMetrics	object	EnvoyMetrics enables the collection of additional Envoy metrics with prefix `envoy_`. The default is `false`.
input.istio.envoyMetrics.enabled	boolean	Enabled specifies that Envoy metrics with prefix `envoy_` are scraped additionally. The default is `false`.
input.istio.namespaces	object	Namespaces configures the namespaces for which the collection should be activated. By default, all namespaces including system namespaces are enabled.
input.istio.namespaces.exclude	[]string	Exclude telemetry data from the specified namespace names only. By default, all namespaces (depending on input type: except system namespaces) are collected. You cannot specify an exclude list together with an include list.
input.istio.namespaces.include	[]string	Include telemetry data from the specified namespace names only. By default, all namespaces (depending on input type: except system namespaces) are included. You cannot specify an include list together with an exclude list.
input.otlp	object	OTLP input configures the push endpoint to receive metrics from an OTLP source.
input.otlp.disabled	boolean	Disabled specifies if the 'otlp' input is deactivated. If set to `true`, no push-based OTLP signals are collected. The default is `false`.
input.otlp.namespaces	object	Namespaces describes whether push-based OTLP signals from specific namespaces are selected. System namespaces are enabled by default.
input.otlp.namespaces.exclude	[]string	Exclude telemetry data from the specified namespace names only. By default, all namespaces (depending on input type: except system namespaces) are collected. You cannot specify an exclude list together with an include list.
input.otlp.namespaces.include	[]string	Include telemetry data from the specified namespace names only. By default, all namespaces (depending on input type: except system namespaces) are included. You cannot specify an include list together with an exclude list.
input.prometheus	object	Prometheus input configures collection of application metrics in the pull-based Prometheus protocol using endpoint discovery based on annotations.
input.prometheus.diagnosticMetrics	object	DiagnosticMetrics configures collection of additional diagnostic metrics. The default is `false`.
input.prometheus.diagnosticMetrics.enabled	boolean	If enabled, diagnostic metrics are collected. The default is `false`.
input.prometheus.enabled	boolean	Enabled specifies if the 'prometheus' input is enabled. If enabled, Service endpoints and Pods marked with `prometheus.io/scrape=true` annotation are scraped. The default is `false`.
input.prometheus.namespaces	object	Namespaces specifies from which namespaces metrics are collected. By default, all namespaces except the system namespaces are enabled. To enable all namespaces including system namespaces, use an empty struct notation.
input.prometheus.namespaces.exclude	[]string	Exclude telemetry data from the specified namespace names only. By default, all namespaces (depending on input type: except system namespaces) are collected. You cannot specify an exclude list together with an include list.
input.prometheus.namespaces.include	[]string	Include telemetry data from the specified namespace names only. By default, all namespaces (depending on input type: except system namespaces) are included. You cannot specify an include list together with an exclude list.
input.runtime	object	Runtime input configures collection of Kubernetes runtime metrics.
input.runtime.enabled	boolean	Enabled specifies if the 'runtime' input is enabled. If enabled, runtime metrics are collected. The default is `false`.
input.runtime.namespaces	object	Namespaces specifies from which namespaces metrics are collected. By default, all namespaces except the system namespaces are enabled. To enable all namespaces including system namespaces, use an empty struct notation.
input.runtime.namespaces.exclude	[]string	Exclude telemetry data from the specified namespace names only. By default, all namespaces (depending on input type: except system namespaces) are collected. You cannot specify an exclude list together with an include list.
input.runtime.namespaces.include	[]string	Include telemetry data from the specified namespace names only. By default, all namespaces (depending on input type: except system namespaces) are included. You cannot specify an include list together with an exclude list.
input.runtime.resources	object	Resources configures the Kubernetes resource types for which metrics are collected.
input.runtime.resources.container	object	Container configures container runtime metrics collection.
input.runtime.resources.container.enabled	boolean	Enabled specifies that the runtime metrics for the resource type are collected. The default is `true`.
input.runtime.resources.daemonset	object	DaemonSet configures DaemonSet runtime metrics collection.
input.runtime.resources.daemonset.enabled	boolean	Enabled specifies that the runtime metrics for the resource type are collected. The default is `true`.
input.runtime.resources.deployment	object	Deployment configures Deployment runtime metrics collection.
input.runtime.resources.deployment.enabled	boolean	Enabled specifies that the runtime metrics for the resource type are collected. The default is `true`.
input.runtime.resources.job	object	Job configures Job runtime metrics collection.
input.runtime.resources.job.enabled	boolean	Enabled specifies that the runtime metrics for the resource type are collected. The default is `true`.
input.runtime.resources.node	object	Node configures Node runtime metrics collection.
input.runtime.resources.node.enabled	boolean	Enabled specifies that the runtime metrics for the resource type are collected. The default is `true`.
input.runtime.resources.pod	object	Pod configures Pod runtime metrics collection.
input.runtime.resources.pod.enabled	boolean	Enabled specifies that the runtime metrics for the resource type are collected. The default is `true`.
input.runtime.resources.statefulset	object	StatefulSet configures StatefulSet runtime metrics collection.
input.runtime.resources.statefulset.enabled	boolean	Enabled specifies that the runtime metrics for the resource type are collected. The default is `true`.
input.runtime.resources.volume	object	Volume configures Volume runtime metrics collection.
input.runtime.resources.volume.enabled	boolean	Enabled specifies that the runtime metrics for the resource type are collected. The default is `true`.
output (required)	object	Output configures the backend to which metrics are sent. You must specify exactly one output per pipeline.
output.otlp (required)	object	OTLP output defines an output using the OpenTelemetry protocol.
output.otlp.authentication	object	Authentication defines authentication options for the OTLP output
output.otlp.authentication.basic	object	Basic activates `Basic` authentication for the destination providing relevant Secrets.
output.otlp.authentication.basic.password (required)	object	Password contains the basic auth password or a Secret reference.
output.otlp.authentication.basic.password.value	string	Value as plain text.
output.otlp.authentication.basic.password.valueFrom	object	ValueFrom is the value as a reference to a resource.
output.otlp.authentication.basic.password.valueFrom.secretKeyRef (required)	object	SecretKeyRef refers to the value of a specific key in a Secret. You must provide `name` and `namespace` of the Secret, as well as the name of the `key`.
output.otlp.authentication.basic.password.valueFrom.secretKeyRef.key (required)	string	Key defines the name of the attribute of the Secret holding the referenced value.
output.otlp.authentication.basic.password.valueFrom.secretKeyRef.name (required)	string	Name of the Secret containing the referenced value.
output.otlp.authentication.basic.password.valueFrom.secretKeyRef.namespace (required)	string	Namespace containing the Secret with the referenced value.
output.otlp.authentication.basic.user (required)	object	User contains the basic auth username or a Secret reference.
output.otlp.authentication.basic.user.value	string	Value as plain text.
output.otlp.authentication.basic.user.valueFrom	object	ValueFrom is the value as a reference to a resource.
output.otlp.authentication.basic.user.valueFrom.secretKeyRef (required)	object	SecretKeyRef refers to the value of a specific key in a Secret. You must provide `name` and `namespace` of the Secret, as well as the name of the `key`.
output.otlp.authentication.basic.user.valueFrom.secretKeyRef.key (required)	string	Key defines the name of the attribute of the Secret holding the referenced value.
output.otlp.authentication.basic.user.valueFrom.secretKeyRef.name (required)	string	Name of the Secret containing the referenced value.
output.otlp.authentication.basic.user.valueFrom.secretKeyRef.namespace (required)	string	Namespace containing the Secret with the referenced value.
output.otlp.endpoint (required)	object	Endpoint defines the host and port (`<host>:<port>`) of an OTLP endpoint.
output.otlp.endpoint.value	string	Value as plain text.
output.otlp.endpoint.valueFrom	object	ValueFrom is the value as a reference to a resource.
output.otlp.endpoint.valueFrom.secretKeyRef (required)	object	SecretKeyRef refers to the value of a specific key in a Secret. You must provide `name` and `namespace` of the Secret, as well as the name of the `key`.
output.otlp.endpoint.valueFrom.secretKeyRef.key (required)	string	Key defines the name of the attribute of the Secret holding the referenced value.
output.otlp.endpoint.valueFrom.secretKeyRef.name (required)	string	Name of the Secret containing the referenced value.
output.otlp.endpoint.valueFrom.secretKeyRef.namespace (required)	string	Namespace containing the Secret with the referenced value.
output.otlp.headers	[]object	Headers defines custom headers to be added to outgoing HTTP or gRPC requests.
output.otlp.headers.name (required)	string	Name defines the header name.
output.otlp.headers.prefix	string	Prefix defines an optional header value prefix. The prefix is separated from the value by a space character.
output.otlp.headers.value	string	Value as plain text.
output.otlp.headers.valueFrom	object	ValueFrom is the value as a reference to a resource.
output.otlp.headers.valueFrom.secretKeyRef (required)	object	SecretKeyRef refers to the value of a specific key in a Secret. You must provide `name` and `namespace` of the Secret, as well as the name of the `key`.
output.otlp.headers.valueFrom.secretKeyRef.key (required)	string	Key defines the name of the attribute of the Secret holding the referenced value.
output.otlp.headers.valueFrom.secretKeyRef.name (required)	string	Name of the Secret containing the referenced value.
output.otlp.headers.valueFrom.secretKeyRef.namespace (required)	string	Namespace containing the Secret with the referenced value.
output.otlp.path	string	Path defines OTLP export URL path (only for the HTTP protocol). This value overrides auto-appended paths `/v1/metrics` and `/v1/traces`
output.otlp.protocol	string	Protocol defines the OTLP protocol (`http` or `grpc`). Default is `grpc`.
output.otlp.tls	object	TLS defines TLS options for the OTLP output.
output.otlp.tls.ca	object	Defines an optional CA certificate for server certificate verification when using TLS. The certificate must be provided in PEM format.
output.otlp.tls.ca.value	string	Value as plain text.
output.otlp.tls.ca.valueFrom	object	ValueFrom is the value as a reference to a resource.
output.otlp.tls.ca.valueFrom.secretKeyRef (required)	object	SecretKeyRef refers to the value of a specific key in a Secret. You must provide `name` and `namespace` of the Secret, as well as the name of the `key`.
output.otlp.tls.ca.valueFrom.secretKeyRef.key (required)	string	Key defines the name of the attribute of the Secret holding the referenced value.
output.otlp.tls.ca.valueFrom.secretKeyRef.name (required)	string	Name of the Secret containing the referenced value.
output.otlp.tls.ca.valueFrom.secretKeyRef.namespace (required)	string	Namespace containing the Secret with the referenced value.
output.otlp.tls.cert	object	Defines a client certificate to use when using TLS. The certificate must be provided in PEM format.
output.otlp.tls.cert.value	string	Value as plain text.
output.otlp.tls.cert.valueFrom	object	ValueFrom is the value as a reference to a resource.
output.otlp.tls.cert.valueFrom.secretKeyRef (required)	object	SecretKeyRef refers to the value of a specific key in a Secret. You must provide `name` and `namespace` of the Secret, as well as the name of the `key`.
output.otlp.tls.cert.valueFrom.secretKeyRef.key (required)	string	Key defines the name of the attribute of the Secret holding the referenced value.
output.otlp.tls.cert.valueFrom.secretKeyRef.name (required)	string	Name of the Secret containing the referenced value.
output.otlp.tls.cert.valueFrom.secretKeyRef.namespace (required)	string	Namespace containing the Secret with the referenced value.
output.otlp.tls.insecure	boolean	Insecure defines whether to send requests using plaintext instead of TLS.
output.otlp.tls.insecureSkipVerify	boolean	InsecureSkipVerify defines whether to skip server certificate verification when using TLS.
output.otlp.tls.key	object	Defines the client key to use when using TLS. The key must be provided in PEM format.
output.otlp.tls.key.value	string	Value as plain text.
output.otlp.tls.key.valueFrom	object	ValueFrom is the value as a reference to a resource.
output.otlp.tls.key.valueFrom.secretKeyRef (required)	object	SecretKeyRef refers to the value of a specific key in a Secret. You must provide `name` and `namespace` of the Secret, as well as the name of the `key`.
output.otlp.tls.key.valueFrom.secretKeyRef.key (required)	string	Key defines the name of the attribute of the Secret holding the referenced value.
output.otlp.tls.key.valueFrom.secretKeyRef.name (required)	string	Name of the Secret containing the referenced value.
output.otlp.tls.key.valueFrom.secretKeyRef.namespace (required)	string	Namespace containing the Secret with the referenced value.

Status:

Parameter	Type	Description
conditions	[]object	An array of conditions describing the status of the pipeline.
conditions.lastTransitionTime (required)	string	lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.
conditions.message (required)	string	message is a human readable message indicating details about the transition. This may be an empty string.
conditions.observedGeneration	integer	observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance.
conditions.reason (required)	string	reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.
conditions.status (required)	string	status of the condition, one of True, False, Unknown.
conditions.type (required)	string	type of condition in CamelCase or in foo.example.com/CamelCase.

MetricPipeline Status

The status of the MetricPipeline is determined by the condition types GatewayHealthy, AgentHealthy, ConfigurationGenerated, and TelemetryFlowHealthy:

Condition Type	Condition Status	Condition Reason	Condition Message
GatewayHealthy	True	GatewayReady	Metric gateway Deployment is ready
GatewayHealthy	True	RolloutInProgress	Pods are being started/updated
GatewayHealthy	False	GatewayNotReady	No Pods deployed
GatewayHealthy	False	GatewayNotReady	Failed to list ReplicaSets: `reason`
GatewayHealthy	False	GatewayNotReady	Failed to fetch ReplicaSets: `reason`
GatewayHealthy	False	GatewayNotReady	Pod is not scheduled: `reason`
GatewayHealthy	False	GatewayNotReady	Pod is in the pending state because container: `container name` is not running due to: `reason`. Please check the container: `container name` logs.
GatewayHealthy	False	GatewayNotReady	Pod is in the failed state due to: `reason`
GatewayHealthy	False	GatewayNotReady	Deployment is not yet created
GatewayHealthy	False	GatewayNotReady	Failed to get Deployment
GatewayHealthy	False	GatewayNotReady	Failed to get latest ReplicaSets
AgentHealthy	True	AgentNotRequired
AgentHealthy	True	AgentReady	Metric agent DaemonSet is ready
AgentHealthy	True	RolloutInProgress	Pods are being started/updated
AgentHealthy	False	AgentNotReady	No Pods deployed
AgentHealthy	False	AgentNotReady	DaemonSet is not yet created
AgentHealthy	False	AgentNotReady	Failed to get DaemonSet
AgentHealthy	False	AgentNotReady	Pod is in the pending state because container: `container name` is not running due to: `reason`
AgentHealthy	False	AgentNotReady	Pod is in the failed state due to: `reason`
ConfigurationGenerated	True	AgentGatewayConfigured	MetricPipeline specification is successfully applied to the configuration of Metric gateway
ConfigurationGenerated	True	TLSCertificateAboutToExpire	TLS (CA) certificate is about to expire, configured certificate is valid until YYYY-MM-DD
ConfigurationGenerated	False	EndpointInvalid	OTLP output endpoint invalid: `reason`
ConfigurationGenerated	False	MaxPipelinesExceeded	Maximum pipeline count limit exceeded
ConfigurationGenerated	False	ReferencedSecretMissing	One or more referenced Secrets are missing: Secret 'my-secret' of Namespace 'my-namespace'
ConfigurationGenerated	False	ReferencedSecretMissing	One or more keys in a referenced Secret are missing: Key 'my-key' in Secret 'my-secret' of Namespace 'my-namespace'"
ConfigurationGenerated	False	ReferencedSecretMissing	Secret reference is missing field/s: (field1, field2, ...)
ConfigurationGenerated	False	TLSCertificateExpired	TLS (CA) certificate expired on YYYY-MM-DD
ConfigurationGenerated	False	TLSConfigurationInvalid	TLS configuration invalid
ConfigurationGenerated	False	ValidationFailed	Pipeline validation failed due to an error from the Kubernetes API server
TelemetryFlowHealthy	True	FlowHealthy	No problems detected in the telemetry flow
TelemetryFlowHealthy	False	GatewayAllTelemetryDataDropped	Backend is not reachable or rejecting metrics. All metrics are dropped. See troubleshooting: No Metrics Arrive at the Backend
TelemetryFlowHealthy	False	GatewayThrottling	Metric gateway is unable to receive metrics at current rate. See troubleshooting: Gateway Throttling
TelemetryFlowHealthy	False	GatewaySomeTelemetryDataDropped	Backend is reachable, but rejecting metrics. Some metrics are dropped. See troubleshooting: Not All Metrics Arrive at the Backend
TelemetryFlowHealthy	False	ConfigurationNotGenerated	No metrics delivered to backend because MetricPipeline specification is not applied to the configuration of Metric gateway. Check the 'ConfigurationGenerated' condition for more details
TelemetryFlowHealthy	Unknown	GatewayProbingFailed	Could not determine the health of the telemetry flow because the self monitor probing failed

Istio Service Mesh

Tutorials

Technical Reference

Troubleshooting

Tutorials

Expose a Workload

Use APIRule v2

Use APIRule v2alpha1

Use APIRule v1beta1

Expose and Secure a Workload

Use APIRule v2

Use APIRule v2alpha1

Use APIRule v1beta1

Security

Custom Resources

APIGateway Custom Resource

APIRule Custom Resource

v2

v2alpha1

v1beta1

APIRule Migration

Technical Reference

Troubleshooting Guides

APIRule and Service Connection Issues

APIRule v2

APIRule v2alpha1

APIRule v1beta1

External DNS Management Errors

APIRule v2 Introduction

Resources

Tutorials

Technical Reference

Runtime Agent

Tutorials

Resources

Tutorials

Register a Service

VPC Peering

Resources

Tutorials

Tutorials

Resources

Resources

Troubleshooting

Tutorials

Resources

Technical Reference

Troubleshooting Guides

Collecting Logs

Collecting Traces

Collecting Metrics

Filtering and Processing Data

Integrate with your OTLP Backend

Architecture

Integration Guides

Resources

Tutorials

Resources

Technical Reference

Tutorials

Commands

MetricPipeline ​

Sample Custom Resource ​

Custom Resource Parameters ​

MetricPipeline.telemetry.kyma-project.io/v1alpha1 ​

MetricPipeline Status ​

MetricPipeline

Sample Custom Resource

Custom Resource Parameters

MetricPipeline.telemetry.kyma-project.io/v1alpha1

MetricPipeline Status