Skip to content

LogPipeline ​

The logpipeline.telemetry.kyma-project.io CustomResourceDefinition (CRD) is a detailed description of the kind of data and the format used to filter and ship application logs in Kyma. To get the current CRD and show the output in the YAML format, run this command:

bash
kubectl get crd logpipeline.telemetry.kyma-project.io -o yaml

Sample Custom Resource ​

The following LogPipeline object defines a pipeline integrating with the otlp output. It uses mTLS taking connection details from a Secret, excludes OTLP logs from "namespaceA" and includes application logs emitted in "namespaceB".

yaml
apiVersion: telemetry.kyma-project.io/v1alpha1
kind: LogPipeline
metadata:
  name: backend
spec:
  otlp:
    namespaces:
      exclude:
      - namespaceA
  input:
    application:
      namespaces:
        include:
        - namespaceB
      enabled: true
      keepOriginalBody: true
  output:
    otlp:
      protocol: grpc
      endpoint:
        valueFrom:
          secretKeyRef:
            key: ingest-otlp-endpoint
            name: mySecret
            namespace: default
      tls:
        cert:
          valueFrom:
            secretKeyRef:
              key: ingest-otlp-cert
              name: mySecret
              namespace: default
        key:
          valueFrom:
            secretKeyRef:
              key: ingest-otlp-key
              name: mySecret
              namespace: default
status:
  conditions:
  - lastTransitionTime: "2025-06-13T08:58:38Z"
    message: Log gateway Deployment is ready
    observedGeneration: 4
    reason: GatewayReady
    status: "True"
    type: GatewayHealthy
  - lastTransitionTime: "2025-03-30T20:32:39Z"
    message: LogPipeline specification is successfully applied to the configuration
      of Log gateway
    observedGeneration: 4
    reason: GatewayConfigured
    status: "True"
    type: ConfigurationGenerated
  - lastTransitionTime: "2025-03-30T20:32:39Z"
    message: Log agent DaemonSet is ready
    observedGeneration: 4
    reason: AgentReady
    status: "True"
    type: AgentHealthy
  - lastTransitionTime: "2025-07-07T05:23:03Z"
    message: No problems detected in the telemetry flow
    observedGeneration: 4
    reason: FlowHealthy
    status: "True"
    type: TelemetryFlowHealthy

For further examples, see the samples directory.

Custom Resource Parameters ​

For details, see the LogPipeline specification file.

LogPipeline.telemetry.kyma-project.io/v1alpha1 ​

Spec:

ParameterTypeDescription
files[]objectFiles is a list of content snippets that are mounted as files in the Fluent Bit configuration, which can be linked in the custom filters and a custom output. Only available when using an output of type http and custom.
files.​content (required)stringContent of the file to be mounted in the Fluent Bit configuration.
files.​name (required)stringName of the file under which the content is mounted in the Fluent Bit configuration.
filters[]objectFluentBitFilters configures custom Fluent Bit filters to transform logs. Only available when using an output of type http and custom.
filters.​customstringCustom defines a custom filter in the Fluent Bit syntax. If you use a custom filter, you put the LogPipeline in unsupported mode. Only available when using an output of type http and custom.
inputobjectInput configures additional inputs for log collection.
input.​applicationobjectApplication input configures the log collection from application containers stdout/stderr by tailing the log files of the underlying container runtime.
input.​application.​containersobjectContainers describes whether application logs from specific containers are selected. The options are mutually exclusive.
input.​application.​containers.​exclude[]stringExclude specifies to exclude only the container logs with the specified container names.
input.​application.​containers.​include[]stringInclude specifies to include only the container logs with the specified container names.
input.​application.​dropLabelsbooleanDropLabels defines whether to drop all Kubernetes labels. The default is false. Only available when using an output of type http and custom. For an otlp output, use the label enrichement feature in the Telemetry resource instead.
input.​application.​enabledbooleanEnabled specifies if the 'application' input is enabled. If enabled, logs are collected from application containers stdout/stderr. The default is true.
input.​application.​keepAnnotationsbooleanKeepAnnotations defines whether to keep all Kubernetes annotations. The default is false. Only available when using an output of type http and custom.
input.​application.​keepOriginalBodybooleanKeepOriginalBody retains the original log data if the log data is in JSON and it is successfully parsed. If set to false, the original log data is removed from the log record. The default is true.
input.​application.​namespacesobjectNamespaces describes whether application logs from specific namespaces are selected. The options are mutually exclusive. System namespaces are excluded by default. Use the system attribute with value true to enable them.
input.​application.​namespaces.​exclude[]stringExclude specifies the list of namespace names to exclude when collecting container logs. By default, logs from all namespaces are collected, except system namespaces. You cannot specify an exclude list together with an include list.
input.​application.​namespaces.​include[]stringInclude specifies the list of namespace names to include when collecting container logs. By default, logs from all namespaces are collected, except system namespaces. You cannot specify an include list together with an exclude list.
input.​application.​namespaces.​systembooleanSystem specifies whether to collect logs from system namespaces. If set to true, you collect logs from all namespaces including system namespaces, such as like kube-system, istio-system, and kyma-system. The default is false.
input.​otlpobjectOTLP input configures the push endpoint to receive logs from a OTLP source.
input.​otlp.​disabledbooleanDisabled specifies if the 'otlp' input is deactivated. If set to true, no push-based OTLP signals are collected. The default is false.
input.​otlp.​namespacesobjectNamespaces describes whether push-based OTLP signals from specific namespaces are selected. System namespaces are enabled by default.
input.​otlp.​namespaces.​exclude[]stringExclude telemetry data from the specified namespace names only. By default, all namespaces (depending on input type: except system namespaces) are collected. You cannot specify an exclude list together with an include list.
input.​otlp.​namespaces.​include[]stringInclude telemetry data from the specified namespace names only. By default, all namespaces (depending on input type: except system namespaces) are included. You cannot specify an include list together with an exclude list.
output (required)objectOutput configures the backend to which logs are sent. You must specify exactly one output per pipeline.
output.​customstringCustom defines a custom output in the Fluent Bit syntax where you want to push the logs. If you use a custom output, you put the LogPipeline in unsupported mode. Only available when using an output of type http and custom.
output.​httpobjectHTTP configures an HTTP-based output compatible with the Fluent Bit HTTP output plugin.
output.​http.​compressstringCompress defines the compression algorithm to use. Either none or gzip. Default is none.
output.​http.​dedotbooleanDedot enables de-dotting of Kubernetes labels and annotations. For compatibility with OpenSearch-based backends, dots (.) are replaced by underscores (_). Default is false.
output.​http.​formatstringFormat is the data format to be used in the HTTP request body. Either gelf, json, json_stream, json_lines, or msgpack. Default is json.
output.​http.​host (required)objectHost defines the host of the HTTP backend.
output.​http.​host.​valuestringValue as plain text.
output.​http.​host.​valueFromobjectValueFrom is the value as a reference to a resource.
output.​http.​host.​valueFrom.​secretKeyRef (required)objectSecretKeyRef refers to the value of a specific key in a Secret. You must provide name and namespace of the Secret, as well as the name of the key.
output.​http.​host.​valueFrom.​secretKeyRef.​key (required)stringKey defines the name of the attribute of the Secret holding the referenced value.
output.​http.​host.​valueFrom.​secretKeyRef.​name (required)stringName of the Secret containing the referenced value.
output.​http.​host.​valueFrom.​secretKeyRef.​namespace (required)stringNamespace containing the Secret with the referenced value.
output.​http.​passwordobjectPassword defines the basic auth password.
output.​http.​password.​valuestringValue as plain text.
output.​http.​password.​valueFromobjectValueFrom is the value as a reference to a resource.
output.​http.​password.​valueFrom.​secretKeyRef (required)objectSecretKeyRef refers to the value of a specific key in a Secret. You must provide name and namespace of the Secret, as well as the name of the key.
output.​http.​password.​valueFrom.​secretKeyRef.​key (required)stringKey defines the name of the attribute of the Secret holding the referenced value.
output.​http.​password.​valueFrom.​secretKeyRef.​name (required)stringName of the Secret containing the referenced value.
output.​http.​password.​valueFrom.​secretKeyRef.​namespace (required)stringNamespace containing the Secret with the referenced value.
output.​http.​portstringPort defines the port of the HTTP backend. Default is 443.
output.​http.​tlsobjectTLS configures TLS for the HTTP backend.
output.​http.​tls.​caobjectCA defines an optional CA certificate for server certificate verification when using TLS. The certificate must be provided in PEM format.
output.​http.​tls.​ca.​valuestringValue as plain text.
output.​http.​tls.​ca.​valueFromobjectValueFrom is the value as a reference to a resource.
output.​http.​tls.​ca.​valueFrom.​secretKeyRef (required)objectSecretKeyRef refers to the value of a specific key in a Secret. You must provide name and namespace of the Secret, as well as the name of the key.
output.​http.​tls.​ca.​valueFrom.​secretKeyRef.​key (required)stringKey defines the name of the attribute of the Secret holding the referenced value.
output.​http.​tls.​ca.​valueFrom.​secretKeyRef.​name (required)stringName of the Secret containing the referenced value.
output.​http.​tls.​ca.​valueFrom.​secretKeyRef.​namespace (required)stringNamespace containing the Secret with the referenced value.
output.​http.​tls.​certobjectCert defines a client certificate to use when using TLS. The certificate must be provided in PEM format.
output.​http.​tls.​cert.​valuestringValue as plain text.
output.​http.​tls.​cert.​valueFromobjectValueFrom is the value as a reference to a resource.
output.​http.​tls.​cert.​valueFrom.​secretKeyRef (required)objectSecretKeyRef refers to the value of a specific key in a Secret. You must provide name and namespace of the Secret, as well as the name of the key.
output.​http.​tls.​cert.​valueFrom.​secretKeyRef.​key (required)stringKey defines the name of the attribute of the Secret holding the referenced value.
output.​http.​tls.​cert.​valueFrom.​secretKeyRef.​name (required)stringName of the Secret containing the referenced value.
output.​http.​tls.​cert.​valueFrom.​secretKeyRef.​namespace (required)stringNamespace containing the Secret with the referenced value.
output.​http.​tls.​disabledbooleanDisabled specifies if TLS is disabled or enabled. Default is false.
output.​http.​tls.​keyobjectKey defines the client key to use when using TLS. The key must be provided in PEM format.
output.​http.​tls.​key.​valuestringValue as plain text.
output.​http.​tls.​key.​valueFromobjectValueFrom is the value as a reference to a resource.
output.​http.​tls.​key.​valueFrom.​secretKeyRef (required)objectSecretKeyRef refers to the value of a specific key in a Secret. You must provide name and namespace of the Secret, as well as the name of the key.
output.​http.​tls.​key.​valueFrom.​secretKeyRef.​key (required)stringKey defines the name of the attribute of the Secret holding the referenced value.
output.​http.​tls.​key.​valueFrom.​secretKeyRef.​name (required)stringName of the Secret containing the referenced value.
output.​http.​tls.​key.​valueFrom.​secretKeyRef.​namespace (required)stringNamespace containing the Secret with the referenced value.
output.​http.​tls.​skipCertificateValidationbooleanIf true, the validation of certificates is skipped. Default is false.
output.​http.​uristringURI defines the URI of the HTTP backend. Default is "/".
output.​http.​userobjectUser defines the basic auth user.
output.​http.​user.​valuestringValue as plain text.
output.​http.​user.​valueFromobjectValueFrom is the value as a reference to a resource.
output.​http.​user.​valueFrom.​secretKeyRef (required)objectSecretKeyRef refers to the value of a specific key in a Secret. You must provide name and namespace of the Secret, as well as the name of the key.
output.​http.​user.​valueFrom.​secretKeyRef.​key (required)stringKey defines the name of the attribute of the Secret holding the referenced value.
output.​http.​user.​valueFrom.​secretKeyRef.​name (required)stringName of the Secret containing the referenced value.
output.​http.​user.​valueFrom.​secretKeyRef.​namespace (required)stringNamespace containing the Secret with the referenced value.
output.​otlpobjectOTLP defines an output using the OpenTelemetry protocol.
output.​otlp.​authenticationobjectAuthentication defines authentication options for the OTLP output
output.​otlp.​authentication.​basicobjectBasic activates Basic authentication for the destination providing relevant Secrets.
output.​otlp.​authentication.​basic.​password (required)objectPassword contains the basic auth password or a Secret reference.
output.​otlp.​authentication.​basic.​password.​valuestringValue as plain text.
output.​otlp.​authentication.​basic.​password.​valueFromobjectValueFrom is the value as a reference to a resource.
output.​otlp.​authentication.​basic.​password.​valueFrom.​secretKeyRef (required)objectSecretKeyRef refers to the value of a specific key in a Secret. You must provide name and namespace of the Secret, as well as the name of the key.
output.​otlp.​authentication.​basic.​password.​valueFrom.​secretKeyRef.​key (required)stringKey defines the name of the attribute of the Secret holding the referenced value.
output.​otlp.​authentication.​basic.​password.​valueFrom.​secretKeyRef.​name (required)stringName of the Secret containing the referenced value.
output.​otlp.​authentication.​basic.​password.​valueFrom.​secretKeyRef.​namespace (required)stringNamespace containing the Secret with the referenced value.
output.​otlp.​authentication.​basic.​user (required)objectUser contains the basic auth username or a Secret reference.
output.​otlp.​authentication.​basic.​user.​valuestringValue as plain text.
output.​otlp.​authentication.​basic.​user.​valueFromobjectValueFrom is the value as a reference to a resource.
output.​otlp.​authentication.​basic.​user.​valueFrom.​secretKeyRef (required)objectSecretKeyRef refers to the value of a specific key in a Secret. You must provide name and namespace of the Secret, as well as the name of the key.
output.​otlp.​authentication.​basic.​user.​valueFrom.​secretKeyRef.​key (required)stringKey defines the name of the attribute of the Secret holding the referenced value.
output.​otlp.​authentication.​basic.​user.​valueFrom.​secretKeyRef.​name (required)stringName of the Secret containing the referenced value.
output.​otlp.​authentication.​basic.​user.​valueFrom.​secretKeyRef.​namespace (required)stringNamespace containing the Secret with the referenced value.
output.​otlp.​endpoint (required)objectEndpoint defines the host and port (<host>:<port>) of an OTLP endpoint.
output.​otlp.​endpoint.​valuestringValue as plain text.
output.​otlp.​endpoint.​valueFromobjectValueFrom is the value as a reference to a resource.
output.​otlp.​endpoint.​valueFrom.​secretKeyRef (required)objectSecretKeyRef refers to the value of a specific key in a Secret. You must provide name and namespace of the Secret, as well as the name of the key.
output.​otlp.​endpoint.​valueFrom.​secretKeyRef.​key (required)stringKey defines the name of the attribute of the Secret holding the referenced value.
output.​otlp.​endpoint.​valueFrom.​secretKeyRef.​name (required)stringName of the Secret containing the referenced value.
output.​otlp.​endpoint.​valueFrom.​secretKeyRef.​namespace (required)stringNamespace containing the Secret with the referenced value.
output.​otlp.​headers[]objectHeaders defines custom headers to be added to outgoing HTTP or gRPC requests.
output.​otlp.​headers.​name (required)stringName defines the header name.
output.​otlp.​headers.​prefixstringPrefix defines an optional header value prefix. The prefix is separated from the value by a space character.
output.​otlp.​headers.​valuestringValue as plain text.
output.​otlp.​headers.​valueFromobjectValueFrom is the value as a reference to a resource.
output.​otlp.​headers.​valueFrom.​secretKeyRef (required)objectSecretKeyRef refers to the value of a specific key in a Secret. You must provide name and namespace of the Secret, as well as the name of the key.
output.​otlp.​headers.​valueFrom.​secretKeyRef.​key (required)stringKey defines the name of the attribute of the Secret holding the referenced value.
output.​otlp.​headers.​valueFrom.​secretKeyRef.​name (required)stringName of the Secret containing the referenced value.
output.​otlp.​headers.​valueFrom.​secretKeyRef.​namespace (required)stringNamespace containing the Secret with the referenced value.
output.​otlp.​pathstringPath defines OTLP export URL path (only for the HTTP protocol). This value overrides auto-appended paths /v1/metrics and /v1/traces
output.​otlp.​protocolstringProtocol defines the OTLP protocol (http or grpc). Default is grpc.
output.​otlp.​tlsobjectTLS defines TLS options for the OTLP output.
output.​otlp.​tls.​caobjectDefines an optional CA certificate for server certificate verification when using TLS. The certificate must be provided in PEM format.
output.​otlp.​tls.​ca.​valuestringValue as plain text.
output.​otlp.​tls.​ca.​valueFromobjectValueFrom is the value as a reference to a resource.
output.​otlp.​tls.​ca.​valueFrom.​secretKeyRef (required)objectSecretKeyRef refers to the value of a specific key in a Secret. You must provide name and namespace of the Secret, as well as the name of the key.
output.​otlp.​tls.​ca.​valueFrom.​secretKeyRef.​key (required)stringKey defines the name of the attribute of the Secret holding the referenced value.
output.​otlp.​tls.​ca.​valueFrom.​secretKeyRef.​name (required)stringName of the Secret containing the referenced value.
output.​otlp.​tls.​ca.​valueFrom.​secretKeyRef.​namespace (required)stringNamespace containing the Secret with the referenced value.
output.​otlp.​tls.​certobjectDefines a client certificate to use when using TLS. The certificate must be provided in PEM format.
output.​otlp.​tls.​cert.​valuestringValue as plain text.
output.​otlp.​tls.​cert.​valueFromobjectValueFrom is the value as a reference to a resource.
output.​otlp.​tls.​cert.​valueFrom.​secretKeyRef (required)objectSecretKeyRef refers to the value of a specific key in a Secret. You must provide name and namespace of the Secret, as well as the name of the key.
output.​otlp.​tls.​cert.​valueFrom.​secretKeyRef.​key (required)stringKey defines the name of the attribute of the Secret holding the referenced value.
output.​otlp.​tls.​cert.​valueFrom.​secretKeyRef.​name (required)stringName of the Secret containing the referenced value.
output.​otlp.​tls.​cert.​valueFrom.​secretKeyRef.​namespace (required)stringNamespace containing the Secret with the referenced value.
output.​otlp.​tls.​insecurebooleanInsecure defines whether to send requests using plaintext instead of TLS.
output.​otlp.​tls.​insecureSkipVerifybooleanInsecureSkipVerify defines whether to skip server certificate verification when using TLS.
output.​otlp.​tls.​keyobjectDefines the client key to use when using TLS. The key must be provided in PEM format.
output.​otlp.​tls.​key.​valuestringValue as plain text.
output.​otlp.​tls.​key.​valueFromobjectValueFrom is the value as a reference to a resource.
output.​otlp.​tls.​key.​valueFrom.​secretKeyRef (required)objectSecretKeyRef refers to the value of a specific key in a Secret. You must provide name and namespace of the Secret, as well as the name of the key.
output.​otlp.​tls.​key.​valueFrom.​secretKeyRef.​key (required)stringKey defines the name of the attribute of the Secret holding the referenced value.
output.​otlp.​tls.​key.​valueFrom.​secretKeyRef.​name (required)stringName of the Secret containing the referenced value.
output.​otlp.​tls.​key.​valueFrom.​secretKeyRef.​namespace (required)stringNamespace containing the Secret with the referenced value.
variables[]objectVariables is a list of mappings from Kubernetes Secret keys to environment variables. Mapped keys are mounted as environment variables, so that they are available as Variables in the custom filters and a custom output. Only available when using an output of type http and custom.
variables.​name (required)stringName of the variable to map.
variables.​valueFrom (required)objectValueFrom specifies the secret and key to select the value to map.
variables.​valueFrom.​secretKeyRef (required)objectSecretKeyRef refers to the value of a specific key in a Secret. You must provide name and namespace of the Secret, as well as the name of the key.
variables.​valueFrom.​secretKeyRef.​key (required)stringKey defines the name of the attribute of the Secret holding the referenced value.
variables.​valueFrom.​secretKeyRef.​name (required)stringName of the Secret containing the referenced value.
variables.​valueFrom.​secretKeyRef.​namespace (required)stringNamespace containing the Secret with the referenced value.

Status:

ParameterTypeDescription
conditions[]objectAn array of conditions describing the status of the pipeline.
conditions.​lastTransitionTime (required)stringlastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.
conditions.​message (required)stringmessage is a human readable message indicating details about the transition. This may be an empty string.
conditions.​observedGenerationintegerobservedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance.
conditions.​reason (required)stringreason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.
conditions.​status (required)stringstatus of the condition, one of True, False, Unknown.
conditions.​type (required)stringtype of condition in CamelCase or in foo.example.com/CamelCase.
unsupportedModebooleanIs active when the LogPipeline uses a custom output or filter; see unsupported mode.

LogPipeline Status ​

The status of the LogPipeline is determined by the condition types AgentHealthy, ConfigurationGenerated, and TelemetryFlowHealthy:

Condition TypeCondition StatusCondition ReasonCondition Message
AgentHealthyTrueAgentReadyLog agent DaemonSet is ready
AgentHealthyTrueRolloutInProgressPods are being started/updated
AgentHealthyFalseAgentNotReadyNo Pods deployed
AgentHealthyFalseAgentNotReadyDaemonSet is not yet created
AgentHealthyFalseAgentNotReadyFailed to get DaemonSet
AgentHealthyFalseAgentNotReadyPod is in the pending state because container: container name is not running due to: reason. Please check the container: container name logs.
AgentHealthyFalseAgentNotReadyPod is in the failed state due to: reason
ConfigurationGeneratedTrueAgentConfiguredLogPipeline specification is successfully applied to the configuration of Fluent Bit agent
ConfigurationGeneratedTrueTLSCertificateAboutToExpireTLS (CA) certificate is about to expire, configured certificate is valid until YYYY-MM-DD
ConfigurationGeneratedFalseEndpointInvalidHTTP output host invalid: reason
ConfigurationGeneratedFalseReferencedSecretMissingOne or more referenced Secrets are missing: Secret 'my-secret' of Namespace 'my-namespace'
ConfigurationGeneratedFalseReferencedSecretMissingOne or more keys in a referenced Secret are missing: Key 'my-key' in Secret 'my-secret' of Namespace 'my-namespace'"
ConfigurationGeneratedFalseReferencedSecretMissingSecret reference is missing field/s: (field1, field2, ...)
ConfigurationGeneratedFalseTLSCertificateExpiredTLS (CA) certificate expired on YYYY-MM-DD
ConfigurationGeneratedFalseTLSConfigurationInvalidTLS configuration invalid
ConfigurationGeneratedFalseValidationFailedPipeline validation failed due to an error from the Kubernetes API server
TelemetryFlowHealthyTrueFlowHealthyNo problems detected in the telemetry flow
TelemetryFlowHealthyFalseAgentAllTelemetryDataDroppedBackend is not reachable or rejecting logs. All logs are dropped. See troubleshooting: No Logs Arrive at the Backend
TelemetryFlowHealthyFalseAgentBufferFillingUpBuffer nearing capacity. Incoming log rate exceeds export rate. See troubleshooting: Agent Buffer Filling Up
TelemetryFlowHealthyFalseAgentNoLogsDeliveredBackend is not reachable or rejecting logs. Logs are buffered and not yet dropped. See troubleshooting: No Logs Arrive at the Backend
TelemetryFlowHealthyFalseAgentSomeDataDroppedBackend is reachable, but rejecting logs. Some logs are dropped. See troubleshooting: Not All Logs Arrive at the Backend
TelemetryFlowHealthyFalseConfigurationNotGeneratedNo logs delivered to backend because LogPipeline specification is not applied to the configuration of Fluent Bit agent. Check the 'ConfigurationGenerated' condition for more details
TelemetryFlowHealthyUnknownAgentProbingFailedCould not determine the health of the telemetry flow because the self monitor probing failed