Collect logs with the OpenTelemetry Collector
The Grafana Cloud stack includes a logging service powered by Grafana Loki, a Prometheus-inspired log aggregation system.
Loki natively supports ingesting OpenTelemetry logs over HTTP.
For ingesting logs to Loki using the OpenTelemetry Collector, you must use the otlphttp exporter.
For more information about using OpenTelemetry with Grafana products, refer to the Grafana OpenTelemetry documentation.
Loki configuration
When logs are ingested by Loki using an OpenTelemetry protocol (OTLP) ingestion endpoint, some of the data is stored as Structured Metadata.
You must set allow_structured_metadata to true within your Loki config file. Otherwise, Loki will reject the log payload as malformed. Note that Structured Metadata is enabled by default in Loki 3.0 and later.
limits_config:
allow_structured_metadata: trueConfigure the OpenTelemetry Collector to write logs into Loki
You need to make the following changes to the OpenTelemetry Collector config to write logs to Loki on its OTLP ingestion endpoint.
exporters:
otlphttp:
endpoint: http://<loki-addr>/otlpAnd enable it in service.pipelines:
service:
pipelines:
logs:
receivers: [...]
processors: [...]
exporters: [..., otlphttp]If you want to authenticate using basic auth, we recommend the basicauth extension.
extensions:
basicauth/otlp:
client_auth:
username: username
password: password
exporters:
otlphttp:
auth:
authenticator: basicauth/otlp
endpoint: http://<loki-addr>/otlp
service:
extensions: [basicauth/otlp]
pipelines:
logs:
receivers: [...]
processors: [...]
exporters: [..., otlphttp]Format considerations
Since the OpenTelemetry protocol differs from the Loki storage model, here is how data in the OpenTelemetry format will be mapped by default to the Loki data model during ingestion, which can be changed as explained later:
Index labels: Resource attributes map well to index labels in Loki, since both usually identify the source of the logs. The default list of Resource Attributes to store as Index labels can be configured using
default_resource_attributes_as_index_labelsunder distributor’s otlp_config. By default, the following resource attributes will be stored as index labels, while the remaining attributes are stored as Structured Metadata with each log entry:cloud.availability_zone
cloud.region
container.name
deployment.environment
deployment.environment.name
k8s.cluster.name
k8s.container.name
k8s.cronjob.name
k8s.daemonset.name
k8s.deployment.name
k8s.job.name
k8s.namespace.name
k8s.pod.name
k8s.replicaset.name
k8s.statefulset.name
service.instance.id
service.name
service.namespace
Note
Because Loki has a default limit of 15 index labels, we recommend storing only select resource attributes as index labels. Although the default config selects more than 15 Resource Attributes, it should be fine since a few are mutually exclusive.
Tip
For Grafana Cloud Logs, see the current OpenTelemetry guidance.
Service name discovery: Loki requires every log stream to carry a
service_namelabel. If the incoming OTLP data includes a non-emptyservice.nameresource attribute, Loki stores it as theservice_nameindex label. Ifservice.nameis absent or empty, Loki looks through the other index labels for the first match from a configurable discovery list (by default:service,app,application,name, and others). The matched label’s value is copied toservice_name. If no label from the list is present,service_nameis set tounknown_service. You can change the discovery list withdiscover_service_namein the limits_config. For the full default list and more detail, refer to Default labels for all users.Timestamp: One of
LogRecord.TimeUnixNanoorLogRecord.ObservedTimestamp, based on which one is set. If both are not set, the ingestion timestamp will be used.LogLine:
LogRecord.Bodyholds the body of the log. However, since Loki only supports Log body in string format, we will stringify non-string values using the AsString method from the OTel collector lib.Structured Metadata: Anything not stored as an Index label is stored as Structured Metadata. This includes:
- Resource Attributes not stored as Index labels. These are replicated and stored with each log entry.
- Scope (InstrumentationScope) fields, replicated and stored with each log entry:
scope_name: the instrumentation scope name (when non-empty).scope_version: the instrumentation scope version (when non-empty).scope_dropped_attributes_count: the number of scope attributes dropped (when non-zero).- Any Scope Attributes not configured to be dropped.
- LogRecord fields (other than
Bodyand the primary timestamp):observed_timestamp: stored only when bothLogRecord.TimeUnixNanoandLogRecord.ObservedTimestampare set, since one of the two is already used as the entry timestamp.severity_number: the numeric severity (when non-zero).severity_text: the severity string such asINFOorERROR(when non-empty). Always stored as Structured Metadata. Can also be promoted to an index label withseverity_text_as_label.trace_id: the hex-encoded trace ID (when present). Use this to correlate logs with traces.span_id: the hex-encoded span ID (when present).event_name: the OTLP event name field (when non-empty).flags: the log record flags value (when non-zero).dropped_attributes_count: the number of log attributes dropped by the SDK (when non-zero).- Any Log Attributes not configured to be dropped or promoted to index labels.
The default list of resource attributes to store as labels can be configured using default_resource_attributes_as_index_labels under the
distributor’s otlp_config. You can set global limits using
limits_config.otlp_config. If you are using Grafana Cloud, contact support to configure this setting.
Caution
Because of the potential for high cardinality,
k8s.pod.nameandservice.instance.idare no longer recommended as default labels. But because removing these resource attributes from the default labels would be a breaking change for existing users, they have not yet been deprecated as default labels. If you are a new user of Grafana Loki, we recommend that you modify your Alloy or OpenTelemetry Collector configuration to convertk8s.pod.nameandservice.instance.idfrom index labels to structured metadata. For sample configurations, refer to Modify default labels.
Things to note before ingesting OpenTelemetry logs to Loki:
Dots (.) are converted to underscores (_).
Loki does not support
.or any other special characters other than_in label names. The unsupported characters are replaced with an_while converting Attributes to Index Labels or Structured Metadata. Also, please note that while writing the queries, you must use the normalized format, i.e. use_instead of special characters while querying data using OTel Attributes.For example,
service.namein OTLP would becomeservice_namein Loki.Flattening of nested Attributes
While converting Attributes in OTLP to Index labels or Structured Metadata, any nested attribute values are flattened out using
_as a separator. It is done in a similar way as to how it is done in the LogQL json parser.Stringification of non-string Attribute values
While converting Attribute values in OTLP to Index label values or Structured Metadata, any non-string values are converted to string using AsString method from the OTel collector lib.
Changing the default mapping of OTLP to Loki Format
Loki supports per tenant OTLP config which lets you change the default mapping of OTLP to Loki format for each tenant. It supports changing how Attributes and select LogRecord fields are stored. Here is what the config looks like:
# OTLP log ingestion configurations
limits_config:
otlp_config:
# Configuration for Resource Attributes to store them as index labels or
# Structured Metadata or drop them altogether
resource_attributes:
# Configure whether to ignore the default list of resource attributes set in
# 'distributor.otlp.default_resource_attributes_as_index_labels' to be
# stored as index labels and only use the given resource attributes config
[ignore_defaults: <boolean>]
[attributes_config: <list of attributes_configs>]
# Configuration for Scope Attributes to store them as Structured Metadata or
# drop them altogether
[scope_attributes: <list of attributes_configs>]
# Configuration for Log Attributes to store them as index labels or
# Structured Metadata, or to drop them altogether
[log_attributes: <list of attributes_configs>]
# When true, the severity_text field from the log record is also stored as
# an index label (it is always stored as Structured Metadata regardless of
# this setting). Not recommended unless absolutely necessary, since it can
# increase the number of streams.
[severity_text_as_label: <boolean> | default = false]
attributes_config:
# Configures action to take on matching Attributes. It allows one of
# [structured_metadata, drop] for all Attribute types. It additionally allows
# index_label action for Resource Attributes and Log Attributes
[action: <string> | default = ""]
# List of attributes to configure how to store them or drop them altogether
[attributes: <list of strings>]
# Regex to choose attributes to configure how to store them or drop them
# altogether
[regex: <Regexp>]Promoting a Log Attribute to an index label with action: index_label splits log entries into a different stream whenever the value of that attribute changes. Only do this for attributes with a small, stable set of values, since high-cardinality Log Attributes promoted to labels can create a large number of streams. For example, the following configuration promotes a log attribute named http.route to an index label:
limits_config:
otlp_config:
log_attributes:
- action: index_label
attributes:
- http.routeNote
If you are a Grafana Cloud customer, you can use the config self-serve UI to configure your OTLP config.


