Get started with Grafana Tempo on Grafana Labs

Instrument for distributed tracing

Fri, 03 Apr 2026 12:35:46 -0500

Instrument for distributed tracing

Client instrumentation is the first building block to a functioning distributed tracing visualization pipeline. Client instrumentation is the process of adding instrumentation points in the application that create and offload spans.

Check out these resources for help instrumenting tracing with your favorite languages. Most of these guides include complete end-to-end examples with Grafana, Loki, and Tempo.

Instrumentation frameworks

Most of the popular client instrumentation frameworks have SDKs in the most commonly used programming languages. You should pick one according to your application needs.

OpenTelemetry auto-instrumentation

Some languages have support for auto-instrumentation. These libraries capture telemetry information from a client application with minimal manual instrumentation of the codebase.

OpenTelemetry

OpenTelemetry Language Specific Instrumentation

Jaeger

Jaeger Language Specific Instrumentation

Zipkin

Zipkin Language Specific Instrumentation

Grafana Blog

Community Resources

Example setups

Fri, 03 Apr 2026 12:35:46 -0500

Example setups

The following examples show various deployment and configuration options using trace generators so you can get started experimenting with Tempo without an existing application.

For more information about Tempo setup and configuration, see:

If you are interested in instrumentation, see Tempo instrumentation.

Docker Compose

The docker-compose examples are simpler and designed to show minimal configuration.

Some of the examples include:

Trace discovery with Loki
Basic Grafana Agent/OpenTelemetry Setup
Various Backends (S3/GCS/Azure)
K6 with Traces This is a great place to get started with Tempo and learn about various trace discovery flows.

Helm

The Helm example shows a complete microservice based deployment. There are monolithic mode and microservices examples.

To install Tempo on Kubernetes, use the Deploy on Kubernetes using Helm procedure.

Tanka

To view an example of a complete microservice-based deployment, this Jsonnet based example shows a complete microservice based deployment. There are monolithic mode and microservices examples.

To learn how to set up a Tempo cluster, see Deploy on Kubernetes with Tanka.

Introduction to Metrics, Logs and Traces example

The Introduction to Metrics, Logs and Traces in Grafana provides a self-contained environment for learning about Mimir, Loki, Tempo, and Grafana. It includes detailed explanations of each compononent, annotated configurations for each component.

The README.md file has full details on how to quickly download and start the environment. Additionally, you can use the ctl.sh script to alter the environment to send metrics, logs, and traces to Grafana Cloud.

Tempo in Grafana

Fri, 03 Apr 2026 12:35:46 -0500

Tempo in Grafana

Grafana has a built-in Tempo data source that can be used to query Tempo and visualize traces. This page describes the high-level features and their availability. Use the latest versions for best compatibility and stability.

Use TraceQL to dig deep into trace data

Inspired by PromQL and LogQL, TraceQL is a query language designed for selecting traces in Tempo.

The default Tempo search reviews the whole trace. TraceQL provides a method for formulating precise queries so you can quickly identify the traces and spans that you need. Query results are returned faster because the queries limit what is searched.

You can run a TraceQL query either by issuing it to Tempo’s q parameter of the search API endpoint, or, for those using Tempo in conjunction with Grafana, by using Grafana’s TraceQL query editor.

For details about how queries are constructed, read the TraceQL documentation.

View trace by ID

The most basic functionality is to visualize a trace using its ID. Select the Trace ID tab and enter the ID to view it. This functionality is enabled by default and is available in all versions of Grafana.

Log search

Traces can be discovered by searching logs for entries containing trace IDs. This is most useful when your application also logs relevant information about the trace that can also be searched, such as HTTP status code, customer ID, etc. This feature requires Grafana 7.5 or later, with a linked Loki data source, and a traceID derived field.

Find traces using Tempo tags search

Search for traces using common dimensions such as time range, duration, span tags, service names, and more. Use the trace view to quickly diagnose errors and high-latency events in your system.

Non-deterministic search

Most search functions are deterministic: using the same search criteria results in the same results.

However, Tempo search is non-deterministic. If you perform the same search twice, you’ll get different lists, assuming the possible number of results for your search is greater than the number of results you have your search set to return.

When performing a search, Tempo does a massively parallel search over the given time range, and takes the first N results. Even identical searches will differ due to things like machine load and network latency. This approach values speed over predictability and is quite simple; enforcing that the search results are consistent would introduce additional complexity (and increase the time the user spends waiting for results). TraceQL follows the same behavior.

Service graph view

Grafana provides a built-in service graph view available in Grafana Cloud and Grafana 9.1. The service graph view visualizes the span metrics (traces data for rates, error rates, and durations (RED)) and service graphs. Once the requirements are set up, this pre-configured view is immediately available in Explore > Service Graphs.

For more information, refer to the service graph view.

Metrics from spans

RED metrics can be used to drive service graphs and other ready-to-go visualizations of your span data. RED metrics represent:

Rate, the number of requests per second
Errors, the number of those requests that are failing
Duration, the amount of time those requests take

For more information about RED method, refer to The RED Method: How to instrument your services.

Note: Metrics generation is disabled by default. Contact Grafana Support to enable metrics generation in your organization.

After the metrics generator is enabled in your organization, refer to Metrics-generator configuration for information about metrics-generator options.

These metrics exist in your Hosted Metrics instance and can also be easily used to generate powerful custom dashboards.

The metrics generator automatically generates exemplars as well which allows easy metrics to trace linking. Exemplars are GA in Grafana Cloud so you can also push your own.

View JSON file

A local JSON file containing a trace can be uploaded and viewed in the Grafana UI. This is useful in cases where access to the original Tempo data source is limited, or for preserving traces outside of Tempo. The JSON data can be downloaded via the Tempo API or the Inspector panel while viewing the trace in Grafana.

Linking traces and metrics

Grafana can correlate different signals by adding the functionality to link between traces and metrics. The trace to metrics feature, a beta feature in Grafana 9.1, lets you quickly see trends or aggregated data related to each span.

You can try it out by enabling the traceToMetrics feature toggle in your Grafana configuration file. Refer to the instructions for enabling Grafana feature toggles.

For example, you can use span attributes to metric labels by using the $__tags keyword to convert span attributes to metrics labels.

For more information, refer to the trace to metric configuration documentation.

Quick start for Tempo

Fri, 03 Apr 2026 12:35:46 -0500

Quick start for Tempo

The Tempo repository provides multiple examples to help you quickly get started using Tempo and distributed tracing data.

Every example has a docker-compose.yaml manifest that includes all of the options needed to explore trace data in Grafana, including resource configuration and trace data generation. The Tempo examples running with Docker using docker-compose all include a version of Tempo and a storage configuration like Amazon S3 and GCS.

This quick start guide uses the local example running Tempo as a single binary (monolithic). Any data is stored locally in the tempo-data folder. To learn more, read the local storage example README.

Alternative: Complete MLTP example

If you would like to use a demo with Tempo, Mimir, Loki, and Grafana, then try the Introduction to Metrics, Logs, Traces, and Profiling in Grafana. Intro-to-mltp provides a self-contained environment for learning about Mimir, Loki, Tempo, and Grafana. The project includes detailed explanations of each component and annotated configurations for a single-instance deployment. Data from intro-to-mltp can also be pushed to Grafana Cloud.

Before you begin

To follow this guide, you need:

Git
Docker
The docker-compose plugin

Clone the Tempo repository and start Docker

Clone the Tempo repository:

git clone https://github.com/grafana/tempo.git

Go into the examples directory:
Bash
```
cd tempo/example/docker-compose/local
```
Start the services defined in the docker-compose file:
Bash
```
docker compose up -d
```

Verify that the services are running using docker compose ps. You should see something like:

docker compose ps
NAME                 COMMAND                  SERVICE             STATUS              PORTS
local-grafana-1      "/run.sh"                grafana             running             0.0.0.0:3000->3000/tcp
local-k6-tracing-1   "/k6-tracing run /ex…"   k6-tracing          running
local-prometheus-1   "/bin/prometheus --c…"   prometheus          running             0.0.0.0:9090->9090/tcp
local-tempo-1        "/tempo -config.file…"   tempo               running             0.0.0.0:3200->3200/tcp, 0.0.0.0:4317-4318->4317-4318/tcp, 0.0.0.0:9411->9411/tcp, 0.0.0.0:14268->14268/tcp

As part of the docker compose manifest, Grafana is now accessible on port 3000. Open a browser to localhost:3000 and sign in to Grafana.
Once logged in, navigate to the Explore page, select the Tempo data source and select the Search tab. Select Run query to list the recent traces stored in Tempo. Select one to view the trace diagram:
A couple of minutes after Tempo has started up successfully, select the Service graph tab for the Tempo data source in the Explore page. Select Run query to view a service graph, generated by Tempo’s metrics-generator.
To stop the services:
```
docker-compose down -v
```