Pyroscope on Grafana Labs

Configure the Grafana Pyroscope data source

Fri, 03 Apr 2026 12:35:46 -0500

Configure the Grafana Pyroscope data source

The Pyroscope data source sets how Grafana connects to your Pyroscope database.

You can configure the data source using either the data source interface in Grafana or using a configuration file. This page explains how to set up and enable the data source capabilities using Grafana.

If you make any changes, select Save & test to preserve those changes.

If you’re using your own installation of Grafana, you can provision the Pyroscope data source using a YAML configuration file. For more information about provisioning and available configuration options, refer to Provisioning Grafana.

Before you begin

To configure a Pyroscope data source, you need administrator rights to your Grafana instance and a Pyroscope instance configured to send data to Grafana.

If you’re provisioning a Pyroscope data source, then you also need administrative rights on the server hosting your Grafana instance.

Add or modify a data source

You can use these procedures to configure a new Pyroscope data source or to edit an existing one.

Create a new data source

To configure basic settings for the data source, complete the following steps:

Select Connections in the main menu.
Enter Grafana Pyroscope in the search bar.
Select Grafana Pyroscope.
Select Add new data source in the top-right corner of the page.
On the Settings tab, complete the Name, Connection, and Authentication sections.

Use the Name field to specify the name used for the data source in panels, queries, and Explore. Toggle the Default switch for the data source to be pre-selected for new panels.
Under Connection, enter the URL of the Pyroscope instance. For example, https://example.com:4100. Refer to Connection URL for more information.
Complete the Authentication section.

Optional: Use Additional settings to configure other options.
Select Save & test.

Update an existing data source

To modify an existing Pyroscope data source:

Select Connections in the main menu.
Select Data sources to view a list of configured data sources.
Select the Pyroscope data source you wish to modify.
Optional: Use Additional settings to configure or modify other options.
After completing your updates, select Save & test.

Connection URL

The data source connection URL should point to a location of a running Pyroscope backend.

Grafana Cloud Profiles

Your Grafana Cloud instance automatically includes a fully provisioned data source.

If you are running a self-managed Grafana instance or need to configure an additional Pyroscope data source pointing to Grafana Cloud Profiles, you can find the Pyroscope URL under the Manage your stack section for your organization.

Self-managed Pyroscope backend

The connection URL for a self-managed Pyroscope backend depends on how Pyroscope is deployed. Refer to the steps under Query profiles in Grafana for more information on how to configure the data source.

If you plan to use the Profiles Drilldown application and you are running a self-managed Pyroscope backend in microservices mode, the data source connection URL should point to a gateway or proxy that routes requests to the corresponding Pyroscope service. Refer to the Helm ingress configuration for specific routing requirements.

Authentication

Use this section to select an authentication method to access the data source.

Note
Use Transport Layer Security (TLS) for an additional layer of security when working with Pyroscope. For additional information on setting up TLS encryption with Pyroscope, refer to Pyroscope configuration.

To set up authentication:

Select an authentication method from the drop-down list:
- Basic authentication: Authenticates your data source using a username and password
- Forward OAuth identity: Forwards the OAuth access token and the OIDC ID token, if available, of the user querying to the data source
- No authentication: No authentication is required to access the data source
For Basic authentication only: Enter the User and Password.
Optional: Complete the TLS settings for additional security methods.

TLS Client Authentication

Toggle on to use client authentication. When enabled, it adds the Server name, Client cert, and Client key fields. The client provides a certificate that is validated by the server to establish the client’s trusted identity. The client key encrypts the data between client and server. These details are encrypted and stored in the Grafana database.

Add self-signed certificate

Activate this option to use a self-signed TLS certificate. You can add your own Certificate Authority (CA) certificate on top of one generated by the certificate authorities for additional security measure.

Skip TLS certification validation

When activated, it bypasses TLS certificate verification. Not recommended, unless absolutely necessary for testing.
Optional: Add HTTP Headers. You can pass along additional context and metadata data about the request and response. Select Add header to add Header and Value fields.
Select Save & test to preserve your changes.

Additional settings

Use the down arrow to expand the Additional settings section to view these options.

Advanced HTTP settings

The Grafana Proxy deletes forwarded cookies. Use the Allowed cookies field to specify cookies that should be forwarded to the data source by name.

The Timeout field sets the HTTP request timeout in seconds.

Querying

Minimum step is used for queries returning time-series data. The default value is 15 seconds.

Adjusting this option can help prevent gaps when you zoom in to profiling data.

Private data source connect

Note
This feature is only available in Grafana Cloud.

Use private data source connect (PDC) to connect to and query data within a secure network without opening that network to inbound traffic from Grafana Cloud.

Refer to Private data source connect for more information on how PDC works and Configure Grafana private data source connect (PDC) for steps on setting up a PDC connection.

Use the drop-down list to select a configured private data source. If you make changes, select Test & save to preserve your changes.

Use Manage private data source connect to configure and manage any private data sources you have configured.

How profiling and tracing work together

Fri, 03 Apr 2026 12:35:46 -0500

How profiling and tracing work together

Profiles, continuous profiling, and distributed traces are all tools that can be used to improve the performance and reliability of applications. However, each tool has its own strengths and weaknesses, and it is important to choose the right tool for the job as well as understand when to use both.

Profiling

Profiling offers a deep-dive into an application’s performance at the code level, highlighting resource usage and performance hotspots.

Usage	During development, major releases, or upon noticing performance quirks.
Benefits	Business: Boosts user experience through enhanced application performance. Technical: Gives clear insights into code performance and areas of refinement.
Example	A developer uses profiling upon noting slow app performance, identifies a CPU-heavy function, and optimizes it.

Continuous profiling

Continuous profiling provides ongoing performance insights, capturing long-term trends and intermittent issues.

Usage	Mainly in production, especially for high-priority applications.
Benefits	Business: Preemptively addresses inefficiencies, potentially saving costs. Technical: Highlights performance trends and issues like potential memory leaks over time.
Example	A month-long data from continuous profiling suggests increasing memory consumption, hinting at a memory leak.

Distributed tracing

Traces requests as they cross multiple services, revealing interactions and service dependencies.

Usage	Essential for systems like microservices where requests touch multiple services.
Benefits	Business: Faster issue resolution, reduced downtimes, and strengthened customer trust. Technical: A broad view of the system's structure, revealing bottlenecks and inter-service dependencies.
Example	In e-commerce, a user's checkout request might involve various services. Tracing depicts this route, pinpointing where time is most spent.

Combined power of tracing and profiling

When used together, tracing and profiling provide a powerful tool for understanding system and application performance.

Usage	For comprehensive system-to-code insights, especially when diagnosing complex issues spread across services and codebases.
Benefits	Business: Reduces downtime, optimizes user experience, and safeguards revenues. Technical: Holistic view: Tracing pinpoints bottle-necked services, while profiling delves into the responsible code segments. End-to-end insight: Visualizes a request's full journey and the performance of individual code parts. Efficient diagnosis: Tracing identifies service latency; profiling zeroes in on its cause, be it database queries, API calls, or specific code inefficiencies.
Example	Tracing reveals latency in a payment service. Combined with profiling, it's found that a particular function, making third-party validation calls, is the culprit. This insight guides optimization, refining system efficiency.

Configure traces to profiles

Fri, 03 Apr 2026 12:35:46 -0500

Configure traces to profiles

Note
Your application must be instrumented for profiles and traces. For more information, refer to Link traces to profiles.

Using Trace to profiles, you can use Grafana’s ability to correlate different signals by adding the functionality to link between traces and profiles.

Trace to profiles lets you link your Grafana Pyroscope data source to tracing data. When configured, this connection lets you run queries from a trace span into the profile data using Explore. Each span links to your queries. Clicking a link runs the query in a split panel. If tags are configured, Grafana dynamically inserts the span attribute values into the query. The query runs over the time range of the (span start time - 60) to (span end time + 60 seconds).

Embedded flame graphs are also inserted into each span details section that has a linked profile. This lets you see resource consumption in a flame graph visualization for each span without having to navigate away from the current view.

There are two ways to configure the trace to profiles feature:

Use a basic configuration with default query, or
Configure a custom query where you can use a template language to interpolate variables from the trace or span.

Before you begin

Traces to profile requires a Tempo data source with Traces to profiles configured and a Grafana Pyroscope data source.

As with traces, your application needs to be instrumented to emit profiling data. For more information, refer to Linking tracing and profiling with span profiles.

Use a basic configuration

To use a basic configuration, follow these steps:

In the left menu, select Connections > Data sources.
Select your configured Tempo data source from the Data source list.
Scroll down to the Traces to profiles section.
Select a Pyroscope data source in the Data source drop-down.
Optional: Add one or more tags to use in the query. If left blank, the default values of service.name and service.namespace are used.

The tags you configure must be present in the spans attributes or resources for a trace-to-profiles span link to appear.

You can optionally configure a new name for the tag. This is useful if the tag has dots in the name and the target data source doesn’t allow dots in labels. In that case, you can remap service.name to service_name.
Select one or more profile types to use in the query. Select the drop-down and choose options from the menu.

The profile type or app must be selected for the query to be valid. Grafana doesn’t show any data if the profile type or app isn’t selected when a query runs.
Select Save and Test.

If you have configured a Pyroscope data source and no profile data is available or the Profiles for this span button and the embedded flame graph isn’t visible, verify that the pyroscope.profile.id key-value pair exists in your span tags.

Configure a custom query

To use a custom query with the configuration, follow these steps:

In the left menu, select Connections > Data sources.
Select a configured Tempo data source from the Data source list.
Scroll down to the Traces to profiles section.
Select a Pyroscope data source in the Data source drop-down.
Optional: Choose any tags to use in the query. If left blank, the default values of service.name and service.namespace are used.

These tags can be used in the custom query with ${__tags} variable. This variable interpolates the mapped tags as list in an appropriate syntax for the data source. Only tags present in the span are included. Tags that aren’t present are omitted.

You can also configure a name for the tag. Tag names are useful where the tag has dots in the name and the target data source doesn’t allow using dots in labels. For example, you can remap service.name to service_name. If you don’t map any tags here, you can still use any tag in the query, for example: method="${__span.tags.method}". Learn more about custom query variables.
Select one or more profile types to use in the query. Select the drop-down and choose options from the menu.
Switch on Use custom query to enter a custom query.
Specify a custom query to be used to query profile data. You can use various variables to make that query relevant for current span. The link shows only if all the variables are interpolated with non-empty values to prevent creating an invalid query. You can interpolate the configured tags using the $__tags keyword.
Select Save and Test.

Configuration options

The following table describes options for configuring your Trace to profiles settings:

Setting name	Description
Data source	Defines the target data source. You can select a Pyroscope [profiling] data source.
Tags	Defines the tags to use in the profile query. Default: `cluster`, `hostname`, `namespace`, `pod`, `service.name`, `service.namespace`. You can change the tag name for example to remove dots from the name if they’re not allowed in the target data source. For example, map `http.status` to `http_status`.
Profile type	Defines the profile type that used in the query.
Use custom query	Toggles use of custom query with interpolation.
Query	Input to write custom query. Use variable interpolation to customize it with variables from span.

Query profile data

Fri, 03 Apr 2026 12:35:46 -0500

https://grafana.com/docs/grafana-cloud/visualizations/panels-visualizations/visualizations/flame-graph/

Query profile data

You can query your profile data using the open source Grafana Profiles Drilldown app or the built-in Grafana Pyroscope data source query editor.

Profiles Drilldown provides a queryless experience for inspecting your profiling data with opinionated workflows to assist your investigation.
Data source query editor provides complete control over your data exploration and is recommended for experienced users.

In addition, you can embed flame graph panels in Grafana dashboards. Refer to the Flame graph panel documentation for details.

Profiles Drilldown

Grafana Profiles Drilldown is a native Grafana application designed to integrate seamlessly with Pyroscope, the open source continuous profiling platform, providing a smooth, queryless experience for browsing and analyzing profiling data.

You can use Profiles Drilldown in Grafana Cloud or in your own Grafana instance. For more information, refer to Access or install Profiles Drilldown.

Use cases

There are several different modes for viewing, analyzing, and comparing profiling data.

The main use cases are the following:

Proactive: Cutting costs, addressing latency issues, or optimizing memory usage for applications
Reactive: Resolving incidents with line-level accuracy or debugging active latency/memory issues

Profiles Drilldown provides an intuitive interface to specifically support proactivee and reactive use cases. You get a holistic view of all of your services and how they’re functioning, but also the ability to drill down for more targeted root cause analysis.

Profiles Drilldown offers a convenient platform to analyze profiles and get insights that are impossible to get from using other traditional signals like logs, metrics, or tracing.

Give it a try using Grafana Play

With Grafana Play, you can explore and see how it works, learning from practical examples to accelerate your development. This feature can be seen on the Grafana Play site.

Try it

Pyroscope query editor

The Pyroscope data source query editor gives you access to a profile type selector, a label selector, and collapsible options.

Like Profiles Drilldown, the query editor also provides a flame graph to visualize data.

To access the query editor:

Sign into Grafana or Grafana Cloud.
Select your Pyroscope data source.
From the menu, choose Explore.
Select a profile type from the drop-down menu.

Profile selector
Use the labels selector input to filter by labels. Pyroscope uses similar syntax to Prometheus to filter labels. Refer to Pyroscope documentation for available operators and syntax.

While the label selector can be left empty to query all profiles without filtering by labels, you must select a profile type or app for the query to be valid.

Grafana doesn’t show any data if the profile type or app isn’t selected when a query runs.
Expand the Options section to view Query Type and Group by.
Select a query type to return the profile data. Data is shown in the Flame Graph, metric data visualized in a graph, or both. You can only select both options in Explore. The panels used on dashboards allow only one visualization.

Using Group by, you can group metric data by a specified label. Without any Group by label, metric data aggregates over all the labels into single time series. You can use multiple labels to group by. Group by only effects the metric data and doesn’t change the profile data results.

In conjunction with Group by, you can set a positive number in the Limit input to limit the maximum number of time series returned by the data source. The series returned are always ordered by descending value for the total aggregated data over the time period.

Profiles query results

Profiles can be visualized in a flame graph. Refer to the Flame Graph documentation to learn about the visualization and its features.

Pyroscope returns profiles aggregated over a selected time range. The absolute values in the flame graph grow as the time range gets bigger while keeping the relative values meaningful. You can zoom in on the time range to get a higher granularity profile up to the point of a single scrape interval.

Metrics query results

Metrics results represent the aggregated sum value over time of the selected profile type.

This allows you to quickly see any spikes in the value of the scraped profiles and zoom in to a particular time range.