Grafana basics on Grafana Labs

Exemplars

Sat, 04 Apr 2026 12:26:57 +0000

Introduction to exemplars

An exemplar is a specific trace representative of a repeated pattern of data in a given time interval. It helps you identify higher cardinality metadata from specific events within time series data.

Suppose your company website is experiencing a surge in traffic volumes. While more than eighty percent of the users are able to access the website in under two seconds, some users are experiencing a higher than normal response time resulting in bad user experience.

To identify the factors that are contributing to the latency, you must compare a trace for a fast response against a trace for a slow response. Given the vast amount of data in a typical production environment, it will be extremely laborious and time-consuming effort.

Use exemplars to help isolate problems within your data distribution by pinpointing query traces exhibiting high latency within a time interval. Once you localize the latency problem to a few exemplar traces, you can combine it with additional system based information or location properties to perform a root cause analysis faster, leading to quick resolutions to performance issues.

Support for exemplars is available for the Prometheus data source only. Once you enable the functionality, exemplars data is available by default. For more information on exemplar configuration and how to enable exemplars, refer to configuring exemplars in Prometheus data source.

Grafana shows exemplars alongside a metric in the Explore view and in dashboards. Each exemplar displays as a highlighted star. You can hover your cursor over an exemplar to view the unique traceID, which is a combination of a key value pair. To investigate further, click the blue button next to the traceID property.

Refer to View exemplar data for instructions on how to drill down and view exemplar trace details from metrics and logs. To know more about exemplars, refer to the blogpost Intro to exemplars, which enable Grafana Tempo’s distributed tracing at massive scale.

Intro to time series

Sat, 04 Apr 2026 12:26:57 +0000

Introduction to time series

Imagine you wanted to know how the temperature outside changes throughout the day. Once every hour, you’d check the thermometer and write down the time along with the current temperature. After a while, you’d have something like this:

Time	Value
09:00	24°C
10:00	26°C
11:00	27°C

Temperature data like this is one example of what we call a time series — a sequence of measurements, ordered in time. Every row in the table represents one individual measurement at a specific time.

Tables are useful when you want to identify individual measurements, but they make it difficult to see the big picture. A more common visualization for time series is the graph, which instead places each measurement along a time axis. Visual representations like the graph make it easier to discover patterns and features of the data that otherwise would be difficult to see.

Temperature data like the one in the example, is far from the only example of a time series. Other examples of time series are:

CPU and memory usage
Sensor data
Stock market index

While each of these examples are sequences of chronologically ordered measurements, they also share other attributes:

New data is appended at the end, at regular intervals — for example, hourly at 09:00, 10:00, 11:00, and so on.
Measurements are seldom updated after they were added — for example, yesterday’s temperature doesn’t change.

Time series are powerful. They help you understand the past by letting you analyze the state of the system at any point in time. Time series could tell you that the server crashed moments after the free disk space went down to zero.

Time series can also help you predict the future, by uncovering trends in your data. If the number of registered users has been increasing monthly by 4% for the past few months, you can predict how big your user base is going to be at the end of the year.

Some time series have patterns that repeat themselves over a known period. For example, the temperature is typically higher during the day, before it dips down at night. By identifying these periodic, or seasonal, time series, you can make confident predictions about the next period. If you know that the system load peaks every day around 18:00, you can add more machines right before.

Aggregating time series

Depending on what you’re measuring, the data can vary greatly. What if you wanted to compare periods longer than the interval between measurements? If you’d measure the temperature once every hour, you’d end up with 24 data points per day. To compare the temperature in August over the years, you’d have to combine the 31 times 24 data points into one.

Combining a collection of measurements is called aggregation. There are several ways to aggregate time series data. Here are some common ones:

Average returns the sum of all values divided by the total number of values.
Min and Max return the smallest and largest value in the collection.
Sum returns the sum of all values in the collection.
Count returns the number of values in the collection.

For example, by aggregating the data in a month, you can determine that August 2017 was, on average, warmer than the year before. Instead, to see which month had the highest temperature, you’d compare the maximum temperature for each month.

How you choose to aggregate your time series data is an important decision and depends on the story you want to tell with your data. It’s common to use different aggregations to visualize the same time series data in different ways.

Time series and monitoring

In the IT industry, time series data is often collected to monitor things like infrastructure, hardware, or application events. Machine-generated time series data is typically collected with short intervals, which allows you to react to any unexpected changes, moments after they occur. As a consequence, data accumulates at a rapid pace, making it vital to have a way to store and query data efficiently. As a result, databases optimized for time series data have seen a rise in popularity in recent years.

Time series databases

A time series database (TSDB) is a database explicitly designed for time series data. While it’s possible to use any regular database to store measurements, a TSDB comes with some useful optimizations.

Modern time series databases take advantage of the fact that measurements are only ever appended, and rarely updated or removed. For example, the timestamps for each measurement change very little over time, which results in redundant data being stored.

Look at this sequence of Unix timestamps:

1572524345, 1572524375, 1572524404, 1572524434, 1572524464

Looking at these timestamps, they all start with 1572524, leading to poor use of disk space. Instead, we could store each subsequent timestamp as the difference, or delta, from the first one:

1572524345, +30, +29, +30, +30

We could even take it a step further, by calculating the deltas of these deltas:

1572524345, +30, -1, +1, +0

If measurements are taken at regular intervals, most of these delta-of-deltas will be 0. Because of optimizations like these, TSDBs use drastically less space than other databases.

Another feature of a TSDB is the ability to filter measurements using tags. Each data point is labeled with a tag that adds context information, such as where the measurement was taken. Here’s an example of the InfluxDB data format that demonstrates how each measurement is stored.

weather,location=us-midwest temperature=82 1465839830100400200
  |    -------------------- --------------  |
  |             |             |             |
  |             |             |             |
+-----------+--------+-+---------+-+---------+
|measurement|,tag_set| |field_set| |timestamp|
+-----------+--------+-+---------+-+---------+

Here are some of the TSDBs supported by Grafana:

Collecting time series data

Now that we have a place to store our time series, how do we actually gather the measurements? To collect time series data, you’d typically install a collector on the device, machine, or instance you want to monitor. Some collectors are made with a specific database in mind, and some support different output destinations.

Here are some examples of collectors:

A collector either pushes data to a database or lets the database pull the data from it. Both methods come with their own set of pros and cons:

	Pros	Cons
Push	Easier to replicate data to multiple destinations.	The TSDB has no control over how much data gets sent.
Pull	Better control of how much data that gets ingested, and its authenticity.	Firewalls, VPNs or load balancers can make it hard to access the agents.

Since it would be inefficient to write every measurement to the database, collectors pre-aggregate the data and write to the time series database at regular intervals.

Time series dimensions

Sat, 04 Apr 2026 12:26:57 +0000

Time series dimensions

In Introduction to time series, the concept of labels, also called tags, is introduced:

Another feature of a TSDB is the ability to filter measurements using tags. Each data point is labeled with a tag that adds context information, such as where the measurement was taken.

With time series data, the data often contain more than a single series, and is a set of multiple time series. Many Grafana data sources support this type of data.

The common case is issuing a single query for a measurement with one or more additional properties as dimensions. For example, querying a temperature measurement along with a location property. In this case, multiple series are returned back from that single query and each series has unique location as a dimension.

To identify unique series within a set of time series, Grafana stores dimensions in labels.

Labels

Each time series in Grafana optionally has labels. Labels are set a of key/value pairs for identifying dimensions. Example labels could be {location=us} or {country=us,state=ma,city=boston}. Within a set of time series, the combination of its name and labels identifies each series. For example, temperature {country=us,state=ma,city=boston} could identify the series of temperature values for the city of Boston in the US.

Different sources of time series data have dimensions stored natively, or common storage patterns that allow the data to be extracted into dimensions.

Time series databases (TSDBs) usually natively support dimensionality. Prometheus also stores dimensions in labels. In TSDBs such as Graphite or OpenTSDB the term tags is used instead.

In table databases such SQL, these dimensions are generally the GROUP BY parameters of a query.

Multiple dimensions in table format

In SQL or SQL-like databases that return table responses, additional dimensions are usually represented as columns in the query response table.

Single dimension

For example, consider a query like:

SELECT BUCKET(StartTime, 1h), AVG(Temperature) AS Temp, Location FROM T
  GROUP BY BUCKET(StartTime, 1h), Location
  ORDER BY time asc

This query would return a table with three columns with data types time, number, and string respectively:

StartTime	Temp	Location
09:00	24	LGA
09:00	20	BOS
10:00	26	LGA
10:00	22	BOS

The table format is a long formatted time series, also called tall. It has repeated time stamps, and repeated values in Location. In this case, we have two time series in the set that would be identified as Temp {Location=LGA} and Temp {Location=BOS}.

Individual time series from the set are extracted by using the time typed column StartTime as the time index of the time series, the numeric typed column Temp as the series name, and the name and values of the string typed Location column to build the labels, such as Location=LGA.

Multiple dimensions

If the query is updated to select and group by more than just one string column, for example, GROUP BY BUCKET(StartTime, 1h), Location, Sensor, then an additional dimension is added:

StartTime	Temp	Location	Sensor
09:00	24	LGA	A
09:00	24.1	LGA	B
09:00	20	BOS	A
09:00	20.2	BOS	B
10:00	26	LGA	A
10:00	26.1	LGA	B
10:00	22	BOS	A
10:00	22.2	BOS	B

In this case the labels that represent the dimensions will have two keys based on the two string typed columns Location and Sensor. This data results four series: Temp {Location=LGA,Sensor=A}, Temp {Location=LGA,Sensor=B}, Temp {Location=BOS,Sensor=A}, and Temp {Location=BOS,Sensor=B}.

Note: More than one dimension is currently only supported in the Logs queries within the Azure Monitor service as of version 7.1.

Note: Multiple dimensions are not supported in a way that maps to multiple alerts in Grafana, but rather they are treated as multiple conditions to a single alert. See the documentation on creating alerts with multiple series.

Multiple values

In the case of SQL-like data sources, more than one numeric column can be selected, with or without additional string columns to be used as dimensions. For example, AVG(Temperature) AS AvgTemp, MAX(Temperature) AS MaxTemp. This, if combined with multiple dimensions, can result in a lot of series. Selecting multiple values is currently only designed to be used with visualization.

Additional technical information on tabular time series formats and how dimensions are extracted can be found in the developer documentation on data frames as time series.

Histograms and heatmaps

Sat, 04 Apr 2026 12:26:57 +0000

Introduction to histograms and heatmaps

A histogram is a graphical representation of the distribution of numerical data. It groups values into buckets (sometimes also called bins) and then counts how many values fall into each bucket.

Instead of graphing the actual values, histograms graph the buckets. Each bar represents a bucket, and the bar height represents the frequency (such as count) of values that fell into that bucket’s interval.

Histogram example

This histogram shows the value distribution of a couple of time series. You can easily see that most values land between 240-300 with a peak between 260-280.

Here is an example showing height distribution of people.

For more information about histogram visualization options, refer to Histogram.

Histograms only look at value distributions over a specific time range. The problem with histograms is that you cannot see any trends or changes in the distribution over time. This is where heatmaps become useful.

Heatmaps

A heatmap is like a histogram, but over time, where each time slice represents its own histogram. Instead of using bar height as a representation of frequency, it uses cells, and colors the cell proportional to the number of values in the bucket.

In this example, you can clearly see what values are more common and how they trend over time.

For more information about heatmap visualization options, refer to Heatmap.

Pre-bucketed data

There are a number of data sources supporting histogram over time, like Elasticsearch (by using a Histogram bucket aggregation) or Prometheus (with histogram metric type and Format as option set to Heatmap). But generally, any data source could be used as long as it meets the requirement that it either returns series with names representing bucket bounds, or that it returns series sorted by the bounds in ascending order.

Raw data vs aggregated

If you use the heatmap with regular time series data (not pre-bucketed), then it’s important to keep in mind that your data is often already aggregated by your time series backend. Most time series queries do not return raw sample data, but instead include a group by time interval or maxDataPoints limit coupled with an aggregation function (usually average).

This all depends on the time range of your query of course. But the important point is to know that the histogram bucketing that Grafana performs might be done on already aggregated and averaged data. To get more accurate heatmaps, it is better to do the bucketing during metric collection, or to store the data in Elasticsearch or any other data source which supports doing histogram bucketing on the raw data.

If you remove or lower the group by time (or raise maxDataPoints) in your query to return more data points, your heatmap will be more accurate, but this can also be very CPU and memory taxing for your browser, possibly causing hangs or crashes if the number of data points becomes unreasonably large.

Glossary

Sat, 04 Apr 2026 12:26:57 +0000

Glossary

This topic lists words and abbreviations that are commonly used in the Grafana documentation and community.

app plugin	An extension of Grafana that allows users to provide additional functionality to enhance their experience by including a set of panel and data source plugins, as well as custom pages. See also data source plugin, panel plugin, and plugin.
dashboard	A set of one or more panels, organized and arranged into one or more rows, that provide an at-a-glance view of related information.
data source	A file, database, or service providing the data. Grafana supports several data sources by default, and can be extended to support additional data sources through plugins.
data source plugin	Extends Grafana with support for additional data sources. See also data source, app plugin, panel plugin, and plugin.
exemplar	An exemplar is any data that serves as a detailed example of one of the observations aggregated into a metric. An exemplar contains the observed value together with an optional timestamp and arbitrary labels, which are typically used to reference a trace.
Explore	Explore allows a user to focus on building a query. Users can refine the query to return the expected metrics before building a dashboard. For more information, refer to the Explore topic.
export or import dashboard	Grafana includes the ability to export your dashboards to a file containing JSON. Community members sometimes share their created dashboards on the Grafana Dashboards page. Dashboards previously exported or found on this site may be imported by other users.
exporter	An exporter translates data that comes out of a data source into a format that Prometheus can digest.
Integration (Grafana Cloud)	Each Integration in Grafana Cloud uses the cloud agent to connect your data source to Grafana Cloud for visualizing. Note: Prometheus uses the word “integrations” to refer to software that exposes Prometheus metrics without needing an exporter, which is a different use of the same word we use here.
graph	A commonly-used visualization that displays data as points, lines, or bars.
mixin	A mixin is a set of Grafana dashboards and Prometheus rules and alerts, written in Jsonnet and packaged together in a bundle.
panel	Basic building block in Grafana, composed by a query and a visualization. Can be moved and resized within a dashboard.
panel plugin	Extends Grafana with additional visualization options. See also panel, plugin, app plugin, and data source plugin.
plugin	An extension of Grafana that allows users to provide additional functionality to enhance their experience. See also app plugin, data source plugin, and panel plugin.
query	Used to request data from a data source. The structure and format of the query depend on the specific data source.
time series	A series of measurements, ordered by time. Time series are stored in data sources and returned as the result of a query.
trace	An observed execution path of a request through a distributed system. For more information, refer to What is Distributed Tracing?
transformation	Transformations process the result set of a query before it’s passed on for visualization. For more information, refer to the Transformations overview topic.
visualization	A graphical representation of query results.