OpenTSDB data source on Grafana Labs

Configure the OpenTSDB data source

Fri, 03 Apr 2026 19:43:06 +0000

Configure the OpenTSDB data source

This document explains how to configure the OpenTSDB data source in Grafana.

Before you begin

Before configuring the OpenTSDB data source, ensure you have:

Grafana permissions: Organization administrator role to add data sources.
OpenTSDB instance: A running OpenTSDB server (version 2.1 or later recommended).
Network access: The Grafana server can reach the OpenTSDB HTTP API endpoint (default port 4242).
Metrics in OpenTSDB: For autocomplete to work, metrics must exist in your OpenTSDB database.

Add the data source

To add and configure the OpenTSDB data source:

Click Connections in the left-side menu.
Click Add new connection.
Type OpenTSDB in the search bar.
Select OpenTSDB.
Click Add new data source in the upper right.
Configure the data source settings as described in the following sections.

Configuration options

The following table describes the available configuration options:

Setting	Description
Name	The data source name. This is how you refer to the data source in panels and queries.
Default	Toggle to make this the default data source for new panels.
URL	The HTTP protocol, IP address, and port of your OpenTSDB server. The default port is `4242`. Example: `http://localhost:4242`.
Allowed cookies	Cookies to forward to the data source. Use this when your OpenTSDB server requires specific cookies for authentication.
Timeout	HTTP request timeout in seconds. Increase this value for slow networks or complex queries.

Auth settings

Configure authentication if your OpenTSDB server requires it:

Setting	Description
Basic auth	Enable to authenticate with a username and password. When enabled, enter the username and password in the fields that appear.
With Credentials	Enable to send cookies or auth headers with cross-site requests. Use this when OpenTSDB is on a different domain and requires credentials.
TLS Client Authentication	Enable to use client certificates for authentication. Requires configuring client certificate and key.
Skip TLS Verify	Enable to skip verification of the OpenTSDB server’s TLS certificate. Only use this in development environments.
Forward OAuth Identity	Enable to forward the user’s OAuth token to the data source. Useful when OpenTSDB is behind an OAuth-protected proxy.
Custom HTTP Headers	Add custom headers to all requests sent to OpenTSDB. Useful for API keys or custom authentication schemes.

OpenTSDB settings

Configure these settings based on your OpenTSDB server version and configuration:

Setting	Description
Version	Select your OpenTSDB version. This affects available query features. Refer to the following section for version-specific features.
Resolution	The resolution of your metric data. Select `second` for second-precision timestamps or `millisecond` for millisecond-precision timestamps.
Lookup limit	Maximum number of results returned by suggest and lookup API calls. Default is `1000`. Increase this if you have many metrics or tag values.

Version-specific features

The version setting enables different query features in Grafana:

Version	Available features
<=2.1	Basic queries with tags. Uses legacy tag-based filtering.
==2.2	Adds filter support (literal_or, wildcard, regexp, and more). Filters replace tags for more flexible queries.
==2.3	Adds explicit tags support for rate calculations and additional filter types.
==2.4	Adds fill policy support for downsampling (none, null, zero, nan). Enables `arrays=true` for alerting compatibility.

Select the version that matches your OpenTSDB server. If you’re unsure, check your OpenTSDB version with the /api/version endpoint.

Verify the connection

Click Save & test to verify that Grafana can connect to your OpenTSDB server. A successful test confirms that the URL is correct and the server is responding.

If the test fails, refer to Troubleshooting for common issues and solutions.

Provision the data source

You can define and configure the data source in YAML files as part of the Grafana provisioning system. For more information about provisioning, and for available configuration options, refer to Provisioning Grafana.

YAML example

The following example provisions an OpenTSDB data source:

apiVersion: 1

datasources:
  - name: OpenTSDB
    type: opentsdb
    access: proxy
    url: http://localhost:4242
    jsonData:
      # OpenTSDB version: 1 = <=2.1, 2 = 2.2, 3 = 2.3, 4 = 2.4
      tsdbVersion: 3
      # Resolution: 1 = second, 2 = millisecond
      tsdbResolution: 1
      # Maximum results for suggest/lookup API calls
      lookupLimit: 1000

YAML example with basic authentication

The following example provisions an OpenTSDB data source with basic authentication:

apiVersion: 1

datasources:
  - name: OpenTSDB
    type: opentsdb
    access: proxy
    url: http://localhost:4242
    basicAuth: true
    basicAuthUser: <USERNAME>
    jsonData:
      tsdbVersion: 3
      tsdbResolution: 1
      lookupLimit: 1000
    secureJsonData:
      basicAuthPassword: <PASSWORD>

YAML example with custom headers

The following example provisions an OpenTSDB data source with custom HTTP headers:

apiVersion: 1

datasources:
  - name: OpenTSDB
    type: opentsdb
    access: proxy
    url: http://localhost:4242
    jsonData:
      tsdbVersion: 3
      tsdbResolution: 1
      lookupLimit: 1000
      httpHeaderName1: X-Custom-Header
    secureJsonData:
      httpHeaderValue1: <HEADER_VALUE>

The following table describes the available fields:

Field	Type	Description
`basicAuth`	boolean	Enable basic authentication.
`basicAuthUser`	string	Username for basic authentication.
`jsonData.tsdbVersion`	number	OpenTSDB version: `1` (<=2.1), `2` (2.2), `3` (2.3), `4` (2.4).
`jsonData.tsdbResolution`	number	Timestamp resolution: `1` (second), `2` (millisecond).
`jsonData.lookupLimit`	number	Maximum results for suggest and lookup API calls. Default: `1000`.
`jsonData.httpHeaderName1`	string	Name of a custom HTTP header. Use incrementing numbers for multiple headers.
`secureJsonData.basicAuthPassword`	string	Password for basic authentication.
`secureJsonData.httpHeaderValue1`	string	Value for the custom HTTP header.

Provision with Terraform

You can provision the OpenTSDB data source using Terraform with the Grafana Terraform provider.

For more information about provisioning resources with Terraform, refer to the Grafana as code using Terraform documentation.

Terraform example

The following example provisions an OpenTSDB data source:

terraform {
  required_providers {
    grafana = {
      source  = "grafana/grafana"
      version = ">= 2.0.0"
    }
  }
}

provider "grafana" {
  url  = "<YOUR_GRAFANA_URL>"
  auth = "<YOUR_SERVICE_ACCOUNT_TOKEN>"
}

resource "grafana_data_source" "opentsdb" {
  type = "opentsdb"
  name = "OpenTSDB"
  url  = "http://localhost:4242"

  json_data_encoded = jsonencode({
    # OpenTSDB version: 1 = <=2.1, 2 = 2.2, 3 = 2.3, 4 = 2.4
    tsdbVersion = 3
    # Resolution: 1 = second, 2 = millisecond
    tsdbResolution = 1
    # Maximum results for suggest/lookup API calls
    lookupLimit = 1000
  })
}

Terraform example with basic authentication

The following example provisions an OpenTSDB data source with basic authentication:

resource "grafana_data_source" "opentsdb_auth" {
  type                = "opentsdb"
  name                = "OpenTSDB"
  url                 = "http://localhost:4242"
  basic_auth_enabled  = true
  basic_auth_username = "<USERNAME>"

  json_data_encoded = jsonencode({
    tsdbVersion    = 3
    tsdbResolution = 1
    lookupLimit    = 1000
  })

  secure_json_data_encoded = jsonencode({
    basicAuthPassword = "<PASSWORD>"
  })
}

Replace the following placeholders:

<YOUR_GRAFANA_URL>: Your Grafana instance URL (for example, https://your-org.grafana.net for Grafana Cloud)
<YOUR_SERVICE_ACCOUNT_TOKEN>: A service account token with data source permissions
<USERNAME>: The username for basic authentication
<PASSWORD>: The password for basic authentication

Next steps

Now that you’ve configured OpenTSDB, you can:

Query OpenTSDB data to build dashboards and visualizations
Use template variables to create dynamic, reusable dashboards
Add annotations to overlay events on your graphs
Set up alerting to get notified when metrics cross thresholds
Troubleshoot issues if you encounter problems

OpenTSDB query editor

Fri, 03 Apr 2026 19:43:06 +0000

OpenTSDB query editor

The query editor allows you to build OpenTSDB queries visually. The available options depend on the OpenTSDB version you configured for the data source.

Access the query editor

The OpenTSDB query editor is located on the Explore page. You can also access the OpenTSDB query editor from a dashboard panel. Click the ellipsis in the upper right of the panel and select Edit.

Create a query

To create a query:

Select the OpenTSDB data source in a panel.
Configure the query using the sections described in the following documentation.

Metric section

The Metric section contains the core query configuration:

Field	Description
Metric	The metric name to query. Start typing to see autocomplete suggestions from your OpenTSDB server.
Aggregator	The aggregation function to combine multiple time series. Default: `sum`.
Alias	Custom display name for the series. Use `$tag_<tagname>` to include tag values in the alias.

Alias patterns

The alias field supports dynamic substitution using tag values. Use the pattern $tag_<tagname> where <tagname> is the name of a tag on your metric.

Pattern	Description	Example output
`$tag_host`	Inserts the value of the `host` tag	`webserver01`
`$tag_env`	Inserts the value of the `env` tag	`production`
`$tag_host - CPU`	Combines tag value with static text	`webserver01 - CPU`
`$tag_host ($tag_env)`	Multiple tag substitutions	`webserver01 (production)`

Downsample section

Downsampling reduces the number of data points returned by aggregating values over time intervals. This improves query performance and reduces the amount of data transferred.

Field	Description
Interval	The time interval for downsampling. Leave blank to use the automatic interval based on the panel’s time range and width.
Aggregator	The aggregation function for downsampling. Default: `avg`.
Fill	(Version 2.2+) The fill policy for missing data points. Default: `none`.
Disable downsampling	Toggle to disable downsampling entirely. Use this when you need raw data points.

Interval format

The interval field accepts time duration strings:

Format	Description	Example
`s`	Seconds	`30s`
`m`	Minutes	`5m`
`h`	Hours	`1h`
`d`	Days	`1d`
`w`	Weeks	`1w`

When the interval is left blank, Grafana automatically calculates an appropriate interval based on the panel’s time range and pixel width. This ensures optimal data density for visualization.

Filters section

Filters (available in OpenTSDB 2.2+) provide advanced filtering capabilities that replace the legacy tag-based filtering.

Field	Description
Key	The tag key to filter on. Select from autocomplete suggestions or type a custom value.
Type	The filter type. Determines how the filter value is matched. Default: `iliteral_or`.
Filter	The filter value or pattern. Supports autocomplete for tag values.
Group by	Toggle to group results by this tag key. When enabled, separate time series are returned for each unique value.

Add, edit, and remove filters

To manage filters:

Click the + button next to “Filters” to add a new filter.
Configure the filter fields (Key, Type, Filter, Group by).
Click add filter to apply the filter.
To edit an existing filter, click the pencil icon next to it.
To remove a filter, click the x icon next to it.

You can add multiple filters to a single query. All filters are combined with AND logic.

Filter types

Type	Description	Example
`literal_or`	Matches exact values. Use `\|` to specify multiple values.	`web01\|web02\|web03`
`iliteral_or`	Case-insensitive literal match.	`WEB01\|web02`
`wildcard`	Matches using `*` as a wildcard character.	`web-*-prod`
`iwildcard`	Case-insensitive wildcard match.	`WEB-*`
`regexp`	Matches using regular expressions.	`web-[0-9]+`
`not_literal_or`	Excludes exact values.	`web01\|web02`
`not_iliteral_or`	Case-insensitive exclusion.	`TEST\|DEV`

Group by behavior

When Group by is enabled for a filter:

Results are split into separate time series for each unique value of the filtered tag.
Each time series is labeled with its tag value.
This is useful for comparing values across hosts, environments, or other dimensions.

When Group by is disabled:

All matching time series are combined using the selected aggregator.
A single aggregated time series is returned.

Tags section

Tags filter metrics by key-value pairs. This is the legacy filtering method for OpenTSDB versions prior to 2.2.

Field	Description
Key	The tag key to filter on. Select from autocomplete suggestions.
Value	The tag value to match. Use `*` to match all values for this key.

Add, edit, and remove tags

To manage tags:

Click the + button next to “Tags” to add a new tag.
Select or type a tag key.
Select or type a tag value (use * for wildcard).
Click add tag to apply the tag filter.
To edit an existing tag, click the pencil icon next to it.
To remove a tag, click the x icon next to it.

Note
Tags are deprecated in OpenTSDB 2.2 and later. Use Filters instead for more powerful filtering options including wildcards, regular expressions, and exclusion patterns.

Caution
Tags and Filters are mutually exclusive. If you have filters defined, you cannot add tags, and vice versa. The query editor displays a warning if you attempt to use both.

Rate section

The Rate section computes the rate of change, which is essential for counter metrics that continuously increment.

Field	Description
Rate	Toggle to enable rate calculation. Computes the per-second rate of change between consecutive values.
Counter	(When Rate is enabled) Toggle to indicate the metric is a monotonically increasing counter that may reset.
Counter max	(When Counter is enabled) The maximum value before the counter wraps around.
Reset value	(When Counter is enabled) The value the counter resets to after wrapping. Default: `0`.
Explicit tags	(Version 2.3+) Toggle to require all specified tags to exist on matching time series.

When to use rate calculation

Enable Rate when your metric is a continuously increasing counter, such as:

Network bytes sent/received
Request counts
Error counts
Disk I/O operations

The rate calculation converts cumulative values into per-second rates, making the data more meaningful for visualization.

Counter settings

Enable Counter when your metric can reset to zero (for example, after a service restart). The counter settings help OpenTSDB calculate correct rates across resets:

Counter max: Set this to the maximum value your counter can reach before wrapping. For 64-bit counters, use 18446744073709551615. For 32-bit counters, use 4294967295.
Reset value: The value the counter resets to, typically 0.

Explicit tags

When Explicit tags is enabled (version 2.3+), OpenTSDB only returns time series that have all the tags specified in your query. This prevents unexpected results when some time series are missing tags that others have.

Aggregators

The aggregator function combines multiple time series into one. Grafana fetches the list of available aggregators from your OpenTSDB server, so you may see additional aggregators beyond those listed here.

Common aggregators

Aggregator	Description	Use case
`sum`	Sum all values at each timestamp.	Total requests across all servers.
`avg`	Average all values at each timestamp.	Average CPU usage across hosts.
`min`	Take the minimum value at each timestamp.	Lowest response time.
`max`	Take the maximum value at each timestamp.	Peak memory usage.
`dev`	Calculate the standard deviation.	Measure variability in response times.
`count`	Count the number of data points.	Number of reporting hosts.

Interpolation aggregators

These aggregators handle missing data points differently:

Aggregator	Description
`zimsum`	Sum values, treating missing data as zero.
`mimmin`	Minimum value, ignoring missing (interpolated) data.
`mimmax`	Maximum value, ignoring missing (interpolated) data.

Note
The available aggregators depend on your OpenTSDB server version and configuration. The aggregator dropdown is populated dynamically from the /api/aggregators endpoint on your OpenTSDB server.

Fill policies

Fill policies (available in OpenTSDB 2.2+) determine how to handle missing data points during downsampling. This is important when your data has gaps or irregular collection intervals.

Policy	Description	Use case
`none`	Don’t fill missing values. Gaps remain in the data.	Default behavior; preserves data fidelity.
`nan`	Fill missing values with NaN (Not a Number).	Useful for calculations that should propagate missing data.
`null`	Fill missing values with null.	Visualizations show gaps at null points.
`zero`	Fill missing values with zero.	Treat missing data as zero values; useful for counters.

Choose the right fill policy

Use none (default) when you want to see actual data gaps in your visualizations.
Use null when you want graphs to show breaks at missing data points.
Use zero when missing data should be interpreted as zero (for example, no requests during a period).
Use nan when you need missing values to propagate through calculations.

Autocomplete suggestions

The query editor provides autocomplete suggestions to help you build queries quickly and accurately.

What autocomplete provides

Field	Source	Description
Metric	`/api/suggest?type=metrics`	Suggests metric names as you type.
Tag keys	Previous query results	Suggests tag keys based on the selected metric.
Tag values	`/api/suggest?type=tagv`	Suggests tag values as you type.
Filter keys	Previous query results	Suggests tag keys for filter configuration.

Autocomplete requirements

For autocomplete to work:

The OpenTSDB suggest API must be enabled on your server.
Metrics must exist in your OpenTSDB database.
The Lookup limit setting in your data source configuration controls the maximum number of suggestions returned.

If autocomplete isn’t working, refer to Troubleshooting.

Use template variables

You can use template variables in any text field in the query editor. Template variables are replaced with their current values when the query executes.

Common uses include:

Metric field: $metric to dynamically select metrics.
Filter values: $host to filter by a variable-selected host.
Tag values: $environment to filter by environment.

For more information about creating and using template variables, refer to Template variables.

Query examples

The following examples demonstrate common query patterns.

Basic metric query with tag filtering

Field	Value
Metric	`sys.cpu.user`
Aggregator	`avg`
Tags	`host=webserver01`

This query returns the average CPU usage for the host webserver01.

Query with wildcard filter (OpenTSDB 2.2+)

Field	Value
Metric	`http.requests.count`
Aggregator	`sum`
Filter Key	`host`
Filter Type	`wildcard`
Filter Value	`web-*`
Group by	enabled

This query sums HTTP request counts across all hosts matching web-* and groups results by host.

Rate calculation for network counters

Field	Value
Metric	`net.bytes.received`
Aggregator	`sum`
Rate	enabled
Counter	enabled
Counter max	`18446744073709551615`

This query calculates the rate of bytes received per second. The counter max is set to the 64-bit unsigned integer maximum to handle counter wraps correctly.

Using alias patterns

Field	Value
Metric	`app.response.time`
Aggregator	`avg`
Tags	`host=*`, `env=production`
Alias	`$tag_host - Response Time`

This query uses the alias pattern to create readable legend labels like webserver01 - Response Time.

Downsampling with custom interval

Field	Value
Metric	`sys.disk.io.bytes`
Aggregator	`sum`
Downsample Interval	`5m`
Downsample Aggregator	`avg`
Fill	`zero`

This query downsamples disk I/O data to 5-minute averages, filling gaps with zero values.

Compare environments with filters

Field	Value
Metric	`app.errors.count`
Aggregator	`sum`
Filter Key	`env`
Filter Type	`literal_or`
Filter Value	`staging\|production`
Group by	enabled

This query shows error counts for both staging and production environments as separate time series for comparison.

Exclude specific hosts

Field	Value
Metric	`sys.cpu.user`
Aggregator	`avg`
Filter Key	`host`
Filter Type	`not_literal_or`
Filter Value	`test-server\|dev-server`
Group by	enabled

This query shows CPU usage for all hosts except test-server and dev-server.

Query with explicit tags (version 2.3+)

Field	Value
Metric	`app.request.latency`
Aggregator	`avg`
Filter Key	`host`
Filter Type	`wildcard`
Filter Value	`*`
Group by	enabled
Explicit tags	enabled

This query only returns time series that have the host tag defined, excluding any time series that are missing this tag.

Next steps

Use template variables to create dynamic, reusable dashboards.
Set up alerting to get notified when metrics cross thresholds.
Troubleshoot issues if you encounter problems with queries.

OpenTSDB template variables

Fri, 03 Apr 2026 19:43:06 +0000

OpenTSDB template variables

Instead of hard-coding server, application, and sensor names in your metric queries, you can use template variables. Variables appear as drop-down menus at the top of the dashboard, making it easy to change the data being displayed without editing queries.

For an introduction to template variables, refer to the Variables documentation.

Query variable

The OpenTSDB data source supports query-type template variables that fetch values directly from OpenTSDB. These variables dynamically populate based on data in your OpenTSDB database.

Supported query functions

Query	Description	API used
`metrics(prefix)`	Returns metric names matching the prefix. Use empty parentheses for all metrics.	`/api/suggest?type=metrics`
`tag_names(metric)`	Returns tag keys (names) that exist for a specific metric.	`/api/search/lookup`
`tag_values(metric, tagkey)`	Returns tag values for a specific metric and tag key.	`/api/search/lookup`
`suggest_tagk(prefix)`	Returns tag keys matching the prefix across all metrics.	`/api/suggest?type=tagk`
`suggest_tagv(prefix)`	Returns tag values matching the prefix across all metrics.	`/api/suggest?type=tagv`

Note
The tag_names and tag_values functions use the OpenTSDB lookup API, which requires metrics to exist in your database. The suggest_tagk and suggest_tagv functions use the suggest API, which searches across all metrics.

Create a query variable

To create a query variable:

Navigate to Dashboard settings > Variables.
Click Add variable.
Enter a Name for your variable (for example, host).
Select Query as the variable type.
Select the OpenTSDB data source.
Enter your query using one of the supported query functions.
Optionally configure Multi-value to allow selecting multiple values.
Optionally configure Include All option to add an “All” option.
Click Apply.

Query variable examples

List all metrics:

metrics()

Returns all metric names in your OpenTSDB database. Useful for creating a metric selector.

List metrics with a prefix:

metrics(sys.cpu)

Returns metrics starting with sys.cpu, such as sys.cpu.user, sys.cpu.system, sys.cpu.idle.

List tag keys for a metric:

tag_names(sys.cpu.user)

Returns tag keys like host, env, datacenter that exist on the sys.cpu.user metric.

List tag values for a metric and tag key:

tag_values(sys.cpu.user, host)

Returns all host values for the sys.cpu.user metric, such as webserver01, webserver02, dbserver01.

Search for tag keys by prefix:

suggest_tagk(host)

Returns tag keys matching host across all metrics, such as host, hostname, host_id.

Search for tag values by prefix:

suggest_tagv(web)

Returns tag values matching web across all metrics, such as webserver01, webserver02, web-prod-01.

If template variables aren’t populating in the Preview of values section, refer to Troubleshooting.

Nested template variables

You can use one template variable to filter values for another. This creates cascading filters, such as selecting a data center first, then showing only hosts in that data center.

Filter syntax

The tag_values function accepts additional tag filters after the tag key:

tag_values(metric, tagkey, tag1=value1, tag2=value2, ...)

Use template variables as filter values to create dynamic dependencies:

tag_values(metric, tagkey, tag1=$variable1, tag2=$variable2)

Nested variable examples

Query	Description
`tag_values(sys.cpu.user, host, env=$env)`	Returns host values filtered by the selected `env` value.
`tag_values(sys.cpu.user, host, env=$env, datacenter=$dc)`	Returns host values filtered by both `env` and `datacenter`.
`tag_values(app.requests, endpoint, service=$service)`	Returns endpoint values for the selected service.

Create cascading filters

To create a hierarchy of dependent variables:

Create the parent variable:
- Name: datacenter
- Query: tag_values(sys.cpu.user, datacenter)
Create the child variable:
- Name: host
- Query: tag_values(sys.cpu.user, host, datacenter=$datacenter)
Create additional levels as needed:
- Name: cpu
- Query: tag_values(sys.cpu.user, cpu, datacenter=$datacenter, host=$host)

When users change the data center selection, the host variable automatically refreshes to show only hosts in that data center.

Use variables in queries

Reference variables in your queries using the $variablename or ${variablename} syntax. Grafana replaces the variable with its current value when the query executes.

Where to use variables

Variables can be used in these query editor fields:

Field	Example	Description
Metric	`$metric`	Dynamically select which metric to query.
Tag value	`host=$host`	Filter by a variable-selected tag value.
Filter value	`$host`	Use in filter value field for filtering.
Alias	`$tag_host - $host`	Include variable values in legend labels.
Downsample interval	`$interval`	Use a variable for the downsample interval.

Variable syntax options

Syntax	Description
`$variablename`	Simple syntax for most cases.
`${variablename}`	Use when the variable is adjacent to other text (for example, `${host}_suffix`).
`${variablename:format}`	Apply a specific format to the variable value.

Multi-value variables

When you enable Multi-value for a variable, users can select multiple values simultaneously. The OpenTSDB data source handles multi-value variables using pipe (|) separation, which is compatible with OpenTSDB’s literal_or filter type.

Configure multi-value variables

When creating the variable, enable Multi-value.
Optionally enable Include All option to add an “All” selection.
Use the variable in a filter with the literal_or filter type.

Multi-value example

With a host variable configured as multi-value:

Field	Value
Filter Key	`host`
Filter Type	`literal_or`
Filter Value	`$host`

If the user selects webserver01, webserver02, and webserver03, the filter value becomes webserver01|webserver02|webserver03.

All value behavior

When the user selects “All”, Grafana sends all available values pipe-separated. For large value sets, consider using a wildcard filter instead:

Field	Value
Filter Key	`host`
Filter Type	`wildcard`
Filter Value	`*`

Interval and auto-interval variables

Grafana provides built-in interval variables that are useful with OpenTSDB downsampling:

Variable	Description
`$__interval`	Automatically calculated interval based on time range and panel width.
`$__interval_ms`	Same as `$__interval` but in milliseconds.

Use these in the downsample interval field for automatic interval adjustment:

Field	Value
Downsample Interval	`$__interval`

Next steps

Build queries using your template variables.
Set up alerting with templated queries.
Troubleshoot issues if variables aren’t populating.

OpenTSDB alerting

Fri, 03 Apr 2026 19:43:06 +0000

OpenTSDB alerting

You can use Grafana Alerting with OpenTSDB to create alerts based on your time series data. This allows you to monitor metrics, detect anomalies, and receive notifications when specific conditions are met.

For general information about Grafana Alerting, refer to Grafana Alerting.

Before you begin

Before creating alerts with OpenTSDB, ensure you have:

An OpenTSDB data source configured in Grafana. Refer to Configure the OpenTSDB data source.
Appropriate permissions to create alert rules.
Understanding of the metrics you want to monitor.

Supported features

OpenTSDB alerting works with standard metric queries that return time series data. The following table summarizes alerting compatibility:

Query type	Alerting support	Notes
Metrics with aggregation	Yes	Recommended for alerting
Metrics with downsampling	Yes	Use appropriate intervals
Metrics with rate calculation	Yes	Useful for counter metrics
Metrics with filters/tags	Yes	Filter to specific hosts or services

Create an alert rule

To create an alert rule using OpenTSDB:

Navigate to Alerting > Alert rules.
Click New alert rule.
Enter a name for the alert rule.
Select your OpenTSDB data source.
Build your query:
- Select the metric to monitor.
- Choose an appropriate aggregator (for example, avg, sum, max).
- Add tag filters to target specific resources.
- Enable downsampling with an interval matching your evaluation frequency.
Configure the alert condition (for example, when the value is above a threshold).
Set the evaluation interval and pending period.
Configure notifications and labels.
Click Save rule.

For detailed instructions, refer to Create a Grafana-managed alert rule.

Example alert queries

The following examples show common alerting scenarios with OpenTSDB.

Alert on high CPU usage

Monitor CPU usage and alert when it exceeds 90%:

Field	Value
Metric	`sys.cpu.user`
Aggregator	`avg`
Tags	`host=*`
Downsample Interval	`1m`
Downsample Aggregator	`avg`

Condition: When average is above 90

Alert on low disk space

Monitor available disk space and alert when it drops below a threshold:

Field	Value
Metric	`sys.disk.free`
Aggregator	`min`
Tags	`host=*`, `mount=/`
Downsample Interval	`5m`
Downsample Aggregator	`min`

Condition: When minimum is below 10737418240 (10 GB in bytes)

Alert on high network traffic rate

Monitor network bytes received and alert on high traffic:

Field	Value
Metric	`net.bytes.received`
Aggregator	`sum`
Tags	`host=webserver01`
Rate	enabled
Counter	enabled
Downsample Interval	`1m`
Downsample Aggregator	`avg`

Condition: When sum is above 104857600 (100 MB/s in bytes)

Alert on error count spike

Monitor application error counts:

Field	Value
Metric	`app.errors.count`
Aggregator	`sum`
Tags	`service=api`, `env=production`
Downsample Interval	`5m`
Downsample Aggregator	`sum`

Condition: When sum is above 100

Limitations

When using OpenTSDB with Grafana Alerting, be aware of the following limitations.

Template variables not supported

Alert queries can’t contain template variables. Grafana evaluates alert rules on the backend without dashboard context, so variables like $hostname or $environment aren’t resolved.

If your dashboard query uses template variables, create a separate query for alerting with hard-coded values.

Query complexity

Complex queries with many tags or long time ranges may timeout or fail to evaluate. Simplify queries for alerting by:

Using specific tag filters instead of wildcards where possible.
Enabling downsampling with appropriate intervals.
Reducing the evaluation time range.

OpenTSDB 2.4 behavior

When using OpenTSDB 2.4 with alerting, Grafana executes queries with the parameter arrays=true. This causes OpenTSDB to return data points as an array of arrays instead of a map of key-value pairs. Grafana automatically converts this data to the appropriate format.

Best practices

Follow these best practices when creating OpenTSDB alerts:

Use specific tag filters: Add tag filters to focus on relevant resources and improve query performance.
Match downsample interval to evaluation: Set the downsample interval to match or be slightly smaller than your alert evaluation interval.
Test queries first: Verify your query returns expected results in Explore before creating an alert.
Set realistic thresholds: Base alert thresholds on historical data patterns to avoid false positives.
Use meaningful names: Give alert rules descriptive names that indicate what they monitor.
Enable downsampling: Always enable downsampling for alerting queries to reduce data volume and improve reliability.
Consider counter resets: For counter metrics, enable the Counter option and set appropriate max values to handle resets correctly.

Next steps

Build queries to explore your metrics before creating alerts.
Troubleshoot issues if alerts aren’t firing as expected.

OpenTSDB annotations

Fri, 03 Apr 2026 19:43:06 +0000

OpenTSDB annotations

Annotations allow you to overlay event information on graphs, providing context for metric changes. The OpenTSDB data source supports both metric-specific annotations and global annotations stored in OpenTSDB.

For general information about annotations in Grafana, refer to Annotate visualizations.

Annotation types

OpenTSDB supports two types of annotations:

Type	Description	Use case
Metric annotations	Annotations attached to a specific time series (TSUID). Retrieved by querying the associated metric.	Track events affecting a specific host or service.
Global annotations	Annotations not tied to any time series. Apply system-wide.	Track deployments, maintenance windows, or infrastructure-wide events.

How Grafana retrieves annotations

When you configure an annotation query, Grafana queries OpenTSDB for the specified metric and retrieves any annotations associated with that metric’s time series. The query includes the globalAnnotations=true parameter, which allows Grafana to also retrieve global annotations when enabled.

Grafana displays the description field from each annotation as the annotation text.

Configure an annotation query

To add OpenTSDB annotations to a dashboard:

Click the dashboard settings icon (gear) in the top navigation.
Select Annotations in the left menu.
Click Add annotation query.
Select the OpenTSDB data source.
Configure the annotation query fields as described in the following table.
Click Save dashboard.

Annotation query fields

Field	Description
Name	A descriptive name for this annotation query. Appears in the annotation legend.
Data source	Select the OpenTSDB data source.
Enabled	Toggle to enable or disable this annotation query.
OpenTSDB metrics query	The metric name to query for annotations (for example, `events.deployment`).
Show Global Annotations	Toggle to include global annotations that aren’t tied to a specific time series.

Example annotation queries

The following examples demonstrate common annotation use cases.

Track application deployments

Monitor when deployments occur for a specific application:

Field	Value
Name	App Deployments
OpenTSDB metrics query	`deploy.myapp`
Show Global Annotations	disabled

This query retrieves annotations attached to the deploy.myapp metric, showing deployment events for that specific application.

Monitor infrastructure-wide events

Capture system-wide events such as network changes or datacenter maintenance:

Field	Value
Name	Infrastructure Events
OpenTSDB metrics query	`events.infrastructure`
Show Global Annotations	enabled

This query retrieves both metric-specific and global annotations, providing a complete picture of infrastructure events.

Track incidents and outages

Mark incident start and resolution times:

Field	Value
Name	Incidents
OpenTSDB metrics query	`events.incident`
Show Global Annotations	enabled

Monitor configuration changes

Track when configuration changes are applied:

Field	Value
Name	Config Changes
OpenTSDB metrics query	`events.config`
Show Global Annotations	disabled

Correlate multiple event types

You can add multiple annotation queries to a single dashboard to correlate different event types. For example:

Add a “Deployments” annotation query for deploy.* metrics.
Add an “Incidents” annotation query for events.incident.
Add a “Maintenance” annotation query with global annotations enabled.

This allows you to see how deployments, incidents, and maintenance windows relate to your metric data.

How annotations appear

Annotations appear as vertical lines on time series panels at the timestamps where events occurred. Hover over an annotation marker to view:

The annotation name (from your query configuration)
The event description (from the OpenTSDB annotation’s description field)
The timestamp

Different annotation queries can be assigned different colors in the dashboard settings to distinguish between event types.

Create annotations in OpenTSDB

To display annotations in Grafana, you must first create them in OpenTSDB. OpenTSDB provides an HTTP API for managing annotations.

Annotation data structure

OpenTSDB annotations have the following fields:

Field	Required	Description
`startTime`	Yes	Unix epoch timestamp in seconds when the event started.
`endTime`	No	Unix epoch timestamp in seconds when the event ended. Useful for duration-based events.
`tsuid`	No	The time series UID to associate this annotation with. If empty, the annotation is global.
`description`	No	Brief description of the event. This text displays in Grafana.
`notes`	No	Detailed notes about the event.
`custom`	No	A map of custom key-value pairs for additional metadata.

Create a global annotation

Use the OpenTSDB API to create a global annotation:

curl -X POST http://<OPENTSDB_HOST>:4242/api/annotation \
  -H "Content-Type: application/json" \
  -d '{
    "startTime": 1609459200,
    "description": "Production deployment v2.5.0",
    "notes": "Deployed new feature flags and performance improvements",
    "custom": {
      "version": "2.5.0",
      "environment": "production",
      "deployer": "jenkins"
    }
  }'

Create a metric-specific annotation

To attach an annotation to a specific time series, include the tsuid:

curl -X POST http://<OPENTSDB_HOST>:4242/api/annotation \
  -H "Content-Type: application/json" \
  -d '{
    "startTime": 1609459200,
    "endTime": 1609462800,
    "tsuid": "000001000001000001",
    "description": "Server maintenance",
    "notes": "Scheduled maintenance window for hardware upgrade"
  }'

To find the TSUID for a metric, use the OpenTSDB /api/uid/tsmeta endpoint.

Create annotations programmatically

Integrate annotation creation into your deployment pipelines or monitoring systems:

Deployment script example:

#!/bin/bash
VERSION=$1
TIMESTAMP=$(date +%s)

curl -X POST http://opentsdb.example.com:4242/api/annotation \
  -H "Content-Type: application/json" \
  -d "{
    \"startTime\": $TIMESTAMP,
    \"description\": \"Deployed version $VERSION\",
    \"custom\": {
      \"version\": \"$VERSION\",
      \"environment\": \"production\"
    }
  }"

For more details on the annotation API, refer to the OpenTSDB annotation API documentation.

Troubleshoot annotation issues

The following section addresses common issues you may encounter when using OpenTSDB annotations.

Annotations don’t appear

Possible causes and solutions:

Cause	Solution
Time range doesn’t include annotations	Expand the dashboard time range to include the annotation timestamps.
Wrong metric name	Verify the metric name in your annotation query matches the metric associated with the annotations in OpenTSDB.
Annotations are global but toggle is off	Enable Show Global Annotations if your annotations don’t have a TSUID.
No annotations exist	Verify annotations exist in OpenTSDB using the API: `curl http://<OPENTSDB_HOST>:4242/api/annotation?startTime=<START>&endTime=<END>`

Annotation text is empty

The annotation displays but has no description text.

Solution: Ensure the description field is populated when creating annotations in OpenTSDB. Grafana displays the description field as the annotation text.

Next steps

Build queries to visualize metrics alongside annotations.
Use template variables to create dynamic dashboards.
Set up alerting to get notified when metrics cross thresholds.

Troubleshoot OpenTSDB data source issues

Fri, 03 Apr 2026 19:43:06 +0000

Troubleshoot OpenTSDB data source issues

This document provides solutions to common issues you may encounter when configuring or using the OpenTSDB data source. For configuration instructions, refer to Configure the OpenTSDB data source.

Connection errors

These errors occur when Grafana can’t connect to the OpenTSDB server.

“Connection refused” or timeout errors

Symptoms:

Save & test fails
Queries return connection errors
Intermittent timeouts

Possible causes and solutions:

Cause	Solution
Wrong URL or port	Verify the URL includes the correct protocol, IP address, and port. The default port is `4242`.
OpenTSDB not running	Check that the OpenTSDB server is running and accessible.
Firewall blocking connection	Ensure firewall rules allow outbound connections from Grafana to the OpenTSDB server on the configured port.
Network issues	Verify network connectivity between Grafana and OpenTSDB. Try pinging the server or using `curl` to test the API.

To test connectivity manually, run:

curl http://<OPENTSDB_HOST>:4242/api/version

Authentication errors

These errors occur when credentials are invalid or misconfigured.

“401 Unauthorized” or “403 Forbidden”

Symptoms:

Save & test fails with authentication error
Queries return authorization errors

Solutions:

Verify that basic authentication credentials are correct in the data source configuration.
Check that the OpenTSDB server is configured to accept the provided credentials.
If using cookies for authentication, ensure the required cookies are listed in Allowed cookies.

Query errors

These errors occur when executing queries against OpenTSDB.

No data returned

Symptoms:

Query executes without error but returns no data
Panels show “No data” message

Possible causes and solutions:

Cause	Solution
Time range doesn’t contain data	Expand the dashboard time range. Verify data exists in OpenTSDB for the selected period.
Wrong metric name	Verify the metric name is correct. Use autocomplete to discover available metrics.
Incorrect tag filters	Remove or adjust tag filters. Use `*` as a wildcard to match all values.
Version mismatch	Ensure the configured OpenTSDB version matches your server. Filters are only available in version 2.2+.
Using both Filters and Tags	Use either Filters or Tags, not both. They’re mutually exclusive in OpenTSDB 2.2+.

Query timeout

Symptoms:

Queries take a long time and then fail
Error message mentions timeout

Solutions:

Reduce the time range of your query.
Add more specific tag filters to reduce the data volume.
Increase the Timeout setting in the data source configuration.
Enable downsampling to reduce the number of data points returned.
Check OpenTSDB server performance and HBase health.

Autocomplete doesn’t work

Symptoms:

No suggestions appear when typing metric names, tag names, or tag values
Drop-down menus are empty

Solutions:

Verify that the OpenTSDB /api/suggest endpoint is accessible. Test it manually with curl http://<OPENTSDB_HOST>:4242/api/suggest?type=metrics.
Increase the Lookup limit setting if you have many metrics or tags.
Verify that the data source connection is working by clicking Save & test.
Check that metrics exist in OpenTSDB. The suggest API only returns metrics that have been written to the database.

Template variables don’t populate

Symptoms:

Template variable drop-down menus are empty
Preview of values shows no results

Solutions:

Enable real-time metadata tracking in OpenTSDB by setting tsd.core.meta.enable_realtime_ts to true in your OpenTSDB configuration.
Sync existing metadata by running tsdb uid metasync on the OpenTSDB server.
Verify the variable query syntax is correct. Refer to Template variables for the correct syntax.
Check that the data source connection is working.

Performance issues

These issues relate to slow queries or high resource usage.

Slow queries

Symptoms:

Dashboards take a long time to load
Queries are slow even for small time ranges

Solutions:

Enable downsampling in the query editor to reduce data volume.
Use more specific tag filters to limit the time series returned.
Reduce the time range.
Check OpenTSDB and HBase performance metrics.
Consider increasing OpenTSDB heap size if memory is constrained.

HBase performance issues

OpenTSDB relies on HBase for data storage. Performance problems in HBase directly affect OpenTSDB query performance.

Solutions:

Monitor HBase region server health and compaction status.
Ensure sufficient heap memory is allocated to HBase region servers.
Check for region hotspots and rebalance if necessary.
Refer to the OpenTSDB troubleshooting guide for HBase-specific issues.

Enable debug logging

To capture detailed error information for troubleshooting:

Set the Grafana log level to debug in the configuration file:
ini
```
[log]
level = debug
```
Review logs in /var/log/grafana/grafana.log (or your configured log location).
Look for OpenTSDB-specific entries that include request and response details.
Reset the log level to info after troubleshooting to avoid excessive log volume.

Get additional help

If you’ve tried the solutions in this document and still encounter issues:

Check the Grafana community forums for similar issues.
Review OpenTSDB issues on GitHub for known bugs.
Consult the OpenTSDB documentation for server-specific guidance.
Contact Grafana Support if you’re a Grafana Enterprise, Cloud Pro, or Cloud Contracted user.

When reporting issues, include:

Grafana version
OpenTSDB version
Error messages (redact sensitive information)
Steps to reproduce
Data source configuration (redact credentials)