Documentation Grafana Cloud Monitor infrastructure Cloud Provider Observability AWS observability CloudWatch metrics Configure CloudWatch metric streams

Configure CloudWatch metric streams

With Grafana Cloud Observability Amazon CloudWatch metric streams, you can push CloudWatch metrics using Amazon Data Firehose, providing real-time insights and scalability while simplifying configuration and reducing manual effort.

What gets created

The configuration process sets up the following components:

• Access policy token: A Grafana Cloud token with metric:write permissions
• AWS IAM roles: Multiple roles that authorize communication among required AWS services and between AWS services and Grafana Cloud
• CloudWatch metric stream: Captures metrics from CloudWatch in real-time and puts them on a Data Firehose delivery stream
• Data Firehose delivery stream: Delivers batches of metrics to Grafana Cloud using an HTTP endpoint
• AWS resource metadata scrape job: Enriches metrics with additional metadata like resource ARNs and tags

Data flow

How it works

1. CloudWatch continuously streams metrics to the metric stream
2. The metric stream forwards data to Data Firehose in OpenTelemetry format
3. Data Firehose batches and delivers metrics to your Grafana Cloud endpoint
4. If delivery fails, Data Firehose backs up failed batches to S3
5. The resource metadata scraper periodically fetches AWS resource tags and metadata
6. Grafana Cloud combines metric stream data with resource metadata for rich context

Before you begin

Before configuring CloudWatch metric streams, ensure you have:

• AWS permissions: Sufficient access to your AWS account to manage IAM roles, CloudWatch metric streams, and Data Firehose delivery streams
• Grafana Cloud access: Sufficient access to your Grafana Cloud portal with permissions to create access policy tokens
• Configuration tool: Either AWS CloudFormation or Terraform installed and configured
  • For CloudFormation: Access to the AWS account you want to monitor
  • For Terraform: Terraform 1.0 or later installed locally

If you are not using either the provided CloudFormation template or Terraform sample code to configure Grafana Cloud AWS Metric Streams, you need the following:

• Grafana Cloud AWS Metric Streams Ingest URL and Prometheus Username/Instance ID: The Grafana Cloud AWS Metric Streams Ingest URL is the URL you configure in the Data Firehose to send metrics to your Grafana Cloud instance. The Prometheus Username/Instance ID is used for the Data Firehose to authenticate when connecting to your Prometheus instance.

  These values are generated in the provided CloudFormation template and Terraform sample code, but if you are using a different method such as creating AWS infrastructure in the AWS console, you need to obtain them.

Obtain the ingest URL and Username

The Grafana Cloud AWS Metric Streams Ingest URL and Prometheus Username/Instance ID are generated in the provided CloudFormation template and Terraform sample code, but if you are using a different method for configuring Grafana Cloud AWS Metric Streams such as creating AWS infrastructure in the AWS console, you need to obtain them.

The Grafana Cloud AWS Metric Streams Ingest URL is the URL you configure in the Data Firehose to send metrics to your Grafana Cloud instance. The Prometheus Username/Instance ID is used for the Data Firehose to authenticate when connecting to your Prometheus instance.

You can obtain them either in the UI in Grafana Cloud or programmatically using bash, jq, and the GCom API.

Obtain the ingest URL and Username in the Grafana Cloud UI

To obtain the Grafana Cloud AWS Metric Streams Ingest URL and Prometheus Username/Instance ID in the Grafana Cloud UI, perform the following steps:

1. Navigate to your Grafana Cloud portal.
2. Select your Grafana Cloud stack.
3. Locate the Prometheus tile, and click Details.
4. Copy and save the values for the Remote Write Endpoint and Username/Instance ID. The Username/Instance ID is used for the Data Firehose to authenticate when connecting to your Prometheus instance.
5. Derive your Grafana Cloud AWS Metric Streams Ingest URL from your Prometheus Remote Write Endpoint using the following steps:
   1. Remove the prometheus- prefix from the hostname to reveal your Mimir cell ID (for example, prod-<number>).
   2. Insert the Mimir cell ID into the following Grafana Cloud Metric Streams URL template: https://aws-metric-streams-<MIMIR_CELL_ID>.grafana.net/aws-metrics/api/v1/push
      • For example, if your Prometheus Remote Write Endpoint is https://prometheus-prod-03-prod-us-central-0.grafana.net, then your Mimir cell ID is prod-03, and your Grafana Cloud AWS Metric Streams URL is https://aws-metric-streams-prod-03.grafana.net/aws-metrics/api/v1/push.

Obtain the ingest URL and Username programatically

You need an access policy token with the stacks:read scope to obtain the Grafana Cloud AWS Metrics Streams Ingest URL and Prometheus Username/Instance ID programmatically.

To obtain the Grafana Cloud AWS Metric Streams Ingest URL and Prometheus Username/Instance ID programmatically, enter the following commands in bash:

Bash Copy

STACK_INFO=$(curl -s -H "Authorization: Bearer $ACCESS_POLICY_TOKEN" \
  "https://grafana.com/api/instances/$STACK_SLUG")

read PROM_URL METRICS_USER <<< $(echo "$STACK_INFO" | jq -r '[.hmInstancePromUrl, (.hmInstancePromId | tostring)] | @tsv')
PROM_HOST=${PROM_URL#https://}
DOMAIN=${PROM_HOST#*.}
MIMIR_CELL_ID=$(echo "${PROM_HOST%%.*}" | grep -oE 'prod-[0-9]+')
echo "Grafana Cloud AWS Metric Streams Ingest URL: https://aws-metric-streams-${MIMIR_CELL_ID}.${DOMAIN}/aws-metrics/api/v1/push"
echo "Prometheus instance ID: ${METRICS_USER}"

Legacy Prometheus Read Write Endpoint hostnames

For some older Prometheus instances, the usual process for deriving the Grafana Cloud AWS Metric Streams URL does not work. In those instances use the following table to derive the Grafana Cloud AWS Metric Streams URL hostname:

Generate an access policy token

Both CloudFormation and Terraform configuration methods require an access policy token with metric:write permissions from Grafana Cloud.

To generate an access policy token:

1. Open your Grafana Cloud portal.
2. Expand Observability > Cloud provider in the main menu.
3. Select AWS, the Configuration tab, and the CloudWatch metric streams card.
4. Enter a name for the token and click Create token.
5. Copy the token value and store it securely. You’ll need it in the next steps.

If you are using Terraform, store this token value in your variables or secrets management system.

Choose your configuration method

You can configure CloudWatch metric streams using either CloudFormation or Terraform. Choose the method that best fits your infrastructure management approach.

CloudFormation

Best for:

• Quick setup with minimal configuration
• Teams primarily using AWS-native tools
• Simple deployments in a single AWS account, region, and namespace

Provides:

• Pre-built template with sensible defaults
• Quick deployment via AWS Console
• Automatic resource creation and dependency management

Configure with CloudFormation →

Terraform

Best for:

• Infrastructure as Code (IaC) workflows
• Multi-account or multi-region deployments
• Integration with existing Terraform configurations
• Version control and automated deployments

Provides:

• Full customization of all resources
• Integration with Terraform state management
• Reusable modules for multiple deployments
• Integration with CI/CD pipelines

Configure with Terraform →