Configure Grafana Mimir on Grafana Labs

About Grafana Mimir configurations

Wed, 03 Jun 2026 09:01:40 +0200

About Grafana Mimir configurations

You can configure Grafana Mimir via a (YAML-based) configuration file or CLI (command-line-interface) flags. It is best to specify your configuration via the configuration file rather than CLI flags. Every parameter that is set in the configuration file can also be set via a corresponding CLI flag. If you specify both CLI flags and configuration parameters, CLI flags take precedence over corresponding values in a configuration file. You can specify the configuration file by using the -config.file CLI flag.

To see the most common CLI flags that you need to get started with Grafana Mimir, run the mimir -help command. To see all of the available CLI flags, run the mimir -help-all command.

A given configuration loads at startup and cannot be modified at runtime. However, Grafana Mimir does have a second configuration file, known as the runtime configuration, that is dynamically reloaded. For more information, refer to About runtime configuration.

To see the current configuration state of any component, use the /config or /runtime_config HTTP API endpoint.

Common configurations

Some configurations, such as object storage backend, are repeated for multiple components. To avoid repetition in the configuration file, use the common configuration section or -common.* CLI flags. Common configurations are first applied to all of the specific configurations, which allows the common configurations to be overridden later by specific values.

For example, the following configuration uses the same Amazon S3 object storage bucket called mimir. The common storage is located in the us-east region for both the ruler and alertmanager stores, and the blocks storage uses the mimir-blocks bucket from the same region:

common:
  storage:
    backend: s3
    s3:
      region: us-east
      bucket_name: mimir

blocks_storage:
  s3:
    bucket_name: mimir-blocks

For a reference of this configuration, see Configure Grafana Mimir object storage backend.

The precedence of the common configuration is as follows, where each configuration overrides the previous one:

YAML common values
YAML specific values
CLI common flags
CLI specific flags

Operational considerations

Use a single configuration file, and either pass it to all replicas of Grafana Mimir (if you are running multiple single-process Mimir replicas) or to all components of Grafana Mimir (if you are running Grafana Mimir as microservices). If you are running Grafana Mimir on Kubernetes, you can achieve this by storing the configuration file in a ConfigMap and mounting it in each Grafana Mimir container.

This recommendation helps to avoid a common misconfiguration pitfall: while certain configuration parameters might look like they’re only needed by one type of component, they might in fact be used by multiple components. For example, the -ingester.ring.replication-factor CLI flag is not only required by ingesters, but also by distributors, queriers, and rulers (in internal operational mode).

By using a single configuration file, you ensure that each component gets all of the configuration that it needs without needing to track which parameter belongs to which component. There is no harm in passing a configuration that is specific to one component (such as an ingester) to another component (such as a querier). In such case, the configuration is simply ignored.

If you need to, you can use advanced CLI flags to override specific values on a particular Grafana Mimir component or replica. This can be helpful if you want to change a parameter that is specific to a certain component, without having to do a full restart of all other components.

The most common use case for CLI flags is to use the -target flag to run Grafana Mimir as microservices. By setting the -target CLI flag, all Grafana Mimir components share the same configuration file, but you can make them behave as a given component by specifying a -target command-line value, such as -target=ingester or -target=querier.

Validate a configuration

To validate your configuration, run the command mimir -modules -config.file <path-to-config-file>. This is useful if you want to validate the configuration before deploying it. You can use the command locally or in a CI environment.

If the configuration file is valid, the command exits with a zero exit code and prints the available modules.
If the configuration file is invalid, the command exits with a non-zero exit code and prints the error message to standard output.

About Grafana Mimir runtime configuration

Wed, 03 Jun 2026 09:01:40 +0200

About Grafana Mimir runtime configuration

A runtime configuration file is a file that contains configuration parameters, which is periodically reloaded while Mimir is running. It allows you to change a subset of Grafana Mimir’s configuration without having to restart the Grafana Mimir component or instance.

Runtime configuration is available for a subset of the configuration that was set at startup. A Grafana Mimir operator can observe the configuration and use runtime configuration to make immediate adjustments to Grafana Mimir.

Runtime configuration values take precedence over command-line options.

If multiple runtime configuration files are specified the runtime configuration files will be merged in a left to right order.

Add a .gz extension to the configuration files that should be decompressed before parsing YAML.

Enable runtime configuration

To enable runtime configuration, specify a comma-separated list of file paths upon startup by using the -runtime-config.file=<filepath>,<filepath> CLI flag or from within your YAML configuration file in the runtime_config block.

By default, Grafana Mimir reloads the contents of these files every 10 seconds and merges these files from left to right. You can configure this interval by using the -runtime-config.reload-period=<duration> CLI flag or by specifying the period value in your YAML configuration file.

When running Grafana Mimir on Kubernetes, store the runtime configuration files in a ConfigMap and mount the ConfigMaps in each container.

Viewing the runtime configuration

Use Grafana Mimir’s /runtime_config endpoint to see the current value of the runtime configuration, including the overrides. To see only the non-default values of the configuration, specify the endpoint with /runtime_config?mode=diff.

Runtime configuration of per-tenant limits

The runtime configuration file is primarily used to set and adjust limits that are appropriate for each tenant based on their ingest and query needs.

The values that are defined in the limits section of your YAML configuration define the default set of limits that are applied to tenants. For example, if you set the ingestion_rate to 25,000 in your YAML configuration file, any tenant in your cluster that sends more than 25,000 samples per second (SPS) is rate limited.

You can use the runtime configuration file to override this behavior. For example, if you have a tenant (tenant1) that needs to send twice as many data points as the current limit, and you have another tenant (tenant2) that needs to send three times as many data points, you can modify the contents of your runtime configuration file as follows:

overrides:
  tenant1:
    ingestion_rate: 50000
  tenant2:
    ingestion_rate: 75000

As a result, Grafana Mimir allows tenant1 to send 50,000 SPS, and tenant2 to send 75,000 SPS, while maintaining a 25,000 SPS rate limit on all other tenants.

On a per-tenant basis, you can override all of the limits listed in the limits block within the runtime configuration file.
For each tenant, you can override different limits.
For any tenant or limit that is not overridden in the runtime configuration file, you can inherit the limit values that are specified in the limits block.

Ingester instance limits

The runtime configuration file can be used to dynamically adjust Grafana Mimir ingester instance limits. While per-tenant limits are limits applied to each tenant, per-ingester-instance limits are limits applied to each ingester process. Ingester limits ensure individual ingesters are not overwhelmed, regardless of any per-tenant limits. These limits can be set under the ingester.instance_limits block in the global configuration file, with CLI flags, or under the ingester_limits field in the runtime configuration file.

The runtime configuration allows you to override initial values, which is useful for advanced operators who need to dynamically change them in response to changes in ingest or query load.

Everything under the instance_limits section within the ingester block can be overridden via runtime configuration. The following example shows a portion of the runtime configuration that changes the ingester limits:

ingester_limits:
  max_ingestion_rate: 20000
  max_series: 1500000
  max_tenants: 1000
  max_inflight_push_requests: 30000

Distributor instance limits

The runtime configuration file can be used to dynamically adjust Grafana Mimir distributor instance limits. While per-tenant limits are limits applied to each tenant, per-distributor-instance limits are limits applied to each distributor process. Distributor limits ensure individual distributors are not overwhelmed, regardless of any per-tenant limits. These limits can be set under the distributor.instance_limits block in the global configuration file, with CLI flags, or under the distributor_limits field in the runtime configuration file.

The runtime configuration allows you to override initial values, which is useful for advanced operators who need to dynamically change them in response to changes in ingest load.

Everything under the instance_limits section within the distributor block can be overridden via runtime configuration. The following example shows a portion of the runtime configuration that changes the distributor limits:

distributor_limits:
  max_ingestion_rate: 75000
  max_inflight_push_requests: 1500
  max_inflight_push_requests_bytes: 314572800

About Grafana Mimir versioning

Wed, 03 Jun 2026 09:01:40 +0200

About Grafana Mimir versioning

This topic describes our guarantees for this Grafana Mimir major release.

Flags, configuration, and minor version upgrades

Upgrading Grafana Mimir from one minor version to the next minor version should work, but we don’t want to bump the major version every time we remove a configuration parameter. We will keep deprecated features in place for two minor releases. You can use the deprecated_flags_inuse_total metric to generate an alert that helps you determine if you’re using a deprecated flag.

These guarantees don’t apply to experimental features.

Reading old data

The Grafana Mimir maintainers commit to ensuring that future versions can read data written by versions within the last two years. In practice, we expect to be able to read data written more than two years ago, but a minimum of two years is our guarantee.

API Compatibility

Grafana Mimir strives to be 100% compatible with the Prometheus HTTP API which is by default served by endpoints with the /prometheus HTTP path prefix /prometheus/*.

We consider any deviation from this 100% API compatibility to be a bug, except for the following scenarios:

Additional API endpoints for creating, removing, modifying alerts, and recording rules.
Additional APIs that push metrics (under /prometheus/api/push).
Additional API endpoints for management of Grafana Mimir, such as the ring. These APIs are not included in any compatibility guarantees.
Delete series API.

Experimental features

Grafana Mimir is an actively developed project and we encourage the introduction of new features and capabilities. Not everything in each release of Grafana Mimir is considered production-ready. We mark as “Experimental” all features and flags that we don’t consider production-ready.

We do not guarantee backwards compatibility for experimental features and flags. Experimental configuration and flags are subject to change.

The following features are currently experimental:

Cost attribution
- Configure labels for cost attribution
  - -validation.cost-attribution-labels-structured
- Configure cost attribution limits, such as cardinality:
  - -validation.max-cost-attribution-cardinality
- Configure cooldown periods and eviction intervals for cost attribution
  - -validation.cost-attribution-cooldown
  - -cost-attribution.eviction-interval
- Configure the metrics endpoint dedicated to cost attribution
  - -cost-attribution.registry-path
- Configure the cost attribution cleanup process run interval
  - -cost-attribution.cleanup-interval
Alertmanager
- Health check grace period for connections to other replicas (-alertmanager.alertmanager-client.health-check-grace-period)
Compactor
- Limit blocks processed in each compaction cycle. Blocks uploaded prior to the maximum lookback aren’t processed.
  - -compactor.max-lookback
Ruler
- Allow defining limits on the maximum number of rules allowed in a rule group by namespace and the maximum number of rule groups by namespace. If set, this supersedes the -ruler.max-rules-per-rule-group and -ruler.max-rule-groups-per-tenant limits.
- -ruler.max-rules-per-rule-group-by-namespace
- -ruler.max-rule-groups-per-tenant-by-namespace
- Allow protecting rule groups from modification by namespace. Rule groups can always be read, and you can use the X-Mimir-Ruler-Override-Namespace-Protection header with namespace names as values to override protection from modification.
- -ruler.protected-namespaces
- Allow control over independent rules to be evaluated concurrently as long as they exceed a certain threshold on their rule group last duration runtime against their interval. We have both a limit on the number of rules that can be executed per ruler and per tenant:
- -ruler.max-independent-rule-evaluation-concurrency
- -ruler.max-independent-rule-evaluation-concurrency-per-tenant
- -ruler.independent-rule-evaluation-concurrency-min-duration-percentage
- -ruler.rule-evaluation-write-enabled
- Allow control over rule sync intervals.
  - ruler.outbound-sync-queue-poll-interval
  - ruler.inbound-sync-queue-poll-interval
- -ruler.min-rule-evaluation-interval
- Configure metric and label name validation scheme
  - -validation.name-validation-scheme
- Health check grace period for connections to other replicas (-ruler.client.health-check-grace-period)
Distributor
- Influx ingestion
  - /api/v1/push/influx/write endpoint
  - -distributor.influx-endpoint-enabled
  - -distributor.max-influx-request-size
- Limit exemplars per series per request
  - -distributor.max-exemplars-per-series-per-request
- Limit OTLP write request byte size
  - -distributor.max-otlp-request-size
- Enforce a maximum pool buffer size for write requests
  - -distributor.max-request-pool-buffer-size
- Enable conversion of OTel start timestamps to Prometheus zero samples to mark series start
  - -distributor.otel-created-timestamp-zero-ingestion-enabled
- Promote a certain set of OTel resource attributes to labels
  - -distributor.otel-promote-resource-attributes
- Allow keeping OpenTelemetry service.instance.id, service.name and service.namespace resource attributes in target_info on top of converting them to the instance and job labels.
  - -distributor.otel-keep-identifying-resource-attributes
- Enable conversion of OTel explicit bucket histograms into native histograms with custom buckets.
  - -distributor.otel-convert-histograms-to-nhcb
- Enable promotion of OTel scope metadata to metric labels
  - -distributor.otel-promote-scope-metadata
- Enable native ingestion of delta OTLP metrics. This means storing the raw delta sample values without converting them to cumulative values and having the metric type set to “Unknown”. Delta support is in an early stage of development. The ingestion and querying process is likely to change over time. You can find considerations around querying and gotchas in the corresponding Prometheus documentation.
  - distributor.otel-native-delta-ingestion
- Configure metric and label name validation scheme
  - -validation.name-validation-scheme
- Configure metric and label name translation strategy in OTLP endpoint
  - -distributor.otel-translation-strategy
- Allow controlling OTLP translation via request headers
  - -api.otlp-translation-headers-enabled
- Configure how to handle label values over the length limit
  - -validation.label-value-length-over-limit-strategy
- Enforce the out-of-order time window on the distributor when past_grace_period is 0
  - -validation.enforce-out-of-order-window-on-distributor
- Ingester health check grace period (-distributor.ingester-health-check-grace-period)
- Evaluate HA deduplication per timeseries within a write request instead of using the first series’ labels for the whole request
  - -distributor.ha-tracker.per-sample-dedupe
Ingester
- Add variance to chunks end time to spread writing across time (-blocks-storage.tsdb.head-chunks-end-time-variance)
- Snapshotting of in-memory TSDB data on disk when shutting down (-blocks-storage.tsdb.memory-snapshot-on-shutdown)
- Out-of-order samples ingestion (-ingester.out-of-order-time-window)
- Shipper labeling out-of-order blocks before upload to cloud storage (-ingester.out-of-order-blocks-external-label-enabled)
- Early TSDB Head compaction to reduce in-memory series:
  - -blocks-storage.tsdb.early-head-compaction-min-in-memory-series
  - -blocks-storage.tsdb.early-head-compaction-min-estimated-series-reduction-percentage
- Per-tenant early TSDB Head compaction based on owned series count:
  - -ingester.early-head-compaction-owned-series-threshold
  - -ingester.early-head-compaction-min-estimated-series-reduction-percentage
- Timely head compaction (-blocks-storage.tsdb.timely-head-compaction-enabled)
- Count owned series and use them to enforce series limits:
  - -ingester.track-ingester-owned-series
  - -ingester.use-ingester-owned-series-for-limits
  - -ingester.owned-series-update-interval
- Per-ingester circuit breaking based on requests timing out or hitting per-instance limits
  - -ingester.push-circuit-breaker.circuit-breaker.enabled
  - -ingester.push-circuit-breaker.failure-threshold-percentage
  - -ingester.push-circuit-breaker.failure-execution-threshold
  - -ingester.push-circuit-breaker.thresholding-period
  - -ingester.push-circuit-breaker.cooldown-period
  - -ingester.push-circuit-breaker.initial-delay
  - -ingester.push-circuit-breaker.request-timeout
  - -ingester.read-circuit-breaker.circuit-breaker.enabled
  - -ingester.read-circuit-breaker.failure-threshold-percentage
  - -ingester.read-circuit-breaker.failure-execution-threshold
  - -ingester.read-circuit-breaker.thresholding-period
  - -ingester.read-circuit-breaker.cooldown-period
  - -ingester.read-circuit-breaker.initial-delay
  - -ingester.read-circuit-breaker.request-timeout
- Reactive concurrency limiters
  - -distributor.reactive-limiter.enabled
  - -distributor.reactive-limiter.min-limit
  - -distributor.reactive-limiter.max-limit
  - -distributor.reactive-limiter.initial-limit
  - -distributor.reactive-limiter.max-limit-factor
  - -distributor.reactive-limiter.recent-window-min-duration
  - -distributor.reactive-limiter.recent-window-max-duration
  - -distributor.reactive-limiter.recent-window-min-samples
  - -distributor.reactive-limiter.recent-quantile
  - -distributor.reactive-limiter.baseline-window-age
  - -distributor.reactive-limiter.correlation-window
  - -distributor.reactive-limiter.initial-rejection-factor
  - -distributor.reactive-limiter.max-rejection-factor
  - -ingester.push-reactive-limiter.enabled
  - -ingester.push-reactive-limiter.min-limit
  - -ingester.push-reactive-limiter.max-limit
  - -ingester.push-reactive-limiter.initial-limit
  - -ingester.push-reactive-limiter.max-limit-factor
  - -ingester.push-reactive-limiter.recent-window-min-duration
  - -ingester.push-reactive-limiter.recent-window-max-duration
  - -ingester.push-reactive-limiter.recent-window-min-samples
  - -ingester.push-reactive-limiter.recent-quantile
  - -ingester.push-reactive-limiter.baseline-window-age
  - -ingester.push-reactive-limiter.correlation-window
  - -ingester.push-reactive-limiter.initial-rejection-factor
  - -ingester.push-reactive-limiter.max-rejection-factor
  - -ingester.read-reactive-limiter.enabled
  - -ingester.read-reactive-limiter.min-limit
  - -ingester.read-reactive-limiter.max-limit
  - -ingester.read-reactive-limiter.initial-limit
  - -ingester.read-reactive-limiter.max-limit-factor
  - -ingester.read-reactive-limiter.recent-window-min-duration
  - -ingester.read-reactive-limiter.recent-window-max-duration
  - -ingester.read-reactive-limiter.recent-window-min-samples
  - -ingester.read-reactive-limiter.recent-quantile
  - -ingester.read-reactive-limiter.baseline-window-age
  - -ingester.read-reactive-limiter.correlation-window
  - -ingester.read-reactive-limiter.initial-rejection-factor
  - -ingester.read-reactive-limiter.max-rejection-factor
  - -ingester.rejection-prioritizer.calibration-interval
- Postings for matchers cache sharing and invalidation. Sharing allows caches to be shared across multiple TSDBs. Invalidation allows cache entries to be marked as invalid, after which they’ll be lazily evicted when they hit their normal TTL or when the cache hits its max size.
  - -blocks-storage.tsdb.shared-postings-for-matchers-cache
  - -blocks-storage.tsdb.head-postings-for-matchers-cache-invalidation
  - -blocks-storage.tsdb.head-postings-for-matchers-cache-versions
- Index lookup planning comparison to validate planning correctness
  - -blocks-storage.tsdb.index-lookup-planning-enabled
  - -blocks-storage.tsdb.index-lookup-planning-comparison-portion
- Per-tenant max number of active series additional custom trackers is configurable via -validation.max-active-series-additional-custom-trackers.
- File based Kafka consumer group offset tracking enforcement
  - -ingest-storage.kafka.consumer-group-offset-commit-file-enforced
Querier
- Max concurrency for tenant federated queries (-tenant-federation.max-concurrent)
- Mimir query engine (-querier.query-engine and -querier.enable-query-engine-fallback, and all flags beginning with -querier.mimir-query-engine)
- Maximum estimated memory consumption per query limit (-querier.max-estimated-memory-consumption-per-query)
- Enable the experimental Prometheus feature for delayed name removal (-querier.enable-delayed-name-removal)
- Ignore deletion marks while querying delay (-blocks-storage.bucket-store.ignore-deletion-marks-while-querying-delay)
- Querier ring (all flags beginning with -querier.ring)
- Query-frontend health check grace period (-querier.frontend-client.health-check-grace-period)
- Store-gateway health check grace period (-querier.store-gateway-client.health-check-grace-period)
- Ingester health check grace period (-distributor.ingester-health-check-grace-period)
Query-frontend
- Lower TTL for cache entries overlapping the out-of-order samples ingestion window (re-using -ingester.out-of-order-window from ingesters)
- Sharding of active series queries (-query-frontend.shard-active-series-queries)
- Server-side write timeout for responses to active series requests (-query-frontend.active-series-write-timeout)
- Blocking HTTP requests on a per-tenant basis (configured with the blocked_requests limit)
- Spinning off (as actual range queries) subqueries from instant queries (-query-frontend.subquery-spin-off-enabled and the subquery_spin_off_enabled per-tenant limit)
- Support for cluster validation via -query-frontend.client-cluster-validation.label or -common.client-cluster-validation.label. Requests with invalid cluster validation labels are tracked via the cortex_client_invalid_cluster_validation_label_requests_total metric.
- Support for duration expressions in PromQL, which are simple arithmetics on numbers in offset and range specification.
- Support for configuring the maximum series limit for cardinality API requests on a per-tenant basis via cardinality_analysis_max_results.
- Mimir query engine (-query-frontend.query-engine and -query-frontend.enable-query-engine-fallback)
- Remote execution of queries in queriers: -query-frontend.enable-remote-execution=true and -query-frontend.enable-multiple-node-remote-execution-requests=true
- Performing query sharding within MQE: -query-frontend.use-mimir-query-engine-for-sharding=true
- Computing multiple aggregations over the same data without buffering: -querier.mimir-query-engine.enable-multi-aggregation=true
- Rewriting of queries to optimize processing: -query-frontend.rewrite-histogram-queries and -query-frontend.rewrite-propagate-matchers
- Enable experimental Prometheus extended range selector modifiers smoothed and anchored (-query-frontend.enabled-promql-extended-range-selectors=smoothed,anchored)
- Experimental PromQL functions and aggregations, including mad_over_time, ts_of_min_over_time, ts_of_max_over_time, ts_of_first_over_time, ts_of_last_over_time, sort_by_label, sort_by_label_desc, limitk and limit_ratio (-query-frontend.enabled-promql-experimental-functions=...)
Query-scheduler
- -query-scheduler.querier-forget-delay
Store-gateway
- Eagerly loading some blocks on startup even when lazy loading is enabled -blocks-storage.bucket-store.index-header.eager-loading-startup-enabled
- Allow more than the default of 3 store-gateways to own recent blocks -store-gateway.dynamic-replication
- Per-zone shard size, useful for computing automatic shard sizes based on the number of zone -store-gateway.tenant-shard-size-per-zone
- Specify the partitioner gap size for chunks (excluding posting & series) -blocks-storage.bucket-store.partitioner-max-gap-bytes-chunks
Metric separation by an additionally configured group label
- -validation.separate-metrics-group-label
- -max-separate-metrics-groups-per-user
Vault
- Fetching TLS secrets from Vault for various clients (-vault.enabled)
- Vault client authentication token lifetime watcher. Ensures the client token is always valid by renewing the token lease or re-authenticating. Includes the metrics:
  - cortex_vault_token_lease_renewal_active
  - cortex_vault_token_lease_renewal_success_total
  - cortex_vault_auth_success_total
Timeseries Unmarshal caching optimization in distributor (-timeseries-unmarshal-caching-optimization-enabled)
Reusing buffers for marshalling write requests in distributors (-distributor.write-requests-buffer-pooling-enabled)
Logging of requests that did not send any HTTP request: -server.http-log-closed-connections-without-response-enabled.
Ingester: track “owned series” and use owned series instead of in-memory series for tenant limits.
- -ingester.use-ingester-owned-series-for-limits
- -ingester.track-ingester-owned-series
- -ingester.owned-series-update-interval
Server
- PROXY protocol support
  - -server.proxy-protocol-enabled
- Cross-cluster validation support for gRPC and HTTP communication
  - -server.cluster-validation.label
  - -server.cluster-validation.grpc.enabled
  - -server.cluster-validation.grpc.soft-validation
  - -server.cluster-validation.http.enabled
  - -server.cluster-validation.http.soft-validation
  - -server.cluster-validation.http.excluded-paths
  - Requests with invalid cluster validation labels are tracked via the cortex_server_invalid_cluster_validation_label_requests_total metric.
gRPC clients
- Cross-cluster validation support for gRPC communication:
  - Assuming that a gRPC client configuration can be reached via -<grpc-client-config-path>, cluster validation label is configured via: -<grpc-client-config-path>.cluster-validation.label.
  - The cluster validation label of all gRPC clients can be configured via -common.client-cluster-validation.label.
  - Requests with invalid cluster validation labels are tracked via the cortex_client_invalid_cluster_validation_label_requests_total metric.
Common
- Instrument a fraction of pooled objects for references that outlive their lifetime.
  - Only implemented for objects embedding mimirpb.BufferHolder.
  - Flags:
    - -common.instrument-reference-leaks.percentage
    - -common.instrument-reference-leaks.before-reuse-period
    - -common.instrument-reference-leaks.max-inflight-instrumented-bytes
Preferred available zones for querying ingesters and store-gateways
- -querier.prefer-availability-zones
Memberlist zone-aware routing
- -memberlist.zone-aware-routing.enabled
- -memberlist.zone-aware-routing.instance-availability-zone
- -memberlist.zone-aware-routing.role
Memberlist rejoin custom seed nodes
- -memberlist.rejoin-seed-nodes
Jsonnet
- $._config.autoscaling_oom_protection_enabled controls whether to add extra KEDA ScaledObject trigger to prevent from down-scaling during OOM kills, if memory trigger is disabled
Configuring the gzip compression level used for compressed HTTP responses with -http.response-compression-level.

Deprecated features

Deprecated features are usable up until the release that indicates their removal. For details about what deprecated means, see Parameter lifecycle.

The following features or configuration parameters are currently deprecated and will be removed in a future release (to be announced):

Tracing configuration through Jaeger JAEGER_* environment variables and Jaeger tracing exposition protocol (deprecated since Mimir 2.17)
- Use OpenTelemetry configuration instead, as Jaeger supports OTLP ingestion natively
Rule group configuration file
- evaluation_delay field: use query_offset instead
The -store-gateway.sharding-ring.auto-forget-enabled is deprecated and will be removed in a future release. Set the -store-gateway.sharding-ring.auto-forget-unhealthy-periods flag to 0 to disable the auto-forget feature. Deprecated since Mimir 2.17.
The -distributor.otel-start-time-quiet-zero parameter no longer has any effect and will be removed in a future release. Deprecated since Mimir 3.0.
Postings for matchers cache size (number of entries) configuration:
- -blocks-storage.tsdb.head-postings-for-matchers-cache-size
- -blocks-storage.tsdb.block-postings-for-matchers-cache-size

The following features or configuration parameters were deprecated in Mimir 3.1 and will be removed in Mimir 3.3:

Considering cached results when calculating the number of processed samples reported in query stats (-query-frontend.cache-samples-processed-stats=true).
The -ingest-storage.kafka.write-clients CLI flag is deprecated and has no effect. Mimir always uses a single Kafka write client.

About Grafana Mimir tenant IDs

Wed, 03 Jun 2026 09:01:40 +0200

About Grafana Mimir tenant IDs

Within a Grafana Mimir cluster, the tenant ID is the unique identifier of a tenant. For information about how Grafana Mimir components use tenant IDs, refer to Authentication and authorization.

Restrictions

Tenant IDs must be less-than or equal-to 150 bytes or characters in length and can only include the following supported characters:

Alphanumeric characters
- 0-9
- a-z
- A-Z
Special characters
- Exclamation point (!)
- Hyphen (-)
- Underscore (_)
- Single period (.)
- Asterisk (*)
- Single quote (')
- Open parenthesis (()
- Close parenthesis ())

Note
For security reasons, . and .. aren’t valid tenant IDs.

Note
__mimir_cluster isn’t a valid tenant ID because Mimir uses the name internally.

All other characters, including slashes and whitespace, aren’t supported.

Configure Grafana Mimir high-availability deduplication

Wed, 03 Jun 2026 09:01:40 +0200

Configure Grafana Mimir high-availability deduplication

You can have more than one Prometheus instance that scrapes the same metrics for redundancy. Grafana Mimir already performs replication for redundancy, so you do not need to ingest the same data twice. In Grafana Mimir, you can deduplicate the data that you receive from HA pairs of Prometheus instances.

Assume that there are two teams, each running their own Prometheus instance, which monitors different services: Prometheus team-1 and Prometheus team-2. If the teams are running Prometheus HA pairs, the individual Prometheus instances would be team-1.a and team-1.b, and team-2.a and team-2.b.

Grafana Mimir only ingests from either team-1.a or team-1.b, and only from team-2.a or team-2.b. It does this by electing a leader replica for each Prometheus server. For example, in the case of team-1, the leader replica would be team-1.a. As long as team-1.a is the leader, the samples that team-1.b sends are dropped. And if Grafana Mimir does not see any new samples from team-1.a for a short period of time (30 seconds by default), it switches the leader to team-1.b.

If team-1.a goes down for more than 30 seconds, Grafana Mimir’s HA sample handling will have switched and elected team-1.b as the leader. The failure timeout ensures that too much data is not dropped before failover to the other replica.

Note
In a scenario where the default scrape period is 15 seconds, and the timeouts in Grafana Mimir are set to the default values, when a leader-election failover occurs, you’ll likely only lose a single scrape of data.

For any query using the rate() function, make the rate time interval at least four times that of the scrape period to account for any of these failover scenarios.

For example, with the default scrape period of 15 seconds, use a rate time-interval at least 1-minute.

Distributor high-availability (HA) tracker

The distributor includes a high-availability (HA) tracker.

The HA tracker deduplicates incoming samples based on a cluster and replica label expected on each incoming series. The cluster label uniquely identifies the cluster of redundant Prometheus servers for a given tenant. The replica label uniquely identifies the replica within the Prometheus cluster. Incoming samples are considered duplicated (and thus dropped) if they are received from any replica that is not the currently elected leader within a cluster.

If the HA tracker is enabled but incoming samples contain only one or none of the cluster and replica labels, these samples are accepted by default and never deduplicated.

Note: for performance reasons, the HA tracker only checks the cluster and replica label of the first series in the request to determine whether all series in the request should be deduplicated. This assumes that all series inside the request have the same cluster and replica labels, which is typically true when Prometheus is configured with external labels. Ensure this requirement is honored if you have a non-standard Prometheus setup (for example, you’re using Prometheus federation or have a metrics proxy in between).

If all series inside a single write request can’t be guaranteed to share the same cluster and replica labels, enable the experimental per-series HA deduplication via -distributor.ha-tracker.per-sample-dedupe (or the ha_tracker_per_sample_dedupe per-tenant limit). With this setting enabled, each timeseries in the request is evaluated independently, so non-elected replicas are dropped even when mixed with elected-replica or non-HA series in the same request.

Error responses

When the HA tracker drops samples, Mimir returns one of the following errors depending on the reason:

Replicas did not match: When samples are received from a non-elected replica, Mimir returns an HTTP 202 Accepted response with the message replicas did not match, rejecting sample: replica=<replica>, elected=<elected>. This indicates that the samples were successfully deduplicated and can be safely ignored.
Too many HA clusters: When the number of HA clusters for a tenant exceeds the configured limit, Mimir returns an HTTP 400 Bad Request response with the error ID err-mimir-tenant-too-many-ha-clusters. To adjust this limit, configure -distributor.ha-tracker.max-clusters or contact your service administrator.

Configuration

This section includes information about how to configure Prometheus and Grafana Mimir.

How to configure Prometheus

To configure Prometheus, set two identifiers for each Prometheus server, one for the cluster. For example, set team-1 or team-2, and one to identify the replica in the cluster, for example a or b. The easiest approach is to set external labels. The default labels are cluster and __replica__.

The following example shows how to set identifiers in Prometheus:

global:
  external_labels:
    cluster: prom-team1
    __replica__: replica1

and

global:
  external_labels:
    cluster: prom-team1
    __replica__: replica2

Note
The preceding labels are external labels and have nothing to do with remote_write configuration.

These two label names are configurable on a per-tenant basis within Grafana Mimir. For example, if the label name of one cluster is used by some workloads, set the label name of another cluster to something else that uniquely identifies the second cluster.

Set the replica label so that the value for each Prometheus cluster is unique in that cluster.

Note
Grafana Mimir drops this label when ingesting data, but preserves the cluster label. This way, your time series won’t change when replicas change.

How to configure Grafana Mimir

The minimal configuration required is as follows:

Enable the HA tracker.
Configure the HA tracker KV store.
Configure expected label names for each cluster and its replica.

Enable the HA tracker

To enable the HA tracker feature, set the -distributor.ha-tracker.enable=true CLI flag (or its YAML configuration option) in the distributor.

Next, decide whether you want to enable it for all tenants or just a subset of tenants. To enable it for all tenants, set -distributor.ha-tracker.enable-for-all-users=true. Alternatively, you can enable the HA tracker only on a per-tenant basis, keeping the default -distributor.ha-tracker.enable-for-all-users=false and overriding it on a per-tenant basis setting accept_ha_samples in the overrides section of the runtime configuration.

Configure the HA tracker KV store

The HA tracker requires a key-value (KV) store to coordinate which replica is currently elected. Starting from Mimir 3.0, memberlist is the recommended and default KV store backend for the HA tracker.

To migrate from Consul or etcd to memberlist without downtime, see Migrate HA tracker from Consul or etcd to memberlist.

The following CLI flags (and their respective YAML configuration options) are available for configuring the HA tracker KV store:

-distributor.ha-tracker.store: Backend storage to use (default: memberlist).
-memberlist.*: Memberlist client configuration. This is shared by multiple components.

The memberlist configuration is typically shared across multiple Mimir components (distributors, ingesters, etc.), so if you already have memberlist configured for hash ring synchronization, no additional configuration is required for the HA tracker.

Configure expected label names for each Prometheus cluster and replica

The HA tracker deduplicates incoming series that have cluster and replica labels. You can configure the name of these labels either globally or on a per-tenant basis.

Configure the default cluster and replica label names using the following CLI flags (or their respective YAML configuration options):

-distributor.ha-tracker.cluster: Name of the label whose value uniquely identifies a Prometheus HA cluster (defaults to cluster).
-distributor.ha-tracker.replica: Name of the label whose value uniquely identifies a Prometheus replica within the HA cluster (defaults to __replica__).

Note
The HA label names can be overridden on a per-tenant basis by setting ha_cluster_label and ha_replica_label in the overrides section of the runtime configuration.

Example configuration

The following configuration example snippet enables the HA tracker for all tenants via a YAML configuration file:

limits:
  accept_ha_samples: true

distributor:
  ha_tracker:
    enable_ha_tracker: true
    kvstore:
      store: memberlist

memberlist:
  # Memberlist configuration (typically shared with other components)
  join_members:
    - <IP_OR_DNS:PORT>
    - <IP_OR_DNS:PORT>

Note
If memberlist is already configured for other Mimir components, such as the hash ring, the HA tracker automatically uses that configuration. In most deployments, you don’t need any additional memberlist configuration.

For more information, see distributor. The HA tracker flags are prefixed with -distributor.ha-tracker.*.

Migrate HA tracker from Consul or etcd to memberlist without downtime

Wed, 03 Jun 2026 09:01:40 +0200

Migrate HA tracker from Consul or etcd to memberlist without downtime

Since Grafana Mimir 2.17, the HA tracker supports memberlist as a key-value (KV) store backend. Memberlist is the recommended KV store for the HA tracker.

Follow this guidance to migrate your HA tracker configuration from Consul or etcd to memberlist without any downtime.

Why migrate to memberlist

Memberlist eliminates the need to deploy and maintain a separate consistent KV store (Consul or etcd) for the HA tracker. Benefits include:

Simplified operations: No separate service to deploy, monitor, and maintain
Reduced infrastructure: One less component in your deployment
Built-in: Memberlist is already used by other Mimir components for hash rings and can be used for the HA tracker as well

Before you begin

Before starting this migration:

Ensure you’re running Grafana Mimir 2.17 or later
Have a running Mimir cluster with HA tracker currently using Consul or etcd

Migration steps

The migration uses Mimir’s multi KV store feature to transition from Consul or etcd to memberlist without downtime. The process involves:

Configuring a multi KV store with your current backend as primary and memberlist as secondary
Enabling mirroring to write to both stores
Switching memberlist to be the primary store
Removing the old backend

Configure multi KV store

Configure the HA tracker to use the multi KV store with your current backend, Consul or etcd, as the primary store and memberlist as the secondary store.

For Consul:

distributor:
  ha_tracker:
    enable_ha_tracker: true
    kvstore:
      store: multi
      multi:
        primary: consul
        secondary: memberlist
        # Mirroring is disabled by default, we'll enable it in Step 2
        mirror_enabled: false
      consul:
        # Your existing Consul configuration
        host: "consul.default.svc.cluster.local:8500"
      memberlist:
        # Memberlist configuration - typically shared with other components
        join_members:
          - mimir-gossip-ring.default.svc.cluster.local:7946

For etcd:

distributor:
  ha_tracker:
    enable_ha_tracker: true
    kvstore:
      store: multi
      multi:
        primary: etcd
        secondary: memberlist
        # Mirroring is disabled by default, we'll enable it in Step 2
        mirror_enabled: false
      etcd:
        # Your existing etcd configuration
        endpoints:
          - etcd.default.svc.cluster.local:2379
      memberlist:
        # Memberlist configuration - typically shared with other components
        join_members:
          - mimir-gossip-ring.default.svc.cluster.local:7946

Apply this configuration change. This step requires a rollout of all distributor instances.

After applying this step, all distributors expose the /memberlist admin page on their HTTP port, which you can use to check the health of the memberlist cluster.

Enable KV store mirroring

Enable mirroring to write the HA tracker state to both the primary (Consul or etcd) and secondary (memberlist) stores.

Update the configuration:

distributor:
  ha_tracker:
    enable_ha_tracker: true
    kvstore:
      store: multi
      multi:
        primary: consul # or etcd
        secondary: memberlist
        mirror_enabled: true # Changed in this step

Verification:

Monitor the following metrics to verify mirroring is working:

cortex_multikv_mirror_enabled: Shows which distributors have mirroring enabled. This should be set to 1 for all distributors.
rate(cortex_multikv_mirror_writes_total[$__rate_interval]): Rate of writes to secondary store, memberlist
rate(cortex_multikv_mirror_write_errors_total[$__rate_interval]): Rate of write errors. This should be 0.

After enabling mirroring, you should see HA tracker keys in the memberlist cluster information on the /memberlist admin page.

Switch to memberlist as the primary store

Switch the primary and secondary stores so that memberlist is the primary store.

Update the configuration:

distributor:
  ha_tracker:
    enable_ha_tracker: true
    kvstore:
      store: multi
      multi:
        primary: memberlist # Changed in this step
        secondary: consul # or etcd - changed in this step
        mirror_enabled: true

From this point on, distributors read from memberlist and mirror updates to Consul or etcd.

Verification:

Monitor cortex_multikv_primary_store to verify all distributors are using memberlist as the primary store. The store label should have the value memberlist.

Disable mirroring

Stop mirroring writes to Consul or etcd.

Update the configuration:

distributor:
  ha_tracker:
    enable_ha_tracker: true
    kvstore:
      store: multi
      multi:
        primary: memberlist
        secondary: consul # or etcd
        mirror_enabled: false # Changed in this step

Verification:

Monitor cortex_multikv_mirror_enabled to verify that all distributors have disabled mirroring. This value should be 0.

Remove multi KV store configuration

Configure the HA tracker to use memberlist directly, removing the multi KV store wrapper.

Update the configuration:

distributor:
  ha_tracker:
    enable_ha_tracker: true
    kvstore:
      store: memberlist # Changed in this step
      memberlist:
        join_members:
          - mimir-gossip-ring.default.svc.cluster.local:7946

Apply this configuration change. This step requires a rollout of all distributor instances.

After the rollout completes, distributors use memberlist directly for the HA tracker. The cortex_multikv_* metrics are no longer exposed.

Cleanup

You have successfully migrated the HA tracker from Consul or etcd to memberlist without downtime.

If you are no longer using Consul or etcd for any other purpose in your Mimir deployment, you can:

Remove the Consul or etcd deployment
Remove any Consul or etcd monitoring dashboards and alerts
Remove Consul or etcd client configuration from your Mimir configuration

Helm-specific guidance

If you’re using the mimir-distributed Helm chart, the migration steps are the same but use Helm values instead of direct YAML configuration. For example:

mimir:
  structuredConfig:
    distributor:
      ha_tracker:
        enable_ha_tracker: true
        kvstore:
          store: multi
          multi:
            primary: consul # or etcd
            secondary: memberlist
            mirror_enabled: false
          consul:
            host: "{{ .Release.Name }}-consul-server.{{ .Release.Namespace }}.svc.cluster.local:8500"
          # or for etcd:
          # etcd:
          #   endpoints:
          #     - "{{ .Release.Name }}-etcd.{{ .Release.Namespace }}.svc.cluster.local:2379"

About Grafana Mimir anonymous usage statistics reporting

Wed, 03 Jun 2026 09:01:40 +0200

About Grafana Mimir anonymous usage statistics reporting

Grafana Mimir includes a system that optionally and anonymously reports non-sensitive, non-personally identifiable information about the running Mimir cluster to a remote statistics server. Mimir maintainers use this anonymous information to learn more about how the open source community runs Mimir and what the Mimir team should focus on when working on the next features and documentation improvements.

The anonymous usage statistics reporting is enabled by default. You can opt-out setting the CLI flag -usage-stats.enabled=false or its respective YAML configuration option.

The statistics server

When usage statistics reporting is enabled, information is collected by a server that Grafana Labs runs. Statistics are collected at https://stats.grafana.org.

Which information is collected

When the usage statistics reporting is enabled, Grafana Mimir collects the following information:

Information about the Mimir cluster and version:
- A unique, randomly-generated Mimir cluster identifier, such as 3749b5e2-b727-4107-95ae-172abac27496.
- The timestamp when the anonymous usage statistics reporting was enabled for the first time, and the cluster identifier was created.
- The Mimir version, such as 2.3.0.
- The Mimir branch, revision, and Golang version that was used to build the binary.
- The installation mode used to deploy Mimir, such as helm.
Information about the environment where Mimir is running:
- The operating system, such as linux.
- The architecture, such as amd64.
- The Mimir memory utilization and number of goroutines.
- The CPU usage of the Mimir process.
Information about the Mimir configuration:
- The -target parameter value, such as all when running Mimir in monolithic mode.
- The -blocks-storage.backend value, such as s3.
- The -ingester.ring.replication-factor value, such as 3.
- The -ingester.ring.store value, such as memberlist.
- The minimum and maximum value of -ingester.out-of-order-time-window, which can be overridden on a per-tenant basis (the tenant ID is not shared).
Information about the Mimir cluster scale:
- Ingester:
  - The number of in-memory series.
  - The number of active series.
  - The number of tenants that have in-memory series.
  - The number of tenants that have out-of-order ingestion enabled.
  - The number of samples and exemplars ingested.
- Querier, where no information is tracked about the actual request or query:
  - The number of requests to queriers that are split by API endpoint type:
    - Remote read.
    - Instant query.
    - Range query.
    - Exemplars query.
    - Labels query.
    - Series query.
    - Metadata query.
    - Cardinality analysis query.

Note
Mimir maintainers commit to keeping the list of tracked information updated over time, and reporting any change both via the CHANGELOG and the release notes.

Disable the anonymous usage statistics reporting

If possible, we ask you to keep the usage reporting feature enabled and help us understand more about how the open source community runs Mimir. In case you want to opt-out from anonymous usage statistics reporting, set the CLI flag -usage-stats.enabled=false or change the following YAML configuration:

usage_stats:
  enabled: false

Configure Grafana Mimir object storage backend

Wed, 03 Jun 2026 09:01:40 +0200

Configure Grafana Mimir object storage backend

Grafana Mimir can use different object storage services to persist blocks containing the metrics data, as well as recording rules and Alertmanager state.

Mimir doesn’t create the configured storage bucket, you must create it yourself. The supported backends are:

Amazon S3 (and compatible implementations like MinIO)
Google Cloud Storage
Azure Blob Storage
Swift (OpenStack Object Storage)

Note
Like Amazon S3, the chosen object storage implementation must not create directories. Grafana Mimir doesn’t have any notion of object storage directories, and so will leave empty directories behind when removing blocks. For example, if you use Azure Blob Storage, you must disable hierarchical namespace.

To configure the storage retention on a per-tenant basis, set overrides in the runtime configuration:

Ruler and alertmanager support a local implementation, which is similar to filesystem in the way that it uses the local file system, but it is a read-only data source and can be used to provision state into those components.

Common configuration

To avoid repetition, you can use the common configuration and fill the common configuration block or by providing the -common.storage.* CLI flags.

To use environment variables in the configuration file, ensure that you enable expansion for the variables.

Note
Blocks storage can’t be located in the same path of the same bucket as the ruler and Alertmanager stores.

When using the common configuration, make blocks_storage use either a:

different bucket, overriding the common bucket name

storage prefix

Grafana Mimir will fail to start if you configure blocks storage to use the same bucket and storage prefix that the Alertmanager or ruler store uses.

Find examples of setting up the different object stores below:

Note
If you’re using a mixture of YAML files and CLI flags, pay attention to their precedence logic.

S3

common:
  storage:
    backend: s3
    s3:
      endpoint: s3.us-east-2.amazonaws.com
      region: us-east-2
      secret_access_key: "${AWS_SECRET_ACCESS_KEY}" # This is a secret injected via an environment variable
      access_key_id: "${AWS_ACCESS_KEY_ID}" # This is a secret injected via an environment variable

blocks_storage:
  s3:
    bucket_name: mimir-blocks

alertmanager_storage:
  s3:
    bucket_name: mimir-alertmanager

ruler_storage:
  s3:
    bucket_name: mimir-ruler

Note
Don’t prefix the endpoint value with your bucket name. For example, if your bucket is named “my-storage-bucket”, you cannot name the endpoint “my-storage-bucket.s3.amazonaws.com”.

GCS

common:
  storage:
    backend: gcs
    gcs:
      # This is an example to illustrate what the service account content should look like.
      # We recommend injecting the service_account via an environment variable instead.
      service_account: |
        {
          "type": "service_account",
          "project_id": "my-project",
          "private_key_id": "1234abc",
          "private_key": "-----BEGIN PRIVATE KEY-----\n\n-----END PRIVATE KEY-----\n",
          "client_email": "test@my-project.iam.gserviceaccount.com",
          "client_id": "5678",
          "auth_uri": "https://accounts.google.com/o/oauth2/auth",
          "token_uri": "https://oauth2.googleapis.com/token",
          "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
          "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/test%40my-project.iam.gserviceaccount.com"
        }

blocks_storage:
  gcs:
    bucket_name: mimir-blocks

alertmanager_storage:
  gcs:
    bucket_name: mimir-alertmanager

ruler_storage:
  gcs:
    bucket_name: mimir-ruler

Azure Blob Storage

You must disable hierarchical namespace, otherwise Grafana Mimir will leave empty directories behind when deleting blocks.

common:
  storage:
    backend: azure
    azure:
      account_key: "${AZURE_ACCOUNT_KEY}" # This is a secret injected via an environment variable
      account_name: mimirprod
      endpoint_suffix: "blob.core.windows.net"

blocks_storage:
  azure:
    container_name: mimir-blocks

alertmanager_storage:
  azure:
    container_name: mimir-alertmanager

ruler_storage:
  azure:
    container_name: mimir-ruler

Azure Workload Identity

Here is an example configuration for using Azure Workload Identity.

---
common:
  storage:
    backend: azure
    azure:
      account_name: mimirprod
      endpoint_suffix: "blob.core.windows.net"
blocks_storage:
  azure:
    container_name: mimir-blocks
alertmanager_storage:
  azure:
    container_name: mimir-alertmanager
ruler_storage:
  azure:
    container_name: mimir-ruler
serviceAccount:
  create: true
  name: mimir-storage
  annotation:
    "azure.workload.identity/use: "true"
    "azure.workload.identity/client-id": "${USER_ASSIGNED_IDENTITY_CLIENT_ID}"
  labels:
    "azure.workload.identity/use: "true"
global:
  podlabels:
    "azure.workload.identity/use: "true"

Note
Unlike with Tempo, federated tokens are not supported with Mimir.

OpenStack SWIFT

common:
  storage:
    backend: swift
    swift:
      auth_url: http://10.121.xx.xx:5000/v3
      username: mimir
      user_domain_name: Default
      password: "${OPENSTACK_API_KEY}" # This is a secret injected via an environment variable
      project_name: mimir-prod
      domain_name: Default

blocks_storage:
  swift:
    container_name: mimir-blocks

alertmanager_storage:
  swift:
    container_name: mimir-alertmanager

ruler_storage:
  swift:
    container_name: mimir-ruler

Configure Grafana Mimir metrics storage retention

Wed, 03 Jun 2026 09:01:40 +0200

Configure Grafana Mimir metrics storage retention

Grafana Mimir stores metrics in object storage.

By default, metrics that are stored in the object storage are never deleted, and the storage utilization will increase over time. You can configure the object storage retention to automatically delete all metrics data older than the configured period.

Configure the storage retention

The compactor is the Mimir component that is responsible for enforcing the storage retention. To configure the storage retention, set the CLI flag -compactor.blocks-retention-period or change the following YAML configuration:

limits:
  # Delete from storage metrics data older than 1 year.
  compactor_blocks_retention_period: 1y

To configure the storage retention on a per-tenant basis, set overrides in the runtime configuration:

overrides:
  tenant1:
    # Delete from storage tenant1's metrics data older than 1 year.
    compactor_blocks_retention_period: 1y
  tenant2:
    # Delete from storage tenant2's metrics data older than 2 years.
    compactor_blocks_retention_period: 2y
  tenant3:
    # Disable retention for tenant3's metrics (never delete its data).
    compactor_blocks_retention_period: 0

Per-series retention

Grafana Mimir doesn’t support per-series deletion and retention, nor does it support Prometheus’ Delete series API.

Configure TSDB block upload

Wed, 03 Jun 2026 09:01:40 +0200

Configure TSDB block upload

Grafana Mimir supports uploading of historic TSDB blocks, sourced from Prometheus, Cortex, or even other Grafana Mimir installations. Upload from Thanos is currently not supported; for more information, see Known limitations of TSDB block upload.

To make performing block upload simple, we’ve built support for it into Mimir’s CLI tool, Mimirtool. For more information, refer to mimirtool backfill.

Block upload is still considered experimental and is therefore disabled by default. You can enable it via the -compactor.block-upload-enabled CLI flag, or via the corresponding limits.compactor_block_upload_enabled configuration parameter:

limits:
  # Enable TSDB block upload
  compactor_block_upload_enabled: true

Validation of blocks

Before uploading block data starts, Grafana Mimir performs the following checks on the meta.json file:

Only TSDB “v1” blocks are supported. This is the format used by Prometheus v2, Grafana Mimir, and Thanos.
Blocks with invalid MinTime or MaxTime are rejected (negative values or MaxTime < MinTime).
Blocks where MinTime or MaxTime is in the future are rejected.
Blocks that are outside of the retention period are rejected.
Blocks covering a time range larger than the maximum compaction range (-compactor.block-ranges option, maximum defaults to 24h) are rejected.
Blocks that cross the boundary of the maximum compaction range are rejected. For example, if the largest compaction range is 24 hours, blocks that start before midnight and finish after midnight are rejected.
Blocks with Thanos downsampling configuration are rejected.
Blocks that are larger than the compactor_block_upload_max_block_size_bytes (per-tenant override) setting are rejected.
Blocks with “external labels” (Thanos feature) are rejected. Some Mimir-specific labels are allowed.

After you upload the block index and chunks, Grafana Mimir performs additional block validation to verify that blocks are well-formed and that they won’t cause problems for Mimir’s operation. You can disable these “full block” validations through the compactor_block_upload_validation_enabled per-tenant override. To disable chunks validation while keeping index-validation, use the compactor_block_upload_verify_chunks per-tenant override instead.

Enable TSDB block upload per tenant

If your Grafana Mimir has multi-tenancy enabled, you can still use the preceding method to enable TSDB block upload for all tenants. If instead you wish to enable it per tenant, you can use the runtime configuration to set a per-tenant override:

Enable runtime configuration.
Add an override for the tenant that should have TSDB block upload enabled:

overrides:
  tenant1:
    compactor_block_upload_enabled: true

Known limitations of TSDB block upload

Thanos blocks cannot be uploaded

Because Thanos blocks contain unsupported labels among their metadata, they cannot be uploaded.

For information about limitations that relate to importing blocks from Thanos as well as existing workarounds, refer to Migrating from Thanos or Prometheus to Grafana Mimir.

The results-cache needs flushing

Grafana Mimir caches samples older than 10 minute (configurable via -query-frontend.max-cache-freshness) in the range query results. After uploading blocks however, queries may return different results – because new data was uploaded. This means that cached results may be wrong. To fix the cache results, Mimir operator can manually flush the results cache. Possible alternative is to decrease time-to-live period for cache results from default 7 days to shorter period, for example 6 hours, by using -query-frontend.results-cache-ttl command line option (or per tenant). This will guarantee that query results will use backfilled data at most after this period.

Blocks that are too new will not be queryable until later

When queriers receive a query for a given [start, end] period, they consult this period to decide whether to read data from storage, ingesters, or both. Say -querier.query-store-after is set to 12h. It means that a query [now-13h, now] will read data from storage. But a query [now-5h, now] will not. If a user uploads blocks that are “too new”, the querier may not query them, because it is configured to only query ingesters for “fresh” time ranges.

Configure out-of-order samples ingestion

Wed, 03 Jun 2026 09:01:40 +0200

Configure out-of-order samples ingestion

If you have out-of-order samples, due to the nature of your architecture or the system that you are observing, then you can configure Grafana Mimir to set an out-of-order time-window threshold for how old samples can be ingested.

Mimir allows you to ingest out-of-order samples. As a result, no sample is dropped if it is within the configured time window.

Configure out-of-order samples ingestion instance-wide

To configure Mimir to accept out-of-order samples, see the following configuration snippet:

limits:
  # Allow ingestion of out-of-order samples up to 5 minutes since the latest received sample for the series.
  out_of_order_time_window: 5m

Configure out-of-order samples per tenant

If your Mimir has multitenancy enabled, you can still use the preceding method to set a default out-of-order time window threshold for all tenants. If a particular tenant needs a custom threshold, you can use the runtime configuration to set a per-tenant override.

Enable runtime configuration.
Add an override for the tenant that needs a custom out-of-order time window:

overrides:
  tenant1:
    out_of_order_time_window: 2h
  tenant2:
    out_of_order_time_window: 30m

Setting out_of_order_time_window to 0s disables the out-of-order ingestion while you can still continue to query the out-of-order samples ingested till now.

Query caching with out-of-order ingestion enabled

Once a query has been cached, out-of-order samples that get ingested later can potentially change those query results.

To help with that, we recommended you to pass the same limits config to the query-frontend component. It will set a lower TTL (Time-To-Live) of 10 minutes for any query cache entry that falls within the ingestion window for out-of-order samples (i.e. timestamps after now minus out_of_order_time_window).

If you do not want to cache any query for this window at all, you can set -query-frontend.max-cache-freshness to match the out_of_order_time_window so that you don’t cache queries for the time window where you still expect samples to arrive. Doing so can increase the load on your Mimir cluster depending on query characteristics.

Recording rules when out-of-order ingestion is enabled

Similar to the problem above with query caching, the samples recorded via the recording rules can get outdated with new out-of-order samples being ingested. So you should expect some difference in results if you happen to run the raw query of the recording rule. The difference highly depends on your out-of-order ingestion pattern.

If you happen to have a shorter out_of_order_time_window, say less than 10 minutes, then you can use -ruler.evaluation-delay-duration to delay your rule evaluation up to that time.

Understand out-of-order

Previously, Mimir and Prometheus TSDB had a couple of rules over what timestamps are accepted.

The moment that a new series sample arrives, Mimir needs to determine if the series already exists, and whether or not the sample is too old:

If the series exists within the Head block of the TSDB, the incoming sample must have a newer timestamp than the latest sample that is stored for the series. Otherwise, the ingesters consider it to be out-of-order.
If the series does not exist, then the sample has to be within bounds, which go back 1 hour from TSDB’s head-block max time (when using 2 hour block range). If it fails to be within bounds, then the ingesters consider it to be out-of-bounds.

Out-of-order ingestion helps address both issues.

Note
If you’re writing metrics using Prometheus remote write or Grafana Alloy, then you shouldn’t expect out-of-order samples.

Prometheus and Grafana Alloy guarantee that samples are written in-order for the same series.

Configure owned series tracking

Wed, 03 Jun 2026 09:01:40 +0200

Configure owned series tracking

Note
Owned series tracking is an experimental feature in Grafana Mimir versions 2.12 and later.

In a distributed Grafana Mimir deployment, distributors use consistent hashing to determine which series should be routed to which ingester replicas or Kafka partitions. Because stale in-memory series in ingesters are only cleaned up during TSDB head compaction, the count of in-memory series is not suitable as a metric to drive automated ingester scaling; when a new replica is added, the in-memory series per ingester doesn’t drop until the next head compaction, and the scaler has no feedback about the impact of the scaling operation.

Additionally, since each ingester enforces only a fraction of a tenant’s in-memory series limits locally, increasing the ingester count, or a tenant’s shuffle-shard size, can result in ingesters rejecting samples due to exceeded local limits, even if the global limit isn’t breached. For example:

Consider a tenant with a global series limit of 150k with series spread across 3 ingesters; the local limit on each ingester is 50k in-memory series.
The tenant is sending ~90k series, so ~30k in-memory series per ingester.
Now, the tenant is sharded across 6 ingesters, either by adding ingesters or increasing the tenant’s shuffle-shard size.
The local limit on each ingester drops to 25k; the tenant’s 30k series on the original 3 ingesters exceed the new local limit, and those ingesters immediately start to discard samples.

Owned series tracking addresses these problems by counting, per tenant, the subset of in-memory series that hash to a particular ingester or partition, based on the current ring state and shard configuration. When you enable owned series tracking and use owned series for series limits, Mimir ingesters enforce limits against a tenant’s owned series count rather than the full count of in-memory series.

How owned series tracking works

Note
In ingest storage architecture, tokens are assigned to partitions rather than ingesters.

Ingesters are assigned tokens in a hash ring. For each token T assigned to an ingester, that ingester owns the range of hash values from the preceding assigned token through T-1. The preceding token can belong to any ingester in the ring. When shuffle-sharding is enabled, the token ranges owned by an ingester are different for each tenant, depending on which ingesters are part of the tenant’s sub-ring.

A series is owned by an ingester if the hash of its labels and tenant ID falls within a token range owned by that ingester. New series written to an ingester are automatically counted as owned by the ingester, since the distributors use the same algorithm to route series to ingesters.

Every ingester.owned-series-update-interval, each ingester’s owned series service iterates over each tenant and checks whether their owned series should be recomputed. Recomputation triggers are:

Shard size changes
Local series limit changes
Token ranges changes
Compaction events

If any of these conditions trigger an owned series recomputation, the ingester iterates over all of the series in the tenant’s TSDB and compares the series hash against the owned token ranges. If the hash falls within one of the token ranges, it is counted as still owned by the ingester; series with hashes that fall outside of the ranges are not counted, but remain in-memory until compaction.

Because owned series are recomputed shortly after a change in the ring state, for instance, adding new ingesters, the owned series count is suitable to provide an autoscaler feedback about the impact of a scaling operation.

Additionally, if the recomputation is triggered by a series limit decrease, the owned series are recomputed before the new series limit takes effect. Series limit increases take effect immediately. This allows graceful handling of this situation, without dropping samples while the owned series are being recomputed.

Before you begin

Note
These steps are only required to use owned series tracking in classic architecture. They are not required with ingest storage architecture because series replication is handled by the Kafka partitions, rather than writing to multiple ingesters. However, zone-aware replication of ingesters is still recommended for high availability.

Due to the way token ranges are calculated, using owned series tracking requires you to complete the following tasks first:

Enable zone-aware replication.
Set the replication factor equal to the number of zones.

Configure owned series tracking and limits

Enable owned series tracking on ingesters. For example:

# Enable tracking of owned series based on ring state
-ingester.track-ingester-owned-series=true

# How often to check whether owned series need to be recomputed
-ingester.owned-series-update-interval=15s

Verify that owned series are being tracked per tenant by checking for cortex_ingester_owned_series metrics
Use owned series, rather than in-memory series, to enforce per-tenant series limits. For example:
sh
```
-ingester.use-ingester-owned-series-for-limits=true
```

For more information, refer to ingester in Grafana Mimir configuration parameters.

Monitor owned series

Ingesters expose the following metrics related to owned series:

Metric name	Description
`cortex_ingester_owned_series`	Number of currently owned series per tenant
`cortex_ingester_owned_target_info_series`	Number of currently owned `target_info` series per tenant
`cortex_ingester_owned_series_check_duration_seconds`	How long the owned series check takes across all tenants

Configure native histograms

Wed, 03 Jun 2026 09:01:40 +0200

Configure native histograms

You can configure native histograms ingestion via the Prometheus remote write API endpoint globally or per tenant.

Note
To enable support for querying native histograms together with Grafana Mimir query sharding, make sure that the flag -query-frontend.query-result-response-format is set to its default protobuf value on query frontends.

Configure native histograms globally

To enable ingesting Prometheus native histograms over the remote write API endpoint for all tenants, set the flag -ingester.native-histograms-ingestion-enabled=true on the ingesters.

To limit the number of native histogram buckets per sample, set the -validation.max-native-histogram-buckets flag on the distributors. The recommended value is 160 which is the default in the OpenTelemetry SDK for exponential histograms. Exponential histograms in OpenTelemetry are a similar concept to Prometheus native histograms. At the time of ingestion, samples with more buckets than the limit will be scaled down, meaning that the resolution will be reduced and buckets will be merged until either the number of buckets is under the limit or the minimal resolution is reached. The behavior can be changed to dropping such samples by setting the -validation.reduce-native-histogram-over-max-buckets option to false.

Configure native histograms per tenant

To enable ingesting Prometheus native histograms over the remote write API for a tenant, set the native_histograms_ingestion_enabled runtime value to true.

To limit the number of native histogram buckets per sample for a tenant, set the max_native_histogram_buckets runtime value. The recommended value is 160 which is the default in the OpenTelemetry SDK for exponential histograms. Exponential histograms in OpenTelemetry are a similar concept to Prometheus native histograms. At the time of ingestion, samples with more buckets than the limit will be scaled down, meaning that the resolution will be reduced and buckets will be merged until either the number of buckets is under the limit or the minimal resolution is reached. The behavior can be changed to dropping such samples by setting the -validation.reduce-native-histogram-over-max-buckets option to false.

overrides:
  tenant1:
    native_histograms_ingestion_enabled: true
    max_native_histogram_buckets: 160

To learn more about sending native histograms to Mimir or Grafana Cloud Metrics via Grafana Alloy or Prometheus, refer to Send native histograms to Mimir.

Configure the OpenTelemetry Collector to write metrics into Mimir

Wed, 03 Jun 2026 09:01:40 +0200

Configure the OpenTelemetry Collector to write metrics into Mimir

Note
To send OpenTelemetry data to Grafana Cloud, refer to Send data using OpenTelemetry Protocol (OTLP).

When using the OpenTelemetry Collector, you can use the OpenTelemetry protocol (OTLP) or the Prometheus remote write protocol to write metrics into Mimir. It’s recommended that you use the OpenTelemetry protocol.

Use the OpenTelemetry protocol

Mimir supports native OTLP over HTTP. To configure the collector to use the OTLP interface, use the otlphttp exporter and the native Mimir endpoint. For example:

exporters:
  otlphttp:
    endpoint: http://<mimir-endpoint>/otlp

Then, enable it in the service.pipelines block:

service:
  pipelines:
    metrics:
      receivers: [...]
      processors: [...]
      exporters: [..., otlphttp]

If you want to authenticate using basic auth, use the basicauth extension. For example:

extensions:
  basicauth/otlp:
    client_auth:
      username: username
      password: password

exporters:
  otlphttp:
    auth:
      authenticator: basicauth/otlp
    endpoint: http://<mimir-endpoint>/otlp

service:
  extensions: [basicauth/otlp]
  pipelines:
    metrics:
      receivers: [...]
      processors: [...]
      exporters: [..., otlphttp]

Use the Prometheus remote write protocol

To use the Prometheus remote write protocol to send metrics into Mimir, use the prometheusremotewrite exporter in the Collector and the native Mimir endpoint.

In the exporters section, add:

exporters:
  prometheusremotewrite:
    endpoint: http://<mimir-endpoint>/api/v1/push

Then, enable it in the service.pipelines block:

service:
  pipelines:
    metrics:
      receivers: [...]
      processors: [...]
      exporters: [..., prometheusremotewrite]

If you want to authenticate using basic auth, use the basicauth extension. For example:

extensions:
  basicauth/prw:
    client_auth:
      username: username
      password: password

exporters:
  prometheusremotewrite:
    auth:
      authenticator: basicauth/prw
    endpoint: http://<mimir-endpoint>/api/v1/push

service:
  extensions: [basicauth/prw]
  pipelines:
    metrics:
      receivers: [...]
      processors: [...]
      exporters: [..., prometheusremotewrite]

Work with default OpenTelemetry labels

OpenTelemetry metrics use resource attributes to describe the set of characteristics associated with a given resource, or entity, producing telemetry data. For example, a host resource might have multiple attributes, including an ID, an image, and a type.

To optimize the storage of and ability to query this data, you can use the -distributor.otel-promote-resource-attributes option to configure Mimir to promote specified OTel resource attributes to labels at the time of ingestion.

Note
The -distributor.otel-promote-resource-attributes option is an experimental feature in Grafana Mimir.

Mimir stores additional OTel resource attributes in a separate series called target_info, which you can query using a join query or the Prometheus info() function. Refer to Functions in the Prometheus documentation for more information.

To learn more about OpenTelemetry resource attributes, refer to Resources in the OpenTelemetry documentation.

To learn more about ingesting OpenTelemetry data in Grafana Cloud, including how Grafana Cloud promotes OpenTelemetry resouce attributes to labels during ingestion, refer to OTLP: OpenTelemetry Protocol format considerations.

Format considerations

We follow the official OTLP Metric points to Prometheus specification.

By default, Grafana Mimir does not accept OpenTelemetry Exponential Histogram metrics. For Grafana Mimir to accept them, ingestion of Prometheus Native Histogram metrics must first be enabled following the instructions in Configure native histogram ingestion. After this is done, Grafana Mimir will accept OpenTelemetry Exponential Histograms, and convert them into Prometheus Native Histograms following the conventions described in the Exponential Histograms specification.

You might experience the following common issues:

Dots (.) are converted to _

Prometheus metrics do not support . and - characters in metric or label names. Prometheus converts these characters to _.

For example:

requests.duration{http.status_code=500, cloud.region=us-central1} in OTLP

requests_duration{http_status_code=”500”, cloud_region=”us-central1”} in Prometheus
Resource attributes are added to the target_info metric.

However, <service.namespace>/<service.name> or <service.name> (if the namespace is empty), is added as the label job, and service.instance.id is added as the label instance to every metric.

For details, see the OpenTelemetry Resource Attributes specification.

Configure the Grafana Mimir Kafka backend

Wed, 03 Jun 2026 09:57:20 +0200

Configure the Grafana Mimir Kafka backend

Grafana Mimir supports using Kafka as the first layer of ingestion in the ingest storage architecture. This configuration allows for scalable, decoupled ingestion that separates write and read paths to improve performance and resilience.

Starting with Mimir 3.0, ingest storage is the preferred and stable architecture for running Mimir.

Configure ingest storage

Set the following configuration flags to enable Mimir to use ingest storage through a Kafka backend:

-ingest-storage.enabled=true
You must explicitly enable the ingest storage architecture in all Mimir components.
-ingest-storage.kafka.address=<host:port>[,<host:port>...]
The <host:port> is a Kafka seed broker address used to bootstrap the connection. You can configure a comma-separated list of seed broker addresses for higher bootstrap availability.
-ingest-storage.kafka.topic=<name>
The <name> is the name of the Kafka topic that is used for ingesting data.
-ingest-storage.kafka.auto-create-topic-default-partitions=<number>
If the configured topic doesn’t exist in the Kafka backend, the Mimir components, either consumers or producers, create the topic on first access. The <number> parameter sets the number of partitions to create when the topic is automatically created. The number of partitions must be at least the number of ingesters in one zone.

Additionally, you can use these recommended configuration options when running Mimir with ingest storage architecture:

-distributor.remote-timeout=5s
Use this setting to increase the default remote write timeout. This is recommended for writing to Kafka, because pushing to Kafka-compatible backends might be slower than writing directly to ingesters.

Refer to Mimir configuration parameters for detailed descriptions of all available configuration options.

Different Kafka backend implementations

Some Kafka-compatible implementations have different behavior for the Kafka API. To set up Mimir to work with different Kafka backends, you need to configure some parameters. Here are the Kafka flavors and additional configurations needed to set them up in Mimir.

Apache Kafka

In your Kafka broker configuration file (for example, server.properties), set the following property to support the default Mimir record size:

message.max.bytes=16000000

Mimir’s default -ingest-storage.kafka.producer-max-record-size-bytes is approximately 15.2 MB. Apache Kafka’s default message.max.bytes is 1 MB. Increase Kafka’s message.max.bytes to at least 16000000 to match Mimir’s maximum batch size; otherwise, Kafka rejects records larger than 1 MB.

To configure the limit at the topic level instead of the broker level, run:

bin/kafka-configs.sh --bootstrap-server <host:port> \
  --alter --entity-type topics --entity-name <topic-name> \
  --add-config max.message.bytes=16000000

Confluent Kafka

In your Kafka broker configuration file (for example, server.properties), set the following property to support the default Mimir record size:

message.max.bytes=16000000

Mimir’s default -ingest-storage.kafka.producer-max-record-size-bytes is approximately 15.2 MB. Confluent Kafka’s default message.max.bytes is 1 MB. Increase Kafka’s message.max.bytes to at least 16000000 to match Mimir’s maximum batch size; otherwise, Kafka rejects records larger than 1 MB.

To configure the limit at the topic level instead of the broker level, run:

bin/kafka-configs.sh --bootstrap-server <host:port> \
  --alter --entity-type topics --entity-name <topic-name> \
  --add-config max.message.bytes=16000000

Warpstream

Configure the following CLI flags or their YAML equivalent.

-ingest-storage.kafka.use-compressed-bytes-as-fetch-max-bytes=false

Authentication

Mimir supports multiple ways of authenticating with a Kafka cluster. Most methods use the Simple Authentication and Security Layer (SASL) framework, which supports the following mechanisms:

PLAIN: Authenticate with a username and password.
SCRAM-SHA-256 and SCRAM-SHA-512: Authenticate with a username and password using Salted Challenge Response Authentication Mechanism (SCRAM).
OAUTHBEARER: Authenticate with an OAuth 2.0 bearer token.
AWS_MSK_IAM: Authenticate to Amazon Managed Streaming for Apache Kafka (Amazon MSK) using AWS Identity and Access Management (IAM).

You can also connect to Kafka over Transport Layer Security (TLS), including mutual TLS (mTLS) for certificate-based authentication.

Username and password (PLAIN, SCRAM)

Set the following configuration flags to authenticate with a username and password:

-ingest-storage.kafka.sasl-mechanism
The SASL mechanism used to authenticate to Kafka. Set to SCRAM-SHA-256, SCRAM-SHA-512, or PLAIN. The default value is PLAIN.
-ingest-storage.kafka.sasl-username
The username used to authenticate to Kafka.
-ingest-storage.kafka.sasl-password
The password used to authenticate to Kafka.

To enable SASL, configure both the username and the password. For backwards compatibility, the PLAIN mechanism with no username and no password disables SASL.

OAUTHBEARER or AWS_MSK_IAM

Set the following configuration flags to authenticate with OAuth or MSK IAM:

-ingest-storage.kafka.sasl-mechanism
Set to OAUTHBEARER or AWS_MSK_IAM.

Both mechanisms share common patterns: either statically, with values set at startup and then don’t change, or dynamically, through a configuration file or an HTTP callback that are checked anew every time reauthentication is required.

With static configuration

You can set authentication configuration directly as configuration flags. The flags that each mechanism requires are listed in the following sections.

Note
Mimir components that connect to Kafka must be restarted whenever reauthentication is required.

With path to configuration file

Set the following configuration flag to specify the path to a file that contains a JSON object configuring authentication credentials:

-ingest-storage.kafka.sasl-<MECHANISM>-file-path
Replace <MECHANISM> with oauthbearer or msk-iam.

Refer to the following sections for the shape that the JSON object in the file must take.

This file is opened and read anew whenever reauthentication is required. Mimir doesn’t signal when this happens, so you must handle the authentication credentials lifetime and file updates out-of-band. Alternatively, you can configure an HTTP callback, which receives an HTTP request whenever reauthentication is required.

With HTTP callback

Set the following configuration flag to specify the path to a Unix domain socket:

-ingest-storage.kafka.sasl-<MECHANISM>-http-socket-path
Replace <MECHANISM> with oauthbearer or msk-iam.

You can also configure the timeout for the HTTP request:

-ingest-storage.kafka.sasl-<MECHANISM>-http-socket-timeout
The timeout for requesting credentials from the HTTP socket. Replace <MECHANISM> with oauthbearer or msk-iam. The default value is 10s.

A server must be listening for connections on this Unix domain socket. On receiving an HTTP GET request to path /, the server must respond with status 200 OK and a JSON object containing the authentication configuration.

Refer to the following sections for the shape that the JSON object must take.

Using an HTTP callback has some important benefits compared to the static configuration and the file-based approaches:

Because Mimir issues an HTTP request whenever reauthentication is required, you don’t need to manage authentication lifetime out-of-band.
Authentication credentials don’t need to be persisted. They can be obtained on-the-fly and sent back to Mimir for the lifespan of a single HTTP request.

OAUTHBEARER configuration details

For static configuration, set the following configuration flags:

-ingest-storage.kafka.sasl-oauthbearer-token
-ingest-storage.kafka.sasl-oauthbearer-zid
-ingest-storage.kafka.sasl-oauthbearer-extensions (optional)

For dynamic configuration (file or HTTP callback), set a JSON object with the following shape:

{
  "token": "<token>",
  "zid": "<authorization ID>",
  "extensions": {
    "<key>": "<value>"
  } // (optional)
}

Only token is required.

MSK_IAM configuration details

For static configuration, set the following configuration flags:

-ingest-storage.kafka.sasl-msk-iam-access-key
-ingest-storage.kafka.sasl-msk-iam-secret-key
-ingest-storage.kafka.sasl-msk-iam-session-token (optional)
-ingest-storage.kafka.sasl-msk-iam-user-agent (optional)

For dynamic configuration (file or HTTP callback), set a JSON object with the following shape:

{
  "AccessKey": "<access key ID>",
  "SecretKey": "<secret access key>",
  "SessionToken": "<session token>", // (optional)
  "UserAgent": "<user agent>" // (optional)
}

TLS / mTLS

Set the following configuration parameter to enable connecting to the Kafka cluster over TLS:

-ingest-storage.kafka.tls-enabled=true

For mutual authentication (mTLS), set the following additional flags:

-ingest-storage.kafka.tls-ca-path
Path to the CA certificates used to validate the server certificate. If you don’t set this flag, Mimir uses the host’s root CA certificates.
-ingest-storage.kafka.tls-cert-path
Path to the client certificate used to authenticate with the server. Requires the key path to also be configured.
-ingest-storage.kafka.tls-key-path
Path to the key for the client certificate. Requires the client certificate to also be configured.

You can also set the following optional flags to customize the TLS connection:

-ingest-storage.kafka.tls-server-name
Override the expected name on the server certificate.
-ingest-storage.kafka.tls-insecure-skip-verify
Skip validating the server certificate. This option is insecure and is intended only for testing.

Refer to Mimir configuration parameters for details and additional options, such as -ingest-storage.kafka.tls-cipher-suites and -ingest-storage.kafka.tls-min-version.

Grafana Mimir configuration parameters

Wed, 03 Jun 2026 09:01:40 +0200

Grafana Mimir configuration parameters

You can configure Grafana Mimir by using a YAML file or via command-line flags that represent configuration parameters. To specify the YAML file, use the -config.file command-line option. If you specify both the command-line flags and YAML configuration parameters, the command-line flags take precedence over values in a YAML file.

To see the current configuration of any component, go to the /config HTTP API endpoint. Passwords are filtered out of this endpoint.

Parameters are written in YAML format, and brackets indicate that a parameter is optional.

Generic placeholders

<boolean>: a boolean that can take the values true or false
<int>: any integer matching the regular expression [1-9]+[0-9]*
<duration>: a duration matching the regular expression [0-9]+(ns|us|µs|ms|s|m|h|d|w|y) where y = 365 days
<string>: a string
<url>: a URL
<filepath>: a string containing an absolute or relative path and filename to a file on disk
<prefix>: a CLI flag prefix based on the context (look at the parent configuration block to see which CLI flags prefix should be used)
<relabel_config>: a Prometheus relabeling configuration
<time>: a timestamp, with available formats:
- 2006-01-20 (midnight, local timezone)
- 2006-01-20T15:04 (local timezone)
- RFC 3339 formats: 2006-01-20T15:04:05Z (UTC) or 2006-01-20T15:04:05+07:00 (explicit timezone)

Parameter categories

In order to simplify Mimir configuration, we categorize parameters by maturity level and intended use:

Basic parameters are ones that we expect the majority of users to modify. Examples of basic parameters are object store credentials and other dependency connection information. These parameters will generally remain stable for long periods of time, and they focus on user goals. In this reference, any parameters that are not marked as (advanced) or (experimental) are in the basic category.
Advanced parameters are ones that few users will change from their default values. There are strong use cases for altered values. In this reference, advanced-category parameters include (advanced) at the beginning of their description.
Experimental parameters are for new and experimental features. These parameters permit early adopters and Mimir developers to gain confidence with new features. In this reference, experimental-category parameters include (experimental) at the beginning of their description.

Parameter lifecycle

When new parameters are added, they can be introduced as basic, advanced, or experimental.

Experimental parameters will remain experimental until they are either made stable or removed. Parameters that are made stable will be classified as either basic or advanced. We aim to make this decision on an experimental parameter within 6 months of its initial release, but this decision may take longer depending on what we discover during testing, or if upstream dependencies (e.g., Prometheus) of our code changes.

If we decide to eliminate a basic or advanced parameter, we will first mark it deprecated. After two more minor releases, a deprecated flag will be removed entirely. Use the metric deprecated_flags_inuse_total to determine whether you’re using deprecated flags. A configuration parameter is in maintenance and usable as expected between its deprecation and removal. If you configure Mimir with a removed parameter, Mimir will fail to start.

Use environment variables in the configuration

You can use environment variable references in the YAML configuration file to set values that need to be configurable during deployment. To do this, pass -config.expand-env=true on the command line and use ${VAR}, where VAR is the name of the environment variable.

Each variable reference is replaced at startup by the value of the environment variable. The replacement is case-sensitive and occurs before the YAML file is parsed. References to undefined variables are replaced by empty strings unless you specify a default value or custom error text.

To specify a default value, use ${VAR:default_value}, where default_value is the value to use if the environment variable is undefined.

Configuration parameters

# Comma-separated list of components to include in the instantiated process. The
# default value 'all' includes all components that are required to form a
# functional Grafana Mimir instance in single-binary mode. Use the '-modules'
# command line flag to get a list of available components, and to see which
# components are included with 'all'.
# CLI flag: -target
[target: <string> | default = "all"]

# When set to true, incoming HTTP requests must specify tenant ID in HTTP
# X-Scope-OrgId header. When set to false, tenant ID from -auth.no-auth-tenant
# is used instead.
# CLI flag: -auth.multitenancy-enabled
[multitenancy_enabled: <boolean> | default = true]

# (advanced) Tenant ID to use when multitenancy is disabled.
# CLI flag: -auth.no-auth-tenant
[no_auth_tenant: <string> | default = "anonymous"]

# (advanced) How long to wait between SIGTERM and shutdown. After receiving
# SIGTERM, Mimir will report not-ready status via /ready endpoint.
# CLI flag: -shutdown-delay
[shutdown_delay: <duration> | default = 0s]

# (experimental) Maximum number of groups allowed per user by which specified
# distributor and ingester metrics can be further separated.
# CLI flag: -max-separate-metrics-groups-per-user
[max_separate_metrics_groups_per_user: <int> | default = 1000]

# (advanced) Set to true to enable all Go runtime metrics, such as go_sched_*
# and go_memstats_*.
# CLI flag: -enable-go-runtime-metrics
[enable_go_runtime_metrics: <boolean> | default = false]

api:
  # (advanced) Allows to skip label name validation via
  # X-Mimir-SkipLabelNameValidation header on the http write path. Use with
  # caution as it breaks PromQL. Allowing this for external clients allows any
  # client to send invalid label names. After enabling it, requests with a
  # specific HTTP header set to true will not have label names validated.
  # CLI flag: -api.skip-label-name-validation-header-enabled
  [skip_label_name_validation_header_enabled: <boolean> | default = false]

  # (advanced) Allows to disable enforcement of the label count limit
  # "max_label_names_per_series" via X-Mimir-SkipLabelCountValidation header on
  # the http write path. Allowing this for external clients allows any client to
  # send invalid label counts. After enabling it, requests with a specific HTTP
  # header set to true will not have label counts validated.
  # CLI flag: -api.skip-label-count-validation-header-enabled
  [skip_label_count_validation_header_enabled: <boolean> | default = false]

  # (experimental) Allows controlling OTLP metric name suffix addition and
  # translation strategy via X-Mimir-OTLP-AddSuffixes and
  # X-Mimir-OTLP-TranslationStrategy headers on the OTLP push path. Not
  # recommended for general use.
  # CLI flag: -api.otlp-translation-headers-enabled
  [otlp_translation_headers_enabled: <boolean> | default = false]

  # (advanced) HTTP URL path under which the Alertmanager ui and api will be
  # served.
  # CLI flag: -http.alertmanager-http-prefix
  [alertmanager_http_prefix: <string> | default = "/alertmanager"]

  # (advanced) HTTP URL path under which the Prometheus api will be served.
  # CLI flag: -http.prometheus-http-prefix
  [prometheus_http_prefix: <string> | default = "/prometheus"]

  # (experimental) Compression level for HTTP responses when gzip compression is
  # requested by the client. Valid values are 1 (fastest) to 9 (best
  # compression), or -1 for the default compression level.
  # CLI flag: -http.response-compression-level
  [response_compression_level: <int> | default = -1]

# The server block configures the HTTP and gRPC server of the launched
# service(s).
[server: <server>]

# The distributor block configures the distributor.
[distributor: <distributor>]

# The querier block configures the querier.
[querier: <querier>]

# The ingester_client block configures how the distributors connect to the
# ingesters.
[ingester_client: <ingester_client>]

# The ingester block configures the ingester.
[ingester: <ingester>]

# The limits block configures default and per-tenant limits imposed by
# components.
[limits: <limits>]

# The frontend_worker block configures the worker running within the querier,
# picking up and executing queries enqueued by the query-frontend or the
# query-scheduler.
[frontend_worker: <frontend_worker>]

# The frontend block configures the query-frontend.
[frontend: <frontend>]

# The ingest_storage block configures the Kafka-based ingest storage.
[ingest_storage: <ingest_storage>]

# The blocks_storage block configures the blocks storage.
[blocks_storage: <blocks_storage>]

# The compactor block configures the compactor component.
[compactor: <compactor>]

# The store_gateway block configures the store-gateway component.
[store_gateway: <store_gateway>]

tenant_federation:
  # If enabled on all services, queries can be federated across multiple
  # tenants. The tenant IDs involved need to be specified separated by a '|'
  # character in the 'X-Scope-OrgID' header.
  # CLI flag: -tenant-federation.enabled
  [enabled: <boolean> | default = false]

  # (experimental) The number of workers used for each tenant federated query.
  # This setting limits the maximum number of per-tenant queries executed at a
  # time for a tenant federated query.
  # CLI flag: -tenant-federation.max-concurrent
  [max_concurrent: <int> | default = 16]

  # The max number of tenant IDs that may be supplied for a federated query if
  # enabled. 0 to disable the limit.
  # CLI flag: -tenant-federation.max-tenants
  [max_tenants: <int> | default = 0]

activity_tracker:
  # File where ongoing activities are stored. If empty, activity tracking is
  # disabled.
  # CLI flag: -activity-tracker.filepath
  [filepath: <string> | default = "./metrics-activity.log"]

  # (advanced) Max number of concurrent activities that can be tracked. Used to
  # size the file in advance. Additional activities are ignored.
  # CLI flag: -activity-tracker.max-entries
  [max_entries: <int> | default = 1024]

# (experimental) Include tenant ID in pprof labels for profiling. Currently only
# supported by the ingester. This can help debug performance issues for specific
# tenants.
# CLI flag: -include-tenant-id-in-profile-labels
[include_tenant_id_in_profile_labels: <boolean> | default = true]

vault:
  # (experimental) Enables fetching of keys and certificates from Vault
  # CLI flag: -vault.enabled
  [enabled: <boolean> | default = false]

  # (experimental) Location of the Vault server
  # CLI flag: -vault.url
  [url: <string> | default = ""]

  # (experimental) Location of secrets engine within Vault
  # CLI flag: -vault.mount-path
  [mount_path: <string> | default = ""]

  auth:
    # (experimental) Authentication type to use. Supported types are: approle,
    # kubernetes, userpass, token
    # CLI flag: -vault.auth.type
    [type: <string> | default = ""]

    approle:
      # (experimental) Role ID of the AppRole
      # CLI flag: -vault.auth.approle.role-id
      [role_id: <string> | default = ""]

      # (experimental) Secret ID issued against the AppRole
      # CLI flag: -vault.auth.approle.secret-id
      [secret_id: <string> | default = ""]

      # (experimental) Response wrapping token if the Secret ID is response
      # wrapped
      # CLI flag: -vault.auth.approle.wrapping-token
      [wrapping_token: <boolean> | default = false]

      # (experimental) Path if the Vault backend was mounted using a non-default
      # path
      # CLI flag: -vault.auth.approle.mount-path
      [mount_path: <string> | default = ""]

    kubernetes:
      # (experimental) The Kubernetes named role
      # CLI flag: -vault.auth.kubernetes.role-name
      [role_name: <string> | default = ""]

      # (experimental) The Service Account JWT
      # CLI flag: -vault.auth.kubernetes.service-account-token
      [service_account_token: <string> | default = ""]

      # (experimental) Path to where the Kubernetes service account token is
      # mounted. By default it lives at
      # /var/run/secrets/kubernetes.io/serviceaccount/token. Field will be used
      # if the service_account_token is not specified.
      # CLI flag: -vault.auth.kubernetes.service-account-token-path
      [service_account_token_path: <string> | default = ""]

      # (experimental) Path if the Vault backend was mounted using a non-default
      # path
      # CLI flag: -vault.auth.kubernetes.mount-path
      [mount_path: <string> | default = ""]

    userpass:
      # (experimental) The userpass auth method username
      # CLI flag: -vault.auth.userpass.username
      [username: <string> | default = ""]

      # (experimental) The userpass auth method password
      # CLI flag: -vault.auth.userpass.password
      [password: <string> | default = ""]

      # (experimental) Path if the Vault backend was mounted using a non-default
      # path
      # CLI flag: -vault.auth.userpass.mount-path
      [mount_path: <string> | default = ""]

    token:
      # (experimental) The token used to authenticate against Vault
      # CLI flag: -vault.auth.token
      [token: <string> | default = ""]

# The ruler block configures the ruler.
[ruler: <ruler>]

# The ruler_storage block configures the ruler storage backend.
[ruler_storage: <ruler_storage>]

# The alertmanager block configures the alertmanager.
[alertmanager: <alertmanager>]

# The alertmanager_storage block configures the alertmanager storage backend.
[alertmanager_storage: <alertmanager_storage>]

runtime_config:
  # (advanced) How often to check runtime config files.
  # CLI flag: -runtime-config.reload-period
  [period: <duration> | default = 10s]

  # Comma separated list of yaml files or URLs with the configuration that can
  # be updated at runtime. Runtime config files will be merged from left to
  # right.
  # CLI flag: -runtime-config.file
  [file: <string> | default = ""]

  # (advanced) HTTP client timeout when fetching runtime config from URLs.
  # CLI flag: -runtime-config.http-client-timeout
  [http_client_timeout: <duration> | default = 30s]

  http_client_cluster_validation:
    # (experimental) Primary cluster validation label.
    # CLI flag: -runtime-config.http-client-cluster-validation.label
    [label: <string> | default = ""]

# The memberlist block configures the Gossip memberlist.
[memberlist: <memberlist>]

# The query_scheduler block configures the query-scheduler.
[query_scheduler: <query_scheduler>]

usage_stats:
  # Enable anonymous usage reporting.
  # CLI flag: -usage-stats.enabled
  [enabled: <boolean> | default = true]

  # Installation mode. Supported values: custom, helm, jsonnet.
  # CLI flag: -usage-stats.installation-mode
  [installation_mode: <string> | default = "custom"]

overrides_exporter:
  ring:
    # Enable the ring used by override-exporters to deduplicate exported limit
    # metrics.
    # CLI flag: -overrides-exporter.ring.enabled
    [enabled: <boolean> | default = false]

    # The key-value store used to share the hash ring across multiple instances.
    kvstore:
      # Backend storage to use for the ring. Supported values are: consul, etcd,
      # inmemory, memberlist, multi.
      # CLI flag: -overrides-exporter.ring.store
      [store: <string> | default = "memberlist"]

      # (advanced) The prefix for the keys in the store. Should end with a /.
      # CLI flag: -overrides-exporter.ring.prefix
      [prefix: <string> | default = "collectors/"]

      # The consul block configures the consul client.
      # The CLI flags prefix for this block configuration is:
      # overrides-exporter.ring
      [consul: <consul>]

      # The etcd block configures the etcd client.
      # The CLI flags prefix for this block configuration is:
      # overrides-exporter.ring
      [etcd: <etcd>]

      multi:
        # (advanced) Primary backend storage used by multi-client.
        # CLI flag: -overrides-exporter.ring.multi.primary
        [primary: <string> | default = ""]

        # (advanced) Secondary backend storage used by multi-client.
        # CLI flag: -overrides-exporter.ring.multi.secondary
        [secondary: <string> | default = ""]

        # (advanced) Mirror writes to the secondary store.
        # CLI flag: -overrides-exporter.ring.multi.mirror-enabled
        [mirror_enabled: <boolean> | default = false]

        # (advanced) Timeout for storing a value to the secondary store.
        # CLI flag: -overrides-exporter.ring.multi.mirror-timeout
        [mirror_timeout: <duration> | default = 2s]

    # (advanced) Period at which to heartbeat to the ring.
    # CLI flag: -overrides-exporter.ring.heartbeat-period
    [heartbeat_period: <duration> | default = 15s]

    # (advanced) Heartbeat timeout after which Mimir marks overrides-exporters
    # as unhealthy in the ring.
    # CLI flag: -overrides-exporter.ring.heartbeat-timeout
    [heartbeat_timeout: <duration> | default = 1m]

    # (advanced) Instance ID to register in the ring.
    # CLI flag: -overrides-exporter.ring.instance-id
    [instance_id: <string> | default = "<hostname>"]

    # List of network interface names to look up when finding the instance IP
    # address.
    # CLI flag: -overrides-exporter.ring.instance-interface-names
    [instance_interface_names: <list of strings> | default = [<private network interfaces>]]

    # (advanced) Port to advertise in the ring (defaults to
    # -server.grpc-listen-port).
    # CLI flag: -overrides-exporter.ring.instance-port
    [instance_port: <int> | default = 0]

    # (advanced) IP address to advertise in the ring. Default is auto-detected.
    # CLI flag: -overrides-exporter.ring.instance-addr
    [instance_addr: <string> | default = ""]

    # (advanced) Enable using an IPv6 instance address.
    # CLI flag: -overrides-exporter.ring.instance-enable-ipv6
    [instance_enable_ipv6: <boolean> | default = false]

    # (advanced) Minimum time to wait for ring stability at startup, if set to
    # positive value. Set to 0 to disable.
    # CLI flag: -overrides-exporter.ring.wait-stability-min-duration
    [wait_stability_min_duration: <duration> | default = 0s]

    # (advanced) Maximum time to wait for ring stability at startup. If the
    # overrides-exporter ring keeps changing after this period of time, it will
    # start anyway.
    # CLI flag: -overrides-exporter.ring.wait-stability-max-duration
    [wait_stability_max_duration: <duration> | default = 5m]

    # (advanced) Number of consecutive timeout periods after which Mimir
    # automatically removes an unhealthy instance in the ring. Set to 0 to
    # disable auto-forget.
    # CLI flag: -overrides-exporter.ring.auto-forget-unhealthy-periods
    [auto_forget_unhealthy_periods: <int> | default = 4]

  # Comma-separated list of metrics to include in the exporter. Metric names
  # must match yaml tags from the limits section of the configuration.
  # CLI flag: -overrides-exporter.enabled-metrics
  [enabled_metrics: <string> | default = "ingestion_burst_size,ingestion_rate,max_fetched_chunk_bytes_per_query,max_fetched_chunks_per_query,max_fetched_series_per_query,max_global_exemplars_per_user,max_global_series_per_metric,max_global_series_per_user,ruler_max_rule_groups_per_tenant,ruler_max_rules_per_rule_group"]

# The common block holds configurations that configure multiple components at a
# time.
[common: <common>]

# (experimental) Enables optimized marshaling of timeseries.
# CLI flag: -timeseries-unmarshal-caching-optimization-enabled
[timeseries_unmarshal_caching_optimization_enabled: <boolean> | default = true]

# (experimental) Specifies how often inactive cost attributions for received and
# discarded sample trackers are evicted from the counter, ensuring they do not
# contribute to the cost attribution cardinality per user limit. This setting
# does not apply to active series, which are managed separately.
# CLI flag: -cost-attribution.eviction-interval
[cost_attribution_eviction_interval: <duration> | default = 20m]

# (experimental) Defines a custom path for the registry. When specified, Mimir
# exposes cost attribution metrics through this custom path. If not specified,
# cost attribution metrics aren't exposed.
# CLI flag: -cost-attribution.registry-path
[cost_attribution_registry_path: <string> | default = ""]

# (experimental) Time interval at which the cost attribution cleanup process
# runs, ensuring inactive cost attribution entries are purged.
# CLI flag: -cost-attribution.cleanup-interval
[cost_attribution_cleanup_interval: <duration> | default = 3m]

common

The common block holds configurations that configure multiple components at a time.

storage:
  # Backend storage to use. Supported backends are: s3, gcs, azure, swift,
  # filesystem.
  # CLI flag: -common.storage.backend
  [backend: <string> | default = "filesystem"]

  # The s3_backend block configures the connection to Amazon S3 object storage
  # backend.
  # The CLI flags prefix for this block configuration is: common.storage
  [s3: <s3_storage_backend>]

  # The gcs_backend block configures the connection to Google Cloud Storage
  # object storage backend.
  # The CLI flags prefix for this block configuration is: common.storage
  [gcs: <gcs_storage_backend>]

  # The azure_storage_backend block configures the connection to Azure object
  # storage backend.
  # The CLI flags prefix for this block configuration is: common.storage
  [azure: <azure_storage_backend>]

  # The swift_storage_backend block configures the connection to OpenStack
  # Object Storage (Swift) object storage backend.
  # The CLI flags prefix for this block configuration is: common.storage
  [swift: <swift_storage_backend>]

  # The filesystem_storage_backend block configures the usage of local file
  # system as object storage backend.
  # The CLI flags prefix for this block configuration is: common.storage
  [filesystem: <filesystem_storage_backend>]

client_cluster_validation:
  # (experimental) Primary cluster validation label.
  # CLI flag: -common.client-cluster-validation.label
  [label: <string> | default = ""]

instrument_ref_leaks:
  # (experimental) Percentage [0-100] of request or message buffers to
  # instrument for reference leaks. Set to 0 to disable.
  # CLI flag: -common.instrument-reference-leaks.percentage
  [percentage: <float> | default = 0]

  # (experimental) Period after a buffer instrumented for referenced leaks is
  # nominally freed until the buffer is uninstrumented and effectively freed to
  # be reused. After this period, any lingering references to the buffer may
  # potentially be dereferenced again with no detection.
  # CLI flag: -common.instrument-reference-leaks.before-reuse-period
  [before_reuse_period: <duration> | default = 2m]

  # (experimental) Maximum sum of length of buffers instrumented at any given
  # time, in bytes. When surpassed, incoming buffers will not be instrumented,
  # regardless of the configured percentage. Zero means no limit.
  # CLI flag: -common.instrument-reference-leaks.max-inflight-instrumented-bytes
  [max_inflight_instrumented_bytes: <int> | default = 0]

server

The server block configures the HTTP and gRPC server of the launched service(s).

# (advanced) HTTP server listen network, default tcp
# CLI flag: -server.http-listen-network
[http_listen_network: <string> | default = "tcp"]

# HTTP server listen address.
# CLI flag: -server.http-listen-address
[http_listen_address: <string> | default = ""]

# HTTP server listen port.
# CLI flag: -server.http-listen-port
[http_listen_port: <int> | default = 8080]

# (advanced) Maximum number of simultaneous http connections, <=0 to disable
# CLI flag: -server.http-conn-limit
[http_listen_conn_limit: <int> | default = 0]

# (advanced) gRPC server listen network
# CLI flag: -server.grpc-listen-network
[grpc_listen_network: <string> | default = "tcp"]

# gRPC server listen address.
# CLI flag: -server.grpc-listen-address
[grpc_listen_address: <string> | default = ""]

# gRPC server listen port.
# CLI flag: -server.grpc-listen-port
[grpc_listen_port: <int> | default = 9095]

# (advanced) Maximum number of simultaneous grpc connections, <=0 to disable
# CLI flag: -server.grpc-conn-limit
[grpc_listen_conn_limit: <int> | default = 0]

# If true, the max streams by connection gauge will be collected.
# CLI flag: -server.grpc-collect-max-streams-by-conn
[grpc_collect_max_streams_by_conn: <boolean> | default = true]

# (experimental) Enables PROXY protocol.
# CLI flag: -server.proxy-protocol-enabled
[proxy_protocol_enabled: <boolean> | default = false]

# Comma-separated list of cipher suites to use. If blank, the default Go cipher
# suites is used.
# CLI flag: -server.tls-cipher-suites
[tls_cipher_suites: <string> | default = ""]

# Minimum TLS version to use. Allowed values: VersionTLS10, VersionTLS11,
# VersionTLS12, VersionTLS13. If blank, the Go TLS minimum version is used.
# CLI flag: -server.tls-min-version
[tls_min_version: <string> | default = ""]

http_tls_config:
  # Server TLS certificate. This configuration parameter is YAML only.
  [cert: <string> | default = ""]

  # Server TLS key. This configuration parameter is YAML only.
  [key: <string> | default = ""]

  # Root certificate authority used to verify client certificates. This
  # configuration parameter is YAML only.
  [client_ca: <string> | default = ""]

  # (advanced) HTTP server cert path.
  # CLI flag: -server.http-tls-cert-path
  [cert_file: <string> | default = ""]

  # (advanced) HTTP server key path.
  # CLI flag: -server.http-tls-key-path
  [key_file: <string> | default = ""]

  # (advanced) HTTP TLS Client Auth type.
  # CLI flag: -server.http-tls-client-auth
  [client_auth_type: <string> | default = ""]

  # (advanced) HTTP TLS Client CA path.
  # CLI flag: -server.http-tls-ca-path
  [client_ca_file: <string> | default = ""]

grpc_tls_config:
  # Server TLS certificate. This configuration parameter is YAML only.
  [cert: <string> | default = ""]

  # Server TLS key. This configuration parameter is YAML only.
  [key: <string> | default = ""]

  # Root certificate authority used to verify client certificates. This
  # configuration parameter is YAML only.
  [client_ca: <string> | default = ""]

  # (advanced) GRPC TLS server cert path.
  # CLI flag: -server.grpc-tls-cert-path
  [cert_file: <string> | default = ""]

  # (advanced) GRPC TLS server key path.
  # CLI flag: -server.grpc-tls-key-path
  [key_file: <string> | default = ""]

  # (advanced) GRPC TLS Client Auth type.
  # CLI flag: -server.grpc-tls-client-auth
  [client_auth_type: <string> | default = ""]

  # (advanced) GRPC TLS Client CA path.
  # CLI flag: -server.grpc-tls-ca-path
  [client_ca_file: <string> | default = ""]

# (advanced) Register the intrumentation handlers (/metrics etc).
# CLI flag: -server.register-instrumentation
[register_instrumentation: <boolean> | default = true]

# If set to true, gRPC statuses will be reported in instrumentation labels with
# their string representations. Otherwise, they will be reported as "error".
# CLI flag: -server.report-grpc-codes-in-instrumentation-label-enabled
[report_grpc_codes_in_instrumentation_label_enabled: <boolean> | default = true]

# (advanced) Timeout for graceful shutdowns
# CLI flag: -server.graceful-shutdown-timeout
[graceful_shutdown_timeout: <duration> | default = 30s]

# (advanced) Read timeout for entire HTTP request, including headers and body.
# CLI flag: -server.http-read-timeout
[http_server_read_timeout: <duration> | default = 30s]

# Read timeout for HTTP request headers. If set to 0, value of
# -server.http-read-timeout is used.
# CLI flag: -server.http-read-header-timeout
[http_server_read_header_timeout: <duration> | default = 0s]

# (advanced) Write timeout for HTTP server
# CLI flag: -server.http-write-timeout
[http_server_write_timeout: <duration> | default = 2m]

# (advanced) Idle timeout for HTTP server
# CLI flag: -server.http-idle-timeout
[http_server_idle_timeout: <duration> | default = 2m]

# Log closed connections that did not receive any response, most likely because
# client didn't send any request within timeout.
# CLI flag: -server.http-log-closed-connections-without-response-enabled
[http_log_closed_connections_without_response_enabled: <boolean> | default = false]

# (advanced) Limit on the size of a gRPC message this server can receive
# (bytes).
# CLI flag: -server.grpc-max-recv-msg-size-bytes
[grpc_server_max_recv_msg_size: <int> | default = 104857600]

# (advanced) Limit on the size of a gRPC message this server can send (bytes).
# CLI flag: -server.grpc-max-send-msg-size-bytes
[grpc_server_max_send_msg_size: <int> | default = 104857600]

# (advanced) Limit on the number of concurrent streams for gRPC calls per client
# connection (0 = unlimited)
# CLI flag: -server.grpc-max-concurrent-streams
[grpc_server_max_concurrent_streams: <int> | default = 100]

# (advanced) The duration after which an idle connection should be closed.
# Default: infinity
# CLI flag: -server.grpc.keepalive.max-connection-idle
[grpc_server_max_connection_idle: <duration> | default = 2562047h47m16.854775807s]

# (advanced) The duration for the maximum amount of time a connection may exist
# before it will be closed. Default: infinity
# CLI flag: -server.grpc.keepalive.max-connection-age
[grpc_server_max_connection_age: <duration> | default = 2562047h47m16.854775807s]

# (advanced) An additive period after max-connection-age after which the
# connection will be forcibly closed. Default: infinity
# CLI flag: -server.grpc.keepalive.max-connection-age-grace
[grpc_server_max_connection_age_grace: <duration> | default = 2562047h47m16.854775807s]

# (advanced) Duration after which a keepalive probe is sent in case of no
# activity over the connection., Default: 2h
# CLI flag: -server.grpc.keepalive.time
[grpc_server_keepalive_time: <duration> | default = 2h]

# (advanced) After having pinged for keepalive check, the duration after which
# an idle connection should be closed, Default: 20s
# CLI flag: -server.grpc.keepalive.timeout
[grpc_server_keepalive_timeout: <duration> | default = 20s]

# (advanced) Minimum amount of time a client should wait before sending a
# keepalive ping. If client sends keepalive ping more often, server will send
# GOAWAY and close the connection.
# CLI flag: -server.grpc.keepalive.min-time-between-pings
[grpc_server_min_time_between_pings: <duration> | default = 10s]

# (advanced) If true, server allows keepalive pings even when there are no
# active streams(RPCs). If false, and client sends ping when there are no active
# streams, server will send GOAWAY and close the connection.
# CLI flag: -server.grpc.keepalive.ping-without-stream-allowed
[grpc_server_ping_without_stream_allowed: <boolean> | default = true]

# (advanced) If non-zero, configures the amount of GRPC server workers used to
# serve the requests.
# CLI flag: -server.grpc.num-workers
[grpc_server_num_workers: <int> | default = 100]

# Size of the read buffer for each gRPC connection (bytes). A smaller buffer may
# reduce memory usage but may lead to more system calls.
# CLI flag: -server.grpc.read-buffer-size-bytes
[grpc_server_read_buffer_size: <int> | default = 32768]

# Size of the write buffer for each gRPC connection (bytes). A smaller buffer
# may reduce memory usage but may lead to more system calls.
# CLI flag: -server.grpc.write-buffer-size-bytes
[grpc_server_write_buffer_size: <int> | default = 32768]

# Output log messages in the given format. Valid formats: [logfmt, json]
# CLI flag: -log.format
[log_format: <string> | default = "logfmt"]

# Only log messages with the given severity or above. Valid levels: [debug,
# info, warn, error]
# CLI flag: -log.level
[log_level: <string> | default = "info"]

# (advanced) Optionally log the source IPs.
# CLI flag: -server.log-source-ips-enabled
[log_source_ips_enabled: <boolean> | default = false]

# Log all source IPs instead of only the originating one. Only used if
# server.log-source-ips-enabled is true
# CLI flag: -server.log-source-ips-full
[log_source_ips_full: <boolean> | default = false]

# (advanced) Header field storing the source IPs. Only used if
# server.log-source-ips-enabled is true. If not set the default Forwarded,
# X-Real-IP and X-Forwarded-For headers are used
# CLI flag: -server.log-source-ips-header
[log_source_ips_header: <string> | default = ""]

# (advanced) Regex for matching the source IPs. Only used if
# server.log-source-ips-enabled is true. If not set the default Forwarded,
# X-Real-IP and X-Forwarded-For headers are used
# CLI flag: -server.log-source-ips-regex
[log_source_ips_regex: <string> | default = ""]

# Optionally log request headers.
# CLI flag: -server.log-request-headers
[log_request_headers: <boolean> | default = false]

# (advanced) Optionally log requests at info level instead of debug level.
# Applies to request headers as well if server.log-request-headers is enabled.
# CLI flag: -server.log-request-at-info-level-enabled
[log_request_at_info_level_enabled: <boolean> | default = false]

# Comma separated list of headers to exclude from loggin. Only used if
# server.log-request-headers is true.
# CLI flag: -server.log-request-headers-exclude-list
[log_request_exclude_headers_list: <string> | default = ""]

# Optionally add request headers to tracing spans.
# CLI flag: -server.trace-request-headers
[trace_request_headers: <boolean> | default = false]

# Comma separated list of headers to exclude from tracing spans. Only used if
# server.trace-request-headers is true. The following headers are always
# excluded: Authorization, Cookie, X-Access-Token, X-Csrf-Token, X-Grafana-Id.
# CLI flag: -server.trace-request-headers-exclude-list
[trace_request_exclude_headers_list: <string> | default = ""]

# (advanced) Base path to serve all API routes from (e.g. /v1/)
# CLI flag: -server.path-prefix
[http_path_prefix: <string> | default = ""]

cluster_validation:
  # (experimental) Primary cluster validation label.
  # CLI flag: -server.cluster-validation.label
  [label: <string> | default = ""]

  # (experimental) Comma-separated list of additional cluster validation labels
  # that the server will accept from incoming requests.
  # CLI flag: -server.cluster-validation.additional-labels
  [additional_labels: <string> | default = ""]

  grpc:
    # (experimental) When enabled, cluster label validation is executed:
    # configured cluster validation label is compared with the cluster
    # validation label received through the requests.
    # CLI flag: -server.cluster-validation.grpc.enabled
    [enabled: <boolean> | default = false]

    # (experimental) When enabled, soft cluster label validation is executed.
    # Can be enabled only together with server.cluster-validation.grpc.enabled
    # CLI flag: -server.cluster-validation.grpc.soft-validation
    [soft_validation: <boolean> | default = false]

  http:
    # (experimental) When enabled, cluster label validation is executed:
    # configured cluster validation label is compared with the cluster
    # validation label received through the requests.
    # CLI flag: -server.cluster-validation.http.enabled
    [enabled: <boolean> | default = false]

    # (experimental) When enabled, soft cluster label validation is executed.
    # Can be enabled only together with server.cluster-validation.http.enabled
    # CLI flag: -server.cluster-validation.http.soft-validation
    [soft_validation: <boolean> | default = false]

    # (experimental) Comma-separated list of url paths that are excluded from
    # the cluster validation check.
    # CLI flag: -server.cluster-validation.http.excluded-paths
    [excluded_paths: <string> | default = ""]

    # (experimental) Comma-separated list of user agents that are excluded from
    # the cluster validation check.
    # CLI flag: -server.cluster-validation.http.excluded-user-agents
    [excluded_user_agents: <string> | default = ""]

# Creates new traces for each call rather than continuing the existing trace. A
# span link is used to allow navigation to the parent trace. Only works when
# using Open-Telemetry tracing.
# CLI flag: -server.create-new-traces
[create_new_traces: <boolean> | default = false]

distributor

The distributor block configures the distributor.

pool:
  # (advanced) How frequently to clean up clients for ingesters that have gone
  # away.
  # CLI flag: -distributor.client-cleanup-period
  [client_cleanup_period: <duration> | default = 15s]

  # (advanced) Run a health check on each ingester client during periodic
  # cleanup.
  # CLI flag: -distributor.health-check-ingesters
  [health_check_ingesters: <boolean> | default = true]

  # (experimental) The grace period for ingester health checks. If an ingester
  # connection consistently fails health checks for this period, any open
  # connections are closed. The distributor or querier will attempt to reconnect
  # to the ingester if a subsequent request is made to the ingester. Set to 0 to
  # immediately remove ingester connections on the first health check failure.
  # CLI flag: -distributor.ingester-health-check-grace-period
  [ingester_health_check_grace_period: <duration> | default = 0s]

retry_after_header:
  # (advanced) Enables inclusion of the Retry-After header in the response: true
  # includes it for client retry guidance, false omits it.
  # CLI flag: -distributor.retry-after-header.enabled
  [enabled: <boolean> | default = true]

  # (advanced) Minimum duration of the Retry-After HTTP header in responses to
  # 429/5xx errors. Must be greater than or equal to 1s. Backoff is calculated
  # as MinBackoff*2^(RetryAttempt-1) seconds with random jitter of 50% in either
  # direction. RetryAttempt is the value of the Retry-Attempt HTTP header.
  # CLI flag: -distributor.retry-after-header.min-backoff
  [min_backoff: <duration> | default = 6s]

  # (advanced) Minimum duration of the Retry-After HTTP header in responses to
  # 429/5xx errors. Must be greater than or equal to 1s. Backoff is calculated
  # as MinBackoff*2^(RetryAttempt-1) seconds with random jitter of 50% in either
  # direction. RetryAttempt is the value of the Retry-Attempt HTTP header.
  # CLI flag: -distributor.retry-after-header.max-backoff
  [max_backoff: <duration> | default = 1m36s]

ha_tracker:
  # Enable the distributors HA tracker so that it can accept samples from
  # Prometheus HA replicas gracefully (requires labels).
  # CLI flag: -distributor.ha-tracker.enable
  [enable_ha_tracker: <boolean> | default = false]

  # Enable the elected_replica_status metric, which shows the current elected
  # replica. It is disabled by default due to the possible high cardinality of
  # the metric.
  # CLI flag: -distributor.ha-tracker.enable-elected-replica-metric
  [enable_elected_replica_metric: <boolean> | default = false]

  # Backend storage to use for the HA tracker. Supported values are: consul,
  # etcd, inmemory, memberlist, multi. Memberlist is recommended.
  kvstore:
    # Backend storage to use for the ring. Supported values are: consul, etcd,
    # inmemory, memberlist, multi.
    # CLI flag: -distributor.ha-tracker.store
    [store: <string> | default = "memberlist"]

    # (advanced) The prefix for the keys in the store. Should end with a /.
    # CLI flag: -distributor.ha-tracker.prefix
    [prefix: <string> | default = "ha-tracker/"]

    # The consul block configures the consul client.
    # The CLI flags prefix for this block configuration is:
    # distributor.ha-tracker
    [consul: <consul>]

    # The etcd block configures the etcd client.
    # The CLI flags prefix for this block configuration is:
    # distributor.ha-tracker
    [etcd: <etcd>]

    multi:
      # (advanced) Primary backend storage used by multi-client.
      # CLI flag: -distributor.ha-tracker.multi.primary
      [primary: <string> | default = ""]

      # (advanced) Secondary backend storage used by multi-client.
      # CLI flag: -distributor.ha-tracker.multi.secondary
      [secondary: <string> | default = ""]

      # (advanced) Mirror writes to the secondary store.
      # CLI flag: -distributor.ha-tracker.multi.mirror-enabled
      [mirror_enabled: <boolean> | default = false]

      # (advanced) Timeout for storing a value to the secondary store.
      # CLI flag: -distributor.ha-tracker.multi.mirror-timeout
      [mirror_timeout: <duration> | default = 2s]

# (advanced) Max message size in bytes that the distributors will accept for
# incoming push requests to the remote write API. If exceeded, the request will
# be rejected.
# CLI flag: -distributor.max-recv-msg-size
[max_recv_msg_size: <int> | default = 104857600]

# (experimental) Maximum OTLP request size in bytes that the distributors
# accept. Requests exceeding this limit are rejected.
# CLI flag: -distributor.max-otlp-request-size
[max_otlp_request_size: <int> | default = 104857600]

# (experimental) Max size of the pooled buffers used for marshaling write
# requests. If 0, no max size is enforced.
# CLI flag: -distributor.max-request-pool-buffer-size
[max_request_pool_buffer_size: <int> | default = 0]

# (advanced) Timeout for downstream ingesters.
# CLI flag: -distributor.remote-timeout
[remote_timeout: <duration> | default = 2s]

ring:
  # The key-value store used to share the hash ring across multiple instances.
  kvstore:
    # Backend storage to use for the ring. Supported values are: consul, etcd,
    # inmemory, memberlist, multi.
    # CLI flag: -distributor.ring.store
    [store: <string> | default = "memberlist"]

    # (advanced) The prefix for the keys in the store. Should end with a /.
    # CLI flag: -distributor.ring.prefix
    [prefix: <string> | default = "collectors/"]

    # The consul block configures the consul client.
    # The CLI flags prefix for this block configuration is: distributor.ring
    [consul: <consul>]

    # The etcd block configures the etcd client.
    # The CLI flags prefix for this block configuration is: distributor.ring
    [etcd: <etcd>]

    multi:
      # (advanced) Primary backend storage used by multi-client.
      # CLI flag: -distributor.ring.multi.primary
      [primary: <string> | default = ""]

      # (advanced) Secondary backend storage used by multi-client.
      # CLI flag: -distributor.ring.multi.secondary
      [secondary: <string> | default = ""]

      # (advanced) Mirror writes to the secondary store.
      # CLI flag: -distributor.ring.multi.mirror-enabled
      [mirror_enabled: <boolean> | default = false]

      # (advanced) Timeout for storing a value to the secondary store.
      # CLI flag: -distributor.ring.multi.mirror-timeout
      [mirror_timeout: <duration> | default = 2s]

  # (advanced) Period at which to heartbeat to the ring.
  # CLI flag: -distributor.ring.heartbeat-period
  [heartbeat_period: <duration> | default = 15s]

  # (advanced) Heartbeat timeout after which Mimir marks distributors as
  # unhealthy in the ring.
  # CLI flag: -distributor.ring.heartbeat-timeout
  [heartbeat_timeout: <duration> | default = 1m]

  # (advanced) Instance ID to register in the ring.
  # CLI flag: -distributor.ring.instance-id
  [instance_id: <string> | default = "<hostname>"]

  # List of network interface names to look up when finding the instance IP
  # address.
  # CLI flag: -distributor.ring.instance-interface-names
  [instance_interface_names: <list of strings> | default = [<private network interfaces>]]

  # (advanced) Port to advertise in the ring (defaults to
  # -server.grpc-listen-port).
  # CLI flag: -distributor.ring.instance-port
  [instance_port: <int> | default = 0]

  # (advanced) IP address to advertise in the ring. Default is auto-detected.
  # CLI flag: -distributor.ring.instance-addr
  [instance_addr: <string> | default = ""]

  # (advanced) Enable using an IPv6 instance address.
  # CLI flag: -distributor.ring.instance-enable-ipv6
  [instance_enable_ipv6: <boolean> | default = false]

  # (advanced) The availability zone where this instance is running. Used for
  # zone-aware rate limiting.
  # CLI flag: -distributor.ring.instance-availability-zone
  [instance_availability_zone: <string> | default = ""]

  # (advanced) Number of consecutive timeout periods after which Mimir
  # automatically removes an unhealthy instance in the ring. Set to 0 to disable
  # auto-forget.
  # CLI flag: -distributor.ring.auto-forget-unhealthy-periods
  [auto_forget_unhealthy_periods: <int> | default = 10]

instance_limits:
  # (advanced) Max ingestion rate (samples/sec) that this distributor will
  # accept. This limit is per-distributor, not per-tenant. Additional push
  # requests will be rejected. Current ingestion rate is computed as
  # exponentially weighted moving average, updated every second. 0 = unlimited.
  # CLI flag: -distributor.instance-limits.max-ingestion-rate
  [max_ingestion_rate: <float> | default = 0]

  # (advanced) Max inflight push requests that this distributor can handle. This
  # limit is per-distributor, not per-tenant. Additional requests will be
  # rejected. 0 = unlimited.
  # CLI flag: -distributor.instance-limits.max-inflight-push-requests
  [max_inflight_push_requests: <int> | default = 2000]

  # (advanced) The sum of the request sizes in bytes of inflight push requests
  # that this distributor can handle. This limit is per-distributor, not
  # per-tenant. Additional requests will be rejected. 0 = unlimited.
  # CLI flag: -distributor.instance-limits.max-inflight-push-requests-bytes
  [max_inflight_push_requests_bytes: <int> | default = 0]

# (experimental) Enable pooling of buffers used for marshaling write requests.
# CLI flag: -distributor.write-requests-buffer-pooling-enabled
[write_requests_buffer_pooling_enabled: <boolean> | default = true]

# (advanced) Number of pre-allocated workers used to forward push requests to
# the ingesters. If 0, no workers will be used and a new goroutine will be
# spawned for each ingester push request. If not enough workers available, new
# goroutine will be spawned. (Note: this is a performance optimization, not a
# limiting feature.)
# CLI flag: -distributor.reusable-ingester-push-workers
[reusable_ingester_push_workers: <int> | default = 2000]

reactive_limiter:
  # (experimental) Enable reactive limiting when making requests to a service
  # CLI flag: -distributor.reactive-limiter.enabled
  [enabled: <boolean> | default = false]

  # (experimental) Minimum inflight requests limit
  # CLI flag: -distributor.reactive-limiter.min-limit
  [min_limit: <int> | default = 2]

  # (experimental) Maximum inflight requests limit
  # CLI flag: -distributor.reactive-limiter.max-limit
  [max_limit: <int> | default = 200]

  # (experimental) Initial inflight requests limit
  # CLI flag: -distributor.reactive-limiter.initial-limit
  [initial_limit: <int> | default = 20]

  # (experimental) The maximum inflight limit as a multiple of current inflight
  # requests
  # CLI flag: -distributor.reactive-limiter.max-limit-factor
  [max_limit_factor: <float> | default = 5]

  # (experimental) Logarithmic decay applied to the maxLimitFactor based on
  # current inflight requests
  # CLI flag: -distributor.reactive-limiter.max-limit-factor-decay
  [max_limit_factor_decay: <float> | default = 1]

  # (experimental) Minimum limit factor when max-limit-factor-decay is applied
  # CLI flag: -distributor.reactive-limiter.min-limit-factor
  [min_limit_factor: <float> | default = 1.2]

  # (experimental) Minimum duration of the window that is used to collect recent
  # response time samples
  # CLI flag: -distributor.reactive-limiter.recent-window-min-duration
  [recent_window_min_duration: <duration> | default = 1s]

  # (experimental) Maximum duration of the window that is used to collect recent
  # response time samples
  # CLI flag: -distributor.reactive-limiter.recent-window-max-duration
  [recent_window_max_duration: <duration> | default = 30s]

  # (experimental) Minimum number of samples that must be recorded in the recent
  # window before updating the limit
  # CLI flag: -distributor.reactive-limiter.recent-window-min-samples
  [recent_window_min_samples: <int> | default = 50]

  # (experimental) The quantile of recent recorded response times to consider
  # when adjusting the concurrency limit
  # CLI flag: -distributor.reactive-limiter.recent-quantile
  [recent_quantile: <float> | default = 0.9]

  # (experimental) The average age of baseline samples aggregated recent samples
  # are added to
  # CLI flag: -distributor.reactive-limiter.baseline-window-age
  [baseline_window_age: <int> | default = 10]

  # (experimental) How many recent limit and inflight time measurements are
  # stored to detect whether increases in limits correlate with increases in
  # inflight times
  # CLI flag: -distributor.reactive-limiter.correlation-window
  [correlation_window: <int> | default = 50]

  # (experimental) The number of allowed queued requests, as a multiple of
  # current inflight requests, after which rejections start
  # CLI flag: -distributor.reactive-limiter.initial-rejection-factor
  [initial_rejection_factor: <float> | default = 1]

  # (experimental) The number of allowed queued requests, as a multiple of
  # current inflight requests, after which all requests are rejected
  # CLI flag: -distributor.reactive-limiter.max-rejection-factor
  [max_rejection_factor: <float> | default = 2]

ingester

The ingester block configures the ingester.

ring:
  # The key-value store used to share the hash ring across multiple instances.
  # This option needs be set on ingesters, distributors, queriers, and rulers
  # when running in microservices mode.
  kvstore:
    # Backend storage to use for the ring. Supported values are: consul, etcd,
    # inmemory, memberlist, multi.
    # CLI flag: -ingester.ring.store
    [store: <string> | default = "memberlist"]

    # (advanced) The prefix for the keys in the store. Should end with a /.
    # CLI flag: -ingester.ring.prefix
    [prefix: <string> | default = "collectors/"]

    # The consul block configures the consul client.
    # The CLI flags prefix for this block configuration is: ingester.ring
    [consul: <consul>]

    # The etcd block configures the etcd client.
    # The CLI flags prefix for this block configuration is: ingester.ring
    [etcd: <etcd>]

    multi:
      # (advanced) Primary backend storage used by multi-client.
      # CLI flag: -ingester.ring.multi.primary
      [primary: <string> | default = ""]

      # (advanced) Secondary backend storage used by multi-client.
      # CLI flag: -ingester.ring.multi.secondary
      [secondary: <string> | default = ""]

      # (advanced) Mirror writes to the secondary store.
      # CLI flag: -ingester.ring.multi.mirror-enabled
      [mirror_enabled: <boolean> | default = false]

      # (advanced) Timeout for storing a value to the secondary store.
      # CLI flag: -ingester.ring.multi.mirror-timeout
      [mirror_timeout: <duration> | default = 2s]

  # (advanced) Period at which to heartbeat to the ring.
  # CLI flag: -ingester.ring.heartbeat-period
  [heartbeat_period: <duration> | default = 15s]

  # (advanced) The heartbeat timeout after which ingesters are skipped for
  # reads/writes. This option needs be set on ingesters, distributors, queriers,
  # and rulers when running in microservices mode.
  # CLI flag: -ingester.ring.heartbeat-timeout
  [heartbeat_timeout: <duration> | default = 1m]

  # Number of ingesters that each time series is replicated to. This
  # configuration is not used when ingest storage is enabled. This option needs
  # be set on ingesters, distributors, queriers, and rulers when running in
  # microservices mode.
  # CLI flag: -ingester.ring.replication-factor
  [replication_factor: <int> | default = 3]

  # True to enable the zone-awareness and replicate ingested samples across
  # different availability zones. This option needs be set on ingesters,
  # distributors, queriers, and rulers when running in microservices mode.
  # CLI flag: -ingester.ring.zone-awareness-enabled
  [zone_awareness_enabled: <boolean> | default = false]

  # (advanced) Comma-separated list of zones to exclude from the ring. Instances
  # in excluded zones will be filtered out from the ring. This option needs be
  # set on ingesters, distributors, queriers, and rulers when running in
  # microservices mode.
  # CLI flag: -ingester.ring.excluded-zones
  [excluded_zones: <string> | default = ""]

  # File path where tokens are stored. If empty, tokens are not stored at
  # shutdown and restored at startup. Must be empty if
  # -ingester.ring.token-generation-strategy is set to "spread-minimizing".
  # CLI flag: -ingester.ring.tokens-file-path
  [tokens_file_path: <string> | default = ""]

  # (advanced) Number of tokens for each ingester.
  # CLI flag: -ingester.ring.num-tokens
  [num_tokens: <int> | default = 128]

  # (advanced) Instance ID to register in the ring.
  # CLI flag: -ingester.ring.instance-id
  [instance_id: <string> | default = "<hostname>"]

  # (advanced) List of network interface names to look up when finding the
  # instance IP address.
  # CLI flag: -ingester.ring.instance-interface-names
  [instance_interface_names: <list of strings> | default = [<private network interfaces>]]

  # (advanced) Port to advertise in the ring (defaults to
  # -server.grpc-listen-port).
  # CLI flag: -ingester.ring.instance-port
  [instance_port: <int> | default = 0]

  # (advanced) IP address to advertise in the ring. Default is auto-detected.
  # CLI flag: -ingester.ring.instance-addr
  [instance_addr: <string> | default = ""]

  # (advanced) Enable using a IPv6 instance address. (default false)
  # CLI flag: -ingester.ring.instance-enable-ipv6
  [instance_enable_ipv6: <boolean> | default = false]

  # (advanced) The availability zone where this instance is running.
  # CLI flag: -ingester.ring.instance-availability-zone
  [instance_availability_zone: <string> | default = ""]

  # (advanced) Unregister from the ring upon clean shutdown. It can be useful to
  # disable for rolling restarts with consistent naming.
  # CLI flag: -ingester.ring.unregister-on-shutdown
  [unregister_on_shutdown: <boolean> | default = true]

  # (advanced) Observe tokens after generating to resolve collisions. Useful
  # when using gossiping ring.
  # CLI flag: -ingester.ring.observe-period
  [observe_period: <duration> | default = 0s]

  # (advanced) Minimum duration to wait after the internal readiness checks have
  # passed but before succeeding the readiness endpoint. This is used to
  # slowdown deployment controllers (eg. Kubernetes) after an instance is ready
  # and before they proceed with a rolling update, to give the rest of the
  # cluster instances enough time to receive ring updates.
  # CLI flag: -ingester.ring.min-ready-duration
  [min_ready_duration: <duration> | default = 15s]

  # (advanced) Duration to sleep for before exiting, to ensure metrics are
  # scraped.
  # CLI flag: -ingester.ring.final-sleep
  [final_sleep: <duration> | default = 0s]

  # (advanced) Specifies the strategy used for generating tokens for ingesters.
  # Supported values are: random,spread-minimizing.
  # CLI flag: -ingester.ring.token-generation-strategy
  [token_generation_strategy: <string> | default = "random"]

  # (advanced) True to allow this ingester registering tokens in the ring only
  # after all previous ingesters (with ID lower than the current one) have
  # already been registered. This configuration option is supported only when
  # the token generation strategy is set to "spread-minimizing".
  # CLI flag: -ingester.ring.spread-minimizing-join-ring-in-order
  [spread_minimizing_join_ring_in_order: <boolean> | default = false]

  # (advanced) Comma-separated list of zones in which spread minimizing strategy
  # is used for token generation. This value must include all zones in which
  # ingesters are deployed, and must not change over time. This configuration is
  # used only when "token-generation-strategy" is set to "spread-minimizing".
  # CLI flag: -ingester.ring.spread-minimizing-zones
  [spread_minimizing_zones: <string> | default = ""]

partition_ring:
  # The key-value store used to share the hash ring across multiple instances.
  # This option needs be set on ingesters, distributors, queriers, and rulers
  # when running in microservices mode.
  kvstore:
    # Backend storage to use for the ring. Supported values are: consul, etcd,
    # inmemory, memberlist, multi.
    # CLI flag: -ingester.partition-ring.store
    [store: <string> | default = "memberlist"]

    # (advanced) The prefix for the keys in the store. Should end with a /.
    # CLI flag: -ingester.partition-ring.prefix
    [prefix: <string> | default = "collectors/"]

    # The consul block configures the consul client.
    # The CLI flags prefix for this block configuration is:
    # ingester.partition-ring
    [consul: <consul>]

    # The etcd block configures the etcd client.
    # The CLI flags prefix for this block configuration is:
    # ingester.partition-ring
    [etcd: <etcd>]

    multi:
      # (advanced) Primary backend storage used by multi-client.
      # CLI flag: -ingester.partition-ring.multi.primary
      [primary: <string> | default = ""]

      # (advanced) Secondary backend storage used by multi-client.
      # CLI flag: -ingester.partition-ring.multi.secondary
      [secondary: <string> | default = ""]

      # (advanced) Mirror writes to the secondary store.
      # CLI flag: -ingester.partition-ring.multi.mirror-enabled
      [mirror_enabled: <boolean> | default = false]

      # (advanced) Timeout for storing a value to the secondary store.
      # CLI flag: -ingester.partition-ring.multi.mirror-timeout
      [mirror_timeout: <duration> | default = 2s]

  # Minimum number of owners to wait before a PENDING partition gets switched to
  # ACTIVE.
  # CLI flag: -ingester.partition-ring.min-partition-owners-count
  [min_partition_owners_count: <int> | default = 1]

  # How long the minimum number of owners are enforced before a PENDING
  # partition gets switched to ACTIVE.
  # CLI flag: -ingester.partition-ring.min-partition-owners-duration
  [min_partition_owners_duration: <duration> | default = 10s]

  # How long to wait before an INACTIVE partition is eligible for deletion. The
  # partition is deleted only if it has been in INACTIVE state for at least the
  # configured duration and it has no owners registered. A value of 0 disables
  # partitions deletion.
  # CLI flag: -ingester.partition-ring.delete-inactive-partition-after
  [delete_inactive_partition_after: <duration> | default = 13h]

# (advanced) Period at which metadata we have not seen will remain in memory
# before being deleted.
# CLI flag: -ingester.metadata-retain-period
[metadata_retain_period: <duration> | default = 10m]

# (advanced) Period with which to update the per-tenant ingestion rates.
# CLI flag: -ingester.rate-update-period
[rate_update_period: <duration> | default = 15s]

# (advanced) Enable tracking of active series and export them as metrics.
# CLI flag: -ingester.active-series-metrics-enabled
[active_series_metrics_enabled: <boolean> | default = true]

# (advanced) How often to update active series metrics.
# CLI flag: -ingester.active-series-metrics-update-period
[active_series_metrics_update_period: <duration> | default = 1m]

# (advanced) After what time a series is considered to be inactive.
# CLI flag: -ingester.active-series-metrics-idle-timeout
[active_series_metrics_idle_timeout: <duration> | default = 20m]

# (experimental) Period with which to update the per-tenant TSDB configuration.
# CLI flag: -ingester.tsdb-config-update-period
[tsdb_config_update_period: <duration> | default = 15s]

instance_limits:
  # (advanced) Max ingestion rate (samples/sec) that ingester will accept. This
  # limit is per-ingester, not per-tenant. Additional push requests will be
  # rejected. Current ingestion rate is computed as exponentially weighted
  # moving average, updated every second. 0 = unlimited.
  # CLI flag: -ingester.instance-limits.max-ingestion-rate
  [max_ingestion_rate: <float> | default = 0]

  # (advanced) Max tenants that this ingester can hold. Requests from additional
  # tenants will be rejected. 0 = unlimited.
  # CLI flag: -ingester.instance-limits.max-tenants
  [max_tenants: <int> | default = 0]

  # (advanced) Max series that this ingester can hold (across all tenants).
  # Requests to create additional series will be rejected. 0 = unlimited.
  # CLI flag: -ingester.instance-limits.max-series
  [max_series: <int> | default = 0]

  # (advanced) Max inflight push requests that this ingester can handle (across
  # all tenants). Additional requests will be rejected. 0 = unlimited.
  # CLI flag: -ingester.instance-limits.max-inflight-push-requests
  [max_inflight_push_requests: <int> | default = 30000]

  # (advanced) The sum of the request sizes in bytes of inflight push requests
  # that this ingester can handle. This limit is per-ingester, not per-tenant.
  # Additional requests will be rejected. 0 = unlimited.
  # CLI flag: -ingester.instance-limits.max-inflight-push-requests-bytes
  [max_inflight_push_requests_bytes: <int> | default = 0]

# (advanced) Comma-separated list of metric names, for which the
# -ingester.max-global-series-per-metric limit will be ignored. Does not affect
# the -ingester.max-global-series-per-user limit.
# CLI flag: -ingester.ignore-series-limit-for-metric-names
[ignore_series_limit_for_metric_names: <string> | default = ""]

# (advanced) CPU utilization limit, as CPU cores, for CPU/memory utilization
# based read request limiting. Use 0 to disable it.
# CLI flag: -ingester.read-path-cpu-utilization-limit
[read_path_cpu_utilization_limit: <float> | default = 0]

# (advanced) Memory limit, in bytes, for CPU/memory utilization based read
# request limiting. Use 0 to disable it.
# CLI flag: -ingester.read-path-memory-utilization-limit
[read_path_memory_utilization_limit: <int> | default = 0]

# (advanced) Each error will be logged once in this many times. Use 0 to log all
# of them.
# CLI flag: -ingester.error-sample-rate
[error_sample_rate: <int> | default = 10]

# (experimental) When enabled, only series currently owned by ingester according
# to the ring are used when checking user per-tenant series limit.
# CLI flag: -ingester.use-ingester-owned-series-for-limits
[use_ingester_owned_series_for_limits: <boolean> | default = false]

# (experimental) This option enables tracking of ingester-owned series based on
# ring state, even if -ingester.use-ingester-owned-series-for-limits is
# disabled.
# CLI flag: -ingester.track-ingester-owned-series
[track_ingester_owned_series: <boolean> | default = false]

# (experimental) How often to check for ring changes and possibly recompute
# owned series as a result of detected change.
# CLI flag: -ingester.owned-series-update-interval
[owned_series_update_interval: <duration> | default = 15s]

push_circuit_breaker:
  # (experimental) Enable circuit breaking when making requests to ingesters
  # CLI flag: -ingester.push-circuit-breaker.enabled
  [enabled: <boolean> | default = false]

  # (experimental) Max percentage of requests that can fail over period before
  # the circuit breaker opens
  # CLI flag: -ingester.push-circuit-breaker.failure-threshold-percentage
  [failure_threshold_percentage: <int> | default = 10]

  # (experimental) How many requests must have been executed in period for the
  # circuit breaker to be eligible to open for the rate of failures
  # CLI flag: -ingester.push-circuit-breaker.failure-execution-threshold
  [failure_execution_threshold: <int> | default = 100]

  # (experimental) Moving window of time that the percentage of failed requests
  # is computed over
  # CLI flag: -ingester.push-circuit-breaker.thresholding-period
  [thresholding_period: <duration> | default = 1m]

  # (experimental) How long the circuit breaker will stay in the open state
  # before allowing some requests
  # CLI flag: -ingester.push-circuit-breaker.cooldown-period
  [cooldown_period: <duration> | default = 10s]

  # (experimental) Duration, in seconds, after an initial request that an
  # activated circuit breaker should wait before becoming effectively active.
  # During this time, neither failures nor successes are counted.
  # CLI flag: -ingester.push-circuit-breaker.initial-delay
  [initial_delay: <duration> | default = 0s]

  # (experimental) The maximum duration of an ingester's request before it
  # triggers a timeout. This configuration is used for circuit breakers only,
  # and its timeouts aren't reported as errors.
  # CLI flag: -ingester.push-circuit-breaker.request-timeout
  [request_timeout: <duration> | default = 2s]

read_circuit_breaker:
  # (experimental) Enable circuit breaking when making requests to ingesters
  # CLI flag: -ingester.read-circuit-breaker.enabled
  [enabled: <boolean> | default = false]

  # (experimental) Max percentage of requests that can fail over period before
  # the circuit breaker opens
  # CLI flag: -ingester.read-circuit-breaker.failure-threshold-percentage
  [failure_threshold_percentage: <int> | default = 10]

  # (experimental) How many requests must have been executed in period for the
  # circuit breaker to be eligible to open for the rate of failures
  # CLI flag: -ingester.read-circuit-breaker.failure-execution-threshold
  [failure_execution_threshold: <int> | default = 100]

  # (experimental) Moving window of time that the percentage of failed requests
  # is computed over
  # CLI flag: -ingester.read-circuit-breaker.thresholding-period
  [thresholding_period: <duration> | default = 1m]

  # (experimental) How long the circuit breaker will stay in the open state
  # before allowing some requests
  # CLI flag: -ingester.read-circuit-breaker.cooldown-period
  [cooldown_period: <duration> | default = 10s]

  # (experimental) Duration, in seconds, after an initial request that an
  # activated circuit breaker should wait before becoming effectively active.
  # During this time, neither failures nor successes are counted.
  # CLI flag: -ingester.read-circuit-breaker.initial-delay
  [initial_delay: <duration> | default = 0s]

  # (experimental) The maximum duration of an ingester's request before it
  # triggers a timeout. This configuration is used for circuit breakers only,
  # and its timeouts aren't reported as errors.
  # CLI flag: -ingester.read-circuit-breaker.request-timeout
  [request_timeout: <duration> | default = 30s]

rejection_prioritizer:
  # (experimental) The interval at which the rejection threshold is calibrated
  # CLI flag: -ingester.rejection-prioritizer.calibration-interval
  [calibration_interval: <duration> | default = 1s]

push_reactive_limiter:
  # (experimental) Enable reactive limiting when making requests to a service
  # CLI flag: -ingester.push-reactive-limiter.enabled
  [enabled: <boolean> | default = false]

  # (experimental) Minimum inflight requests limit
  # CLI flag: -ingester.push-reactive-limiter.min-limit
  [min_limit: <int> | default = 2]

  # (experimental) Maximum inflight requests limit
  # CLI flag: -ingester.push-reactive-limiter.max-limit
  [max_limit: <int> | default = 200]

  # (experimental) Initial inflight requests limit
  # CLI flag: -ingester.push-reactive-limiter.initial-limit
  [initial_limit: <int> | default = 20]

  # (experimental) The maximum inflight limit as a multiple of current inflight
  # requests
  # CLI flag: -ingester.push-reactive-limiter.max-limit-factor
  [max_limit_factor: <float> | default = 5]

  # (experimental) Logarithmic decay applied to the maxLimitFactor based on
  # current inflight requests
  # CLI flag: -ingester.push-reactive-limiter.max-limit-factor-decay
  [max_limit_factor_decay: <float> | default = 1]

  # (experimental) Minimum limit factor when max-limit-factor-decay is applied
  # CLI flag: -ingester.push-reactive-limiter.min-limit-factor
  [min_limit_factor: <float> | default = 1.2]

  # (experimental) Minimum duration of the window that is used to collect recent
  # response time samples
  # CLI flag: -ingester.push-reactive-limiter.recent-window-min-duration
  [recent_window_min_duration: <duration> | default = 1s]

  # (experimental) Maximum duration of the window that is used to collect recent
  # response time samples
  # CLI flag: -ingester.push-reactive-limiter.recent-window-max-duration
  [recent_window_max_duration: <duration> | default = 30s]

  # (experimental) Minimum number of samples that must be recorded in the recent
  # window before updating the limit
  # CLI flag: -ingester.push-reactive-limiter.recent-window-min-samples
  [recent_window_min_samples: <int> | default = 50]

  # (experimental) The quantile of recent recorded response times to consider
  # when adjusting the concurrency limit
  # CLI flag: -ingester.push-reactive-limiter.recent-quantile
  [recent_quantile: <float> | default = 0.9]

  # (experimental) The average age of baseline samples aggregated recent samples
  # are added to
  # CLI flag: -ingester.push-reactive-limiter.baseline-window-age
  [baseline_window_age: <int> | default = 10]

  # (experimental) How many recent limit and inflight time measurements are
  # stored to detect whether increases in limits correlate with increases in
  # inflight times
  # CLI flag: -ingester.push-reactive-limiter.correlation-window
  [correlation_window: <int> | default = 50]

  # (experimental) The number of allowed queued requests, as a multiple of
  # current inflight requests, after which rejections start
  # CLI flag: -ingester.push-reactive-limiter.initial-rejection-factor
  [initial_rejection_factor: <float> | default = 2]

  # (experimental) The number of allowed queued requests, as a multiple of
  # current inflight requests, after which all requests are rejected
  # CLI flag: -ingester.push-reactive-limiter.max-rejection-factor
  [max_rejection_factor: <float> | default = 3]

read_reactive_limiter:
  # (experimental) Enable reactive limiting when making requests to a service
  # CLI flag: -ingester.read-reactive-limiter.enabled
  [enabled: <boolean> | default = false]

  # (experimental) Minimum inflight requests limit
  # CLI flag: -ingester.read-reactive-limiter.min-limit
  [min_limit: <int> | default = 2]

  # (experimental) Maximum inflight requests limit
  # CLI flag: -ingester.read-reactive-limiter.max-limit
  [max_limit: <int> | default = 200]

  # (experimental) Initial inflight requests limit
  # CLI flag: -ingester.read-reactive-limiter.initial-limit
  [initial_limit: <int> | default = 20]

  # (experimental) The maximum inflight limit as a multiple of current inflight
  # requests
  # CLI flag: -ingester.read-reactive-limiter.max-limit-factor
  [max_limit_factor: <float> | default = 5]

  # (experimental) Logarithmic decay applied to the maxLimitFactor based on
  # current inflight requests
  # CLI flag: -ingester.read-reactive-limiter.max-limit-factor-decay
  [max_limit_factor_decay: <float> | default = 1]

  # (experimental) Minimum limit factor when max-limit-factor-decay is applied
  # CLI flag: -ingester.read-reactive-limiter.min-limit-factor
  [min_limit_factor: <float> | default = 1.2]

  # (experimental) Minimum duration of the window that is used to collect recent
  # response time samples
  # CLI flag: -ingester.read-reactive-limiter.recent-window-min-duration
  [recent_window_min_duration: <duration> | default = 1s]

  # (experimental) Maximum duration of the window that is used to collect recent
  # response time samples
  # CLI flag: -ingester.read-reactive-limiter.recent-window-max-duration
  [recent_window_max_duration: <duration> | default = 30s]

  # (experimental) Minimum number of samples that must be recorded in the recent
  # window before updating the limit
  # CLI flag: -ingester.read-reactive-limiter.recent-window-min-samples
  [recent_window_min_samples: <int> | default = 50]

  # (experimental) The quantile of recent recorded response times to consider
  # when adjusting the concurrency limit
  # CLI flag: -ingester.read-reactive-limiter.recent-quantile
  [recent_quantile: <float> | default = 0.9]

  # (experimental) The average age of baseline samples aggregated recent samples
  # are added to
  # CLI flag: -ingester.read-reactive-limiter.baseline-window-age
  [baseline_window_age: <int> | default = 10]

  # (experimental) How many recent limit and inflight time measurements are
  # stored to detect whether increases in limits correlate with increases in
  # inflight times
  # CLI flag: -ingester.read-reactive-limiter.correlation-window
  [correlation_window: <int> | default = 50]

  # (experimental) The number of allowed queued requests, as a multiple of
  # current inflight requests, after which rejections start
  # CLI flag: -ingester.read-reactive-limiter.initial-rejection-factor
  [initial_rejection_factor: <float> | default = 2]

  # (experimental) The number of allowed queued requests, as a multiple of
  # current inflight requests, after which all requests are rejected
  # CLI flag: -ingester.read-reactive-limiter.max-rejection-factor
  [max_rejection_factor: <float> | default = 3]

querier

The querier block configures the querier.

# (advanced) The time after which a metric should be queried from storage and
# not just ingesters. 0 means all queries are sent to store. If this option is
# enabled, the time range of the query sent to the store-gateway will be
# manipulated to ensure the query end is not more recent than 'now -
# query-store-after'.
# CLI flag: -querier.query-store-after
[query_store_after: <duration> | default = 12h]

store_gateway_client:
  # (advanced) gRPC client max receive message size (bytes).
  # CLI flag: -querier.store-gateway-client.grpc-max-recv-msg-size
  [max_recv_msg_size: <int> | default = 104857600]

  # (advanced) gRPC client max send message size (bytes).
  # CLI flag: -querier.store-gateway-client.grpc-max-send-msg-size
  [max_send_msg_size: <int> | default = 104857600]

  # (advanced) Use compression when sending messages. Supported values are:
  # 'gzip', 'snappy' and '' (disable compression)
  # CLI flag: -querier.store-gateway-client.grpc-compression
  [grpc_compression: <string> | default = ""]

  # (advanced) Rate limit for gRPC client; 0 means disabled.
  # CLI flag: -querier.store-gateway-client.grpc-client-rate-limit
  [rate_limit: <float> | default = 0]

  # (advanced) Rate limit burst for gRPC client.
  # CLI flag: -querier.store-gateway-client.grpc-client-rate-limit-burst
  [rate_limit_burst: <int> | default = 0]

  # (advanced) Enable backoff and retry when we hit rate limits.
  # CLI flag: -querier.store-gateway-client.backoff-on-ratelimits
  [backoff_on_ratelimits: <boolean> | default = false]

  backoff_config:
    # (advanced) Minimum delay when backing off.
    # CLI flag: -querier.store-gateway-client.backoff-min-period
    [min_period: <duration> | default = 100ms]

    # (advanced) Maximum delay when backing off.
    # CLI flag: -querier.store-gateway-client.backoff-max-period
    [max_period: <duration> | default = 10s]

    # (advanced) Number of times to backoff and retry before failing.
    # CLI flag: -querier.store-gateway-client.backoff-retries
    [max_retries: <int> | default = 10]

  # (experimental) Initial stream window size. Values less than the default are
  # not supported and are ignored. Setting this to a value other than the
  # default disables the BDP estimator.
  # CLI flag: -querier.store-gateway-client.initial-stream-window-size
  [initial_stream_window_size: <int> | default = 63KiB1023B]

  # (experimental) Initial connection window size. Values less than the default
  # are not supported and are ignored. Setting this to a value other than the
  # default disables the BDP estimator.
  # CLI flag: -querier.store-gateway-client.initial-connection-window-size
  [initial_connection_window_size: <int> | default = 63KiB1023B]

  # (advanced) Enable TLS in the gRPC client. This flag needs to be enabled when
  # any other TLS flag is set. If set to false, insecure connection to gRPC
  # server will be used.
  # CLI flag: -querier.store-gateway-client.tls-enabled
  [tls_enabled: <boolean> | default = false]

  # (advanced) Path to the client certificate, which will be used for
  # authenticating with the server. Also requires the key path to be configured.
  # CLI flag: -querier.store-gateway-client.tls-cert-path
  [tls_cert_path: <string> | default = ""]

  # (advanced) Path to the key for the client certificate. Also requires the
  # client certificate to be configured.
  # CLI flag: -querier.store-gateway-client.tls-key-path
  [tls_key_path: <string> | default = ""]

  # (advanced) Path to the CA certificates to validate server certificate
  # against. If not set, the host's root CA certificates are used.
  # CLI flag: -querier.store-gateway-client.tls-ca-path
  [tls_ca_path: <string> | default = ""]

  # (advanced) Override the expected name on the server certificate.
  # CLI flag: -querier.store-gateway-client.tls-server-name
  [tls_server_name: <string> | default = ""]

  # (advanced) Skip validating server certificate.
  # CLI flag: -querier.store-gateway-client.tls-insecure-skip-verify
  [tls_insecure_skip_verify: <boolean> | default = false]

  # (advanced) Override the default cipher suite list (separated by commas).
  # Allowed values:
  #
  # Secure Ciphers:
  # - TLS_AES_128_GCM_SHA256
  # - TLS_AES_256_GCM_SHA384
  # - TLS_CHACHA20_POLY1305_SHA256
  # - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
  # - TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
  # - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
  # - TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
  # - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
  # - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
  # - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
  # - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
  # - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
  # - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
  #
  # Insecure Ciphers:
  # - TLS_RSA_WITH_RC4_128_SHA
  # - TLS_RSA_WITH_3DES_EDE_CBC_SHA
  # - TLS_RSA_WITH_AES_128_CBC_SHA
  # - TLS_RSA_WITH_AES_256_CBC_SHA
  # - TLS_RSA_WITH_AES_128_CBC_SHA256
  # - TLS_RSA_WITH_AES_128_GCM_SHA256
  # - TLS_RSA_WITH_AES_256_GCM_SHA384
  # - TLS_ECDHE_ECDSA_WITH_RC4_128_SHA
  # - TLS_ECDHE_RSA_WITH_RC4_128_SHA
  # - TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
  # - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
  # - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
  # CLI flag: -querier.store-gateway-client.tls-cipher-suites
  [tls_cipher_suites: <string> | default = ""]

  # (advanced) Override the default minimum TLS version. Allowed values:
  # VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13
  # CLI flag: -querier.store-gateway-client.tls-min-version
  [tls_min_version: <string> | default = ""]

  # (advanced) The maximum amount of time to establish a connection. A value of
  # 0 means default gRPC client connect timeout and backoff.
  # CLI flag: -querier.store-gateway-client.connect-timeout
  [connect_timeout: <duration> | default = 5s]

  # (advanced) Initial backoff delay after first connection failure. Only
  # relevant if ConnectTimeout > 0.
  # CLI flag: -querier.store-gateway-client.connect-backoff-base-delay
  [connect_backoff_base_delay: <duration> | default = 1s]

  # (advanced) Maximum backoff delay when establishing a connection. Only
  # relevant if ConnectTimeout > 0.
  # CLI flag: -querier.store-gateway-client.connect-backoff-max-delay
  [connect_backoff_max_delay: <duration> | default = 5s]

  cluster_validation:
    # (experimental) Primary cluster validation label.
    # CLI flag: -querier.store-gateway-client.cluster-validation.label
    [label: <string> | default = ""]

  # (experimental) The grace period for health checks. If a store-gateway
  # connection consistently fails health checks for this period, any open
  # connections are closed. The querier will attempt to reconnect to the
  # store-gateway if a subsequent request is made to the store-gateway. Set to 0
  # to immediately remove store-gateway connections on the first health check
  # failure.
  # CLI flag: -querier.store-gateway-client.health-check-grace-period
  [health_check_grace_period: <duration> | default = 0s]

# (advanced) Fetch in-memory series from the minimum set of required ingesters,
# selecting only ingesters which may have received series since
# -querier.query-ingesters-within. If this setting is false or
# -querier.query-ingesters-within is '0', queriers always query all ingesters
# (ingesters shuffle sharding on read path is disabled).
# CLI flag: -querier.shuffle-sharding-ingesters-enabled
[shuffle_sharding_ingesters_enabled: <boolean> | default = true]

# (experimental) Comma-separated list of availability zones to prefer when
# querying ingesters and store-gateways. All zones in the list are given equal
# priority.
# CLI flag: -querier.prefer-availability-zones
[prefer_availability_zones: <string> | default = ""]

# (advanced) Number of series to buffer per ingester when streaming chunks from
# ingesters.
# CLI flag: -querier.streaming-chunks-per-ingester-buffer-size
[streaming_chunks_per_ingester_series_buffer_size: <int> | default = 256]

# (advanced) Number of series to buffer per store-gateway when streaming chunks
# from store-gateways.
# CLI flag: -querier.streaming-chunks-per-store-gateway-buffer-size
[streaming_chunks_per_store_gateway_series_buffer_size: <int> | default = 256]

# (advanced) If true, when querying ingesters, only the minimum required
# ingesters required to reach quorum will be queried initially, with other
# ingesters queried only if needed due to failures from the initial set of
# ingesters. Enabling this option reduces resource consumption for the happy
# path at the cost of increased latency for the unhappy path.
# CLI flag: -querier.minimize-ingester-requests
[minimize_ingester_requests: <boolean> | default = true]

# (advanced) Delay before initiating requests to further ingesters when request
# minimization is enabled and the initially selected set of ingesters have not
# all responded. Ignored if -querier.minimize-ingester-requests is not enabled.
# CLI flag: -querier.minimize-ingester-requests-hedging-delay
[minimize_ingester_requests_hedging_delay: <duration> | default = 3s]

# (experimental) Query engine to use, either 'prometheus' or 'mimir'
# CLI flag: -querier.query-engine
[query_engine: <string> | default = "mimir"]

# (experimental) If set to true and the Mimir query engine is in use, fall back
# to using the Prometheus query engine for any queries not supported by the
# Mimir query engine.
# CLI flag: -querier.enable-query-engine-fallback
[enable_query_engine_fallback: <boolean> | default = true]

# (deprecated) If set to true, the header 'X-Filter-Queryables' can be used to
# filter down the list of queryables that shall be used. This is useful to test
# and monitor single queryables in isolation. Deprecated: has no effect.
# CLI flag: -querier.filter-queryables-enabled
[filter_queryables_enabled: <boolean> | default = false]

# (advanced) Maximum number of remote read queries that can be executed
# concurrently. 0 or negative values mean unlimited concurrency.
# CLI flag: -querier.max-concurrent-remote-read-queries
[max_concurrent_remote_read_queries: <int> | default = 2]

# The number of workers running in each querier process. This setting limits the
# maximum number of concurrent queries in each querier. The minimum value is
# four; lower values are ignored and set to the minimum
# CLI flag: -querier.max-concurrent
[max_concurrent: <int> | default = 20]

# The timeout for a query. This config option should be set on query-frontend
# too when query sharding is enabled. This also applies to queries evaluated by
# the ruler (internally or remotely).
# CLI flag: -querier.timeout
[timeout: <duration> | default = 2m]

# Maximum number of samples a single query can load into memory. This config
# option should be set on query-frontend too when query sharding is enabled.
# CLI flag: -querier.max-samples
[max_samples: <int> | default = 50000000]

# (advanced) The default evaluation interval or step size for subqueries. This
# config option should be set on query-frontend too when query sharding is
# enabled.
# CLI flag: -querier.default-evaluation-interval
[default_evaluation_interval: <duration> | default = 1m]

# (advanced) Time since the last sample after which a time series is considered
# stale and ignored by expression evaluations. This config option should be set
# on query-frontend too when query sharding is enabled.
# CLI flag: -querier.lookback-delta
[lookback_delta: <duration> | default = 5m]

# (experimental) Enable the experimental PromQL feature for delayed name removal
# in the Prometheus engine. Note that this only applies when the Prometheus
# engine is selected or used as fallback from the Mimir Query Engine.
# CLI flag: -querier.enable-delayed-name-removal-prometheus-engine
[enable_delayed_name_removal_prometheus_engine: <boolean> | default = false]

mimir_query_engine:
  # (experimental) Enable pruning query expressions that are toggled off with
  # constants.
  # CLI flag: -querier.mimir-query-engine.enable-prune-toggles
  [enable_prune_toggles: <boolean> | default = true]

  # (experimental) Enable common subexpression elimination when evaluating
  # queries.
  # CLI flag: -querier.mimir-query-engine.enable-common-subexpression-elimination
  [enable_common_subexpression_elimination: <boolean> | default = true]

  # (experimental) Enable subset selector elimination when evaluating queries.
  # CLI flag: -querier.mimir-query-engine.enable-subset-selector-elimination
  [enable_subset_selector_elimination: <boolean> | default = false]

  # (experimental) Enable deduplication of range vector selectors in range
  # queries as part of common subexpression elimination. Requires common
  # subexpression elimination to be enabled.
  # CLI flag: -querier.mimir-query-engine.enable-range-query-range-vector-common-subexpression-elimination
  [enable_range_query_range_vector_common_subexpression_elimination: <boolean> | default = false]

  # (experimental) Enable generating selectors for one side of a binary
  # expression based on results from the other side.
  # CLI flag: -querier.mimir-query-engine.enable-narrow-binary-selectors
  [enable_narrow_binary_selectors: <boolean> | default = false]

  # (experimental) Enable eliminating redundant DeduplicateAndMerge nodes from
  # the query plan when it can be proven that each input series produces a
  # unique output series.
  # CLI flag: -querier.mimir-query-engine.enable-eliminate-deduplicate-and-merge
  [enable_eliminate_deduplicate_and_merge: <boolean> | default = true]

  # (experimental) Enable eliminating duplicate or redundant matchers that are
  # part of selector expressions.
  # CLI flag: -querier.mimir-query-engine.enable-reduce-matchers
  [enable_reduce_matchers: <boolean> | default = true]

  # (experimental) Enable projection pushdown to only fetch labels required for
  # the query from storage.
  # CLI flag: -querier.mimir-query-engine.enable-projection-pushdown
  [enable_projection_pushdown: <boolean> | default = false]

  # (experimental) Enable computing multiple aggregations over the same data
  # without buffering. Requires common subexpression elimination to be enabled.
  # CLI flag: -querier.mimir-query-engine.enable-multi-aggregation
  [enable_multi_aggregation: <boolean> | default = true]

  # (experimental) Enable removing expressions that are guaranteed to produce no
  # results.
  # CLI flag: -querier.mimir-query-engine.enable-remove-statically-empty-expressions
  [enable_remove_statically_empty_expressions: <boolean> | default = true]

  range_vector_splitting:
    # (experimental) Enable splitting function over range vectors queries into
    # smaller blocks for caching.
    # CLI flag: -querier.mimir-query-engine.range-vector-splitting.enabled
    [enabled: <boolean> | default = false]

    # (experimental) Time interval used for splitting function over range
    # vectors queries into cacheable blocks.
    # CLI flag: -querier.mimir-query-engine.range-vector-splitting.split-interval
    [split_interval: <duration> | default = 2h]

    intermediate_results_cache:
      # Backend for intermediate results cache, if not empty. Supported values:
      # memcached.
      # CLI flag: -querier.mimir-query-engine.range-vector-splitting.backend
      [backend: <string> | default = ""]

      # The memcached block configures the Memcached-based caching backend.
      # The CLI flags prefix for this block configuration is:
      # querier.mimir-query-engine.range-vector-splitting
      [memcached: <memcached>]

      # Enable cache compression, if not empty. Supported values are: snappy.
      # CLI flag: -querier.mimir-query-engine.range-vector-splitting.compression
      [compression: <string> | default = ""]

ring:
  # The key-value store used to share the hash ring across multiple instances.
  kvstore:
    # Backend storage to use for the ring. Supported values are: consul, etcd,
    # inmemory, memberlist, multi.
    # CLI flag: -querier.ring.store
    [store: <string> | default = "memberlist"]

    # (advanced) The prefix for the keys in the store. Should end with a /.
    # CLI flag: -querier.ring.prefix
    [prefix: <string> | default = "collectors/"]

    # The consul block configures the consul client.
    # The CLI flags prefix for this block configuration is: querier.ring
    [consul: <consul>]

    # The etcd block configures the etcd client.
    # The CLI flags prefix for this block configuration is: querier.ring
    [etcd: <etcd>]

    multi:
      # (advanced) Primary backend storage used by multi-client.
      # CLI flag: -querier.ring.multi.primary
      [primary: <string> | default = ""]

      # (advanced) Secondary backend storage used by multi-client.
      # CLI flag: -querier.ring.multi.secondary
      [secondary: <string> | default = ""]

      # (advanced) Mirror writes to the secondary store.
      # CLI flag: -querier.ring.multi.mirror-enabled
      [mirror_enabled: <boolean> | default = false]

      # (advanced) Timeout for storing a value to the secondary store.
      # CLI flag: -querier.ring.multi.mirror-timeout
      [mirror_timeout: <duration> | default = 2s]

  # (advanced) Period at which to heartbeat to the ring.
  # CLI flag: -querier.ring.heartbeat-period
  [heartbeat_period: <duration> | default = 15s]

  # (advanced) Heartbeat timeout after which Mimir marks queriers as unhealthy
  # in the ring.
  # CLI flag: -querier.ring.heartbeat-timeout
  [heartbeat_timeout: <duration> | default = 1m]

  # (advanced) Instance ID to register in the ring.
  # CLI flag: -querier.ring.instance-id
  [instance_id: <string> | default = "<hostname>"]

  # List of network interface names to look up when finding the instance IP
  # address.
  # CLI flag: -querier.ring.instance-interface-names
  [instance_interface_names: <list of strings> | default = [<private network interfaces>]]

  # (advanced) Port to advertise in the ring (defaults to
  # -server.grpc-listen-port).
  # CLI flag: -querier.ring.instance-port
  [instance_port: <int> | default = 0]

  # (advanced) IP address to advertise in the ring. Default is auto-detected.
  # CLI flag: -querier.ring.instance-addr
  [instance_addr: <string> | default = ""]

  # (advanced) Enable using an IPv6 instance address.
  # CLI flag: -querier.ring.instance-enable-ipv6
  [instance_enable_ipv6: <boolean> | default = false]

  # (advanced) Number of consecutive timeout periods after which Mimir
  # automatically removes an unhealthy instance in the ring. Set to 0 to disable
  # auto-forget.
  # CLI flag: -querier.ring.auto-forget-unhealthy-periods
  [auto_forget_unhealthy_periods: <int> | default = 10]

frontend

The frontend block configures the query-frontend.

# Log queries that are slower than the specified duration. Set to 0 to disable.
# Set to < 0 to enable on all queries.
# CLI flag: -query-frontend.log-queries-longer-than
[log_queries_longer_than: <duration> | default = 0s]

# (advanced) Comma-separated list of request header names to include in query
# logs. Applies to both query stats and slow queries logs.
# CLI flag: -query-frontend.log-query-request-headers
[log_query_request_headers: <string> | default = ""]

# (advanced) Max body size for downstream prometheus.
# CLI flag: -query-frontend.max-body-size
[max_body_size: <int> | default = 10485760]

# (advanced) False to disable query statistics tracking. When enabled, a message
# with some statistics is logged for every query.
# CLI flag: -query-frontend.query-stats-enabled
[query_stats_enabled: <boolean> | default = true]

# (experimental) Timeout for writing active series responses. 0 means the value
# from `-server.http-write-timeout` is used.
# CLI flag: -query-frontend.active-series-write-timeout
[active_series_write_timeout: <duration> | default = 5m]

# Address of the query-scheduler component, in host:port format. The host should
# resolve to all query-scheduler instances. This option should be set only when
# query-scheduler component is in use and
# -query-scheduler.service-discovery-mode is set to 'dns'.
# CLI flag: -query-frontend.scheduler-address
[scheduler_address: <string> | default = ""]

# (advanced) How often to resolve the scheduler-address, in order to look for
# new query-scheduler instances.
# CLI flag: -query-frontend.scheduler-dns-lookup-period
[scheduler_dns_lookup_period: <duration> | default = 10s]

# (advanced) Number of concurrent workers forwarding queries to single
# query-scheduler.
# CLI flag: -query-frontend.scheduler-worker-concurrency
[scheduler_worker_concurrency: <int> | default = 5]

# Configures the gRPC client used to communicate between the query-frontends and
# the query-schedulers.
# The CLI flags prefix for this block configuration is:
# query-frontend.grpc-client-config
[grpc_client_config: <grpc_client>]

# (advanced) List of network interface names to look up when finding the
# instance IP address. This address is sent to query-scheduler and querier,
# which uses it to send the query response back to query-frontend.
# CLI flag: -query-frontend.instance-interface-names
[instance_interface_names: <list of strings> | default = [<private network interfaces>]]

# (advanced) Enable using a IPv6 instance address (default false).
# CLI flag: -query-frontend.instance-enable-ipv6
[instance_enable_ipv6: <boolean> | default = false]

# (advanced) IP address to advertise to the querier (via scheduler) (default is
# auto-detected from network interfaces).
# CLI flag: -query-frontend.instance-addr
[address: <string> | default = ""]

# (advanced) Port to advertise to querier (via scheduler) (defaults to
# server.grpc-listen-port).
# CLI flag: -query-frontend.instance-port
[port: <int> | default = 0]

# (experimental) Maximum number of series to send in a single remote execution
# response from a querier.
# CLI flag: -query-frontend.remote-execution-batch-size
[remote_execution_batch_size: <int> | default = 128]

# (experimental) Maximum number of series metadata entries to send in a single
# remote execution response from a querier.
# CLI flag: -query-frontend.remote-execution-series-metadata-batch-size
[remote_execution_series_metadata_batch_size: <int> | default = 128]

# (advanced) Split range queries by an interval and execute in parallel. You
# should use a multiple of 24 hours to optimize querying blocks. 0 to disable
# it.
# CLI flag: -query-frontend.split-queries-by-interval
[split_queries_by_interval: <duration> | default = 24h]

results_cache:
  # Backend for query-frontend results cache, if not empty. Supported values:
  # memcached.
  # CLI flag: -query-frontend.results-cache.backend
  [backend: <string> | default = ""]

  # The memcached block configures the Memcached-based caching backend.
  # The CLI flags prefix for this block configuration is:
  # query-frontend.results-cache
  [memcached: <memcached>]

  # Enable cache compression, if not empty. Supported values are: snappy.
  # CLI flag: -query-frontend.results-cache.compression
  [compression: <string> | default = ""]

# Cache query results.
# CLI flag: -query-frontend.cache-results
[cache_results: <boolean> | default = false]

# Cache non-transient errors from queries.
# CLI flag: -query-frontend.cache-errors
[cache_errors: <boolean> | default = false]

# (advanced) Maximum number of retries for a single request; beyond this, the
# downstream error is returned.
# CLI flag: -query-frontend.max-retries-per-request
[max_retries: <int> | default = 5]

# (advanced) Maximum time to wait for the query-frontend to become ready before
# rejecting requests received before the frontend was ready. 0 to disable (i.e.
# fail immediately if a request is received while the frontend is still starting
# up)
# CLI flag: -query-frontend.not-running-timeout
[not_running_timeout: <duration> | default = 2s]

# True to enable query sharding.
# CLI flag: -query-frontend.parallelize-shardable-queries
[parallelize_shardable_queries: <boolean> | default = false]

# (experimental) If set to true and the Mimir query engine is in use, use remote
# execution to evaluate queries in queriers.
# CLI flag: -query-frontend.enable-remote-execution
[enable_remote_execution: <boolean> | default = false]

# (experimental) Set to true to allow evaluating multiple query plan nodes
# within a single remote execution request to queriers.
# CLI flag: -query-frontend.enable-multiple-node-remote-execution-requests
[enable_multiple_node_remote_execution_requests: <boolean> | default = false]

# (experimental) Set to true to enable performing query sharding inside the
# Mimir query engine (MQE). This setting has no effect if sharding is disabled.
# Requires remote execution and MQE to be enabled.
# CLI flag: -query-frontend.use-mimir-query-engine-for-sharding
[use_mimir_query_engine_for_sharding: <boolean> | default = false]

# (experimental) Set to true to enable rewriting histogram queries for a more
# efficient order of execution.
# CLI flag: -query-frontend.rewrite-histogram-queries
[rewrite_histogram_queries: <boolean> | default = false]

# (experimental) Set to true to enable rewriting queries to propagate label
# matchers across binary expressions.
# CLI flag: -query-frontend.rewrite-propagate-matchers
[rewrite_propagate_matchers: <boolean> | default = false]

# (advanced) How many series a single sharded partial query should load at most.
# This is not a strict requirement guaranteed to be honoured by query sharding,
# but a hint given to the query sharding when the query execution is initially
# planned. 0 to disable cardinality-based hints.
# CLI flag: -query-frontend.query-sharding-target-series-per-shard
[query_sharding_target_series_per_shard: <int> | default = 0]

# (advanced) Comma-separated list of request header names to allow to pass
# through to the rest of the query path. This is in addition to a list of
# required headers that the read path needs.
# CLI flag: -query-frontend.extra-propagated-headers
[extra_propagated_headers: <string> | default = ""]

# Format to use when retrieving query results from queriers. Supported values:
# json, protobuf
# CLI flag: -query-frontend.query-result-response-format
[query_result_response_format: <string> | default = "protobuf"]

# (deprecated) Cache statistics of processed samples on results cache.
# Deprecated: has no effect.
# CLI flag: -query-frontend.cache-samples-processed-stats
[cache_samples_processed_stats: <boolean> | default = false]

client_cluster_validation:
  # (experimental) Primary cluster validation label.
  # CLI flag: -query-frontend.client-cluster-validation.label
  [label: <string> | default = ""]

# (experimental) Query engine to use, either 'prometheus' or 'mimir'
# CLI flag: -query-frontend.query-engine
[query_engine: <string> | default = "mimir"]

# (experimental) If set to true and the Mimir query engine is in use, fall back
# to using the Prometheus query engine for any queries not supported by the
# Mimir query engine.
# CLI flag: -query-frontend.enable-query-engine-fallback
[enable_query_engine_fallback: <boolean> | default = true]

query_scheduler

The query_scheduler block configures the query-scheduler.

# Maximum number of outstanding requests per tenant per query-scheduler.
# In-flight requests above this limit will fail with HTTP response status code
# 429.
# CLI flag: -query-scheduler.max-outstanding-requests-per-tenant
[max_outstanding_requests_per_tenant: <int> | default = 100]

# (experimental) If a querier disconnects without sending notification about
# graceful shutdown, the query-scheduler will keep the querier in the tenant's
# shard until the forget delay has passed. This feature is useful to reduce the
# blast radius when shuffle-sharding is enabled.
# CLI flag: -query-scheduler.querier-forget-delay
[querier_forget_delay: <duration> | default = 0s]

# This configures the gRPC client used to report errors back to the
# query-frontend.
# The CLI flags prefix for this block configuration is:
# query-scheduler.grpc-client-config
[grpc_client_config: <grpc_client>]

# (experimental) Service discovery mode that query-frontends and queriers use to
# find query-scheduler instances. When query-scheduler ring-based service
# discovery is enabled, this option needs be set on query-schedulers,
# query-frontends and queriers. Supported values are: dns, ring.
# CLI flag: -query-scheduler.service-discovery-mode
[service_discovery_mode: <string> | default = "dns"]

# The hash ring configuration. The query-schedulers hash ring is used for
# service discovery.
ring:
  # The key-value store used to share the hash ring across multiple instances.
  # When query-scheduler ring-based service discovery is enabled, this option
  # needs be set on query-schedulers, query-frontends and queriers.
  kvstore:
    # Backend storage to use for the ring. Supported values are: consul, etcd,
    # inmemory, memberlist, multi.
    # CLI flag: -query-scheduler.ring.store
    [store: <string> | default = "memberlist"]

    # (advanced) The prefix for the keys in the store. Should end with a /.
    # CLI flag: -query-scheduler.ring.prefix
    [prefix: <string> | default = "collectors/"]

    # The consul block configures the consul client.
    # The CLI flags prefix for this block configuration is: query-scheduler.ring
    [consul: <consul>]

    # The etcd block configures the etcd client.
    # The CLI flags prefix for this block configuration is: query-scheduler.ring
    [etcd: <etcd>]

    multi:
      # (advanced) Primary backend storage used by multi-client.
      # CLI flag: -query-scheduler.ring.multi.primary
      [primary: <string> | default = ""]

      # (advanced) Secondary backend storage used by multi-client.
      # CLI flag: -query-scheduler.ring.multi.secondary
      [secondary: <string> | default = ""]

      # (advanced) Mirror writes to the secondary store.
      # CLI flag: -query-scheduler.ring.multi.mirror-enabled
      [mirror_enabled: <boolean> | default = false]

      # (advanced) Timeout for storing a value to the secondary store.
      # CLI flag: -query-scheduler.ring.multi.mirror-timeout
      [mirror_timeout: <duration> | default = 2s]

  # (advanced) Period at which to heartbeat to the ring.
  # CLI flag: -query-scheduler.ring.heartbeat-period
  [heartbeat_period: <duration> | default = 15s]

  # (advanced) The heartbeat timeout after which query-schedulers are considered
  # unhealthy within the ring. When query-scheduler ring-based service discovery
  # is enabled, this option needs be set on query-schedulers, query-frontends
  # and queriers.
  # CLI flag: -query-scheduler.ring.heartbeat-timeout
  [heartbeat_timeout: <duration> | default = 1m]

  # (advanced) Number of consecutive timeout periods after which Mimir
  # automatically removes an unhealthy instance in the ring. Set to 0 to disable
  # auto-forget.
  # CLI flag: -query-scheduler.ring.auto-forget-unhealthy-periods
  [auto_forget_unhealthy_periods: <int> | default = 10]

  # (advanced) Instance ID to register in the ring.
  # CLI flag: -query-scheduler.ring.instance-id
  [instance_id: <string> | default = "<hostname>"]

  # List of network interface names to look up when finding the instance IP
  # address.
  # CLI flag: -query-scheduler.ring.instance-interface-names
  [instance_interface_names: <list of strings> | default = [<private network interfaces>]]

  # (advanced) Port to advertise in the ring (defaults to
  # -server.grpc-listen-port).
  # CLI flag: -query-scheduler.ring.instance-port
  [instance_port: <int> | default = 0]

  # (advanced) IP address to advertise in the ring. Default is auto-detected.
  # CLI flag: -query-scheduler.ring.instance-addr
  [instance_addr: <string> | default = ""]

  # (advanced) Enable using a IPv6 instance address. (default false)
  # CLI flag: -query-scheduler.ring.instance-enable-ipv6
  [instance_enable_ipv6: <boolean> | default = false]

# The maximum number of query-scheduler instances to use, regardless how many
# replicas are running. This option can be set only when
# -query-scheduler.service-discovery-mode is set to 'ring'. 0 to use all
# available query-scheduler instances.
# CLI flag: -query-scheduler.max-used-instances
[max_used_instances: <int> | default = 0]

ruler

The ruler block configures the ruler.

# URL of alerts return path.
# CLI flag: -ruler.external.url
[external_url: <url> | default = ]

# Configures the gRPC client used to communicate between ruler instances.
ruler_client:
  # (advanced) gRPC client max receive message size (bytes).
  # CLI flag: -ruler.client.grpc-max-recv-msg-size
  [max_recv_msg_size: <int> | default = 104857600]

  # (advanced) gRPC client max send message size (bytes).
  # CLI flag: -ruler.client.grpc-max-send-msg-size
  [max_send_msg_size: <int> | default = 104857600]

  # (advanced) Use compression when sending messages. Supported values are:
  # 'gzip', 'snappy', 's2' and '' (disable compression)
  # CLI flag: -ruler.client.grpc-compression
  [grpc_compression: <string> | default = ""]

  # (advanced) Rate limit for gRPC client; 0 means disabled.
  # CLI flag: -ruler.client.grpc-client-rate-limit
  [rate_limit: <float> | default = 0]

  # (advanced) Rate limit burst for gRPC client.
  # CLI flag: -ruler.client.grpc-client-rate-limit-burst
  [rate_limit_burst: <int> | default = 0]

  # (advanced) Enable backoff and retry when we hit rate limits.
  # CLI flag: -ruler.client.backoff-on-ratelimits
  [backoff_on_ratelimits: <boolean> | default = false]

  backoff_config:
    # (advanced) Minimum delay when backing off.
    # CLI flag: -ruler.client.backoff-min-period
    [min_period: <duration> | default = 100ms]

    # (advanced) Maximum delay when backing off.
    # CLI flag: -ruler.client.backoff-max-period
    [max_period: <duration> | default = 10s]

    # (advanced) Number of times to backoff and retry before failing.
    # CLI flag: -ruler.client.backoff-retries
    [max_retries: <int> | default = 10]

  # (experimental) Initial stream window size. Values less than the default are
  # not supported and are ignored. Setting this to a value other than the
  # default disables the BDP estimator.
  # CLI flag: -ruler.client.initial-stream-window-size
  [initial_stream_window_size: <int> | default = 63KiB1023B]

  # (experimental) Initial connection window size. Values less than the default
  # are not supported and are ignored. Setting this to a value other than the
  # default disables the BDP estimator.
  # CLI flag: -ruler.client.initial-connection-window-size
  [initial_connection_window_size: <int> | default = 63KiB1023B]

  # (advanced) Enable TLS in the gRPC client. This flag needs to be enabled when
  # any other TLS flag is set. If set to false, insecure connection to gRPC
  # server will be used.
  # CLI flag: -ruler.client.tls-enabled
  [tls_enabled: <boolean> | default = false]

  # (advanced) Path to the client certificate, which will be used for
  # authenticating with the server. Also requires the key path to be configured.
  # CLI flag: -ruler.client.tls-cert-path
  [tls_cert_path: <string> | default = ""]

  # (advanced) Path to the key for the client certificate. Also requires the
  # client certificate to be configured.
  # CLI flag: -ruler.client.tls-key-path
  [tls_key_path: <string> | default = ""]

  # (advanced) Path to the CA certificates to validate server certificate
  # against. If not set, the host's root CA certificates are used.
  # CLI flag: -ruler.client.tls-ca-path
  [tls_ca_path: <string> | default = ""]

  # (advanced) Override the expected name on the server certificate.
  # CLI flag: -ruler.client.tls-server-name
  [tls_server_name: <string> | default = ""]

  # (advanced) Skip validating server certificate.
  # CLI flag: -ruler.client.tls-insecure-skip-verify
  [tls_insecure_skip_verify: <boolean> | default = false]

  # (advanced) Override the default cipher suite list (separated by commas).
  # Allowed values:
  #
  # Secure Ciphers:
  # - TLS_AES_128_GCM_SHA256
  # - TLS_AES_256_GCM_SHA384
  # - TLS_CHACHA20_POLY1305_SHA256
  # - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
  # - TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
  # - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
  # - TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
  # - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
  # - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
  # - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
  # - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
  # - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
  # - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
  #
  # Insecure Ciphers:
  # - TLS_RSA_WITH_RC4_128_SHA
  # - TLS_RSA_WITH_3DES_EDE_CBC_SHA
  # - TLS_RSA_WITH_AES_128_CBC_SHA
  # - TLS_RSA_WITH_AES_256_CBC_SHA
  # - TLS_RSA_WITH_AES_128_CBC_SHA256
  # - TLS_RSA_WITH_AES_128_GCM_SHA256
  # - TLS_RSA_WITH_AES_256_GCM_SHA384
  # - TLS_ECDHE_ECDSA_WITH_RC4_128_SHA
  # - TLS_ECDHE_RSA_WITH_RC4_128_SHA
  # - TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
  # - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
  # - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
  # CLI flag: -ruler.client.tls-cipher-suites
  [tls_cipher_suites: <string> | default = ""]

  # (advanced) Override the default minimum TLS version. Allowed values:
  # VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13
  # CLI flag: -ruler.client.tls-min-version
  [tls_min_version: <string> | default = ""]

  # (advanced) The maximum amount of time to establish a connection. A value of
  # 0 means default gRPC client connect timeout and backoff.
  # CLI flag: -ruler.client.connect-timeout
  [connect_timeout: <duration> | default = 5s]

  # (advanced) Initial backoff delay after first connection failure. Only
  # relevant if ConnectTimeout > 0.
  # CLI flag: -ruler.client.connect-backoff-base-delay
  [connect_backoff_base_delay: <duration> | default = 1s]

  # (advanced) Maximum backoff delay when establishing a connection. Only
  # relevant if ConnectTimeout > 0.
  # CLI flag: -ruler.client.connect-backoff-max-delay
  [connect_backoff_max_delay: <duration> | default = 5s]

  cluster_validation:
    # (experimental) Primary cluster validation label.
    # CLI flag: -ruler.client.cluster-validation.label
    [label: <string> | default = ""]

  # (experimental) The grace period for health checks. If a ruler connection
  # consistently fails health checks for this period, any open connections are
  # closed. The ruler will attempt to reconnect to that ruler if a subsequent
  # request is made to that ruler. Set to 0 to immediately remove ruler
  # connections on the first health check failure.
  # CLI flag: -ruler.client.health-check-grace-period
  [health_check_grace_period: <duration> | default = 0s]

# (advanced) How frequently to evaluate rules
# CLI flag: -ruler.evaluation-interval
[evaluation_interval: <duration> | default = 1m]

# (advanced) How frequently the configured rule groups are re-synced from the
# object storage.
# CLI flag: -ruler.poll-interval
[poll_interval: <duration> | default = 10m]

# Directory to store temporary rule files loaded by the Prometheus rule
# managers. This directory is not required to be persisted between restarts.
# CLI flag: -ruler.rule-path
[rule_path: <string> | default = "./data-ruler/"]

# (deprecated) Deprecated, use
# limits.ruler_alertmanager_client_config.alertmanager_url instead.
[alertmanager_url: <string> | default = ""]

# (advanced) How long to wait between refreshing DNS resolutions of Alertmanager
# hosts.
# CLI flag: -ruler.alertmanager-refresh-interval
[alertmanager_refresh_interval: <duration> | default = 1m]

# (advanced) Capacity of the queue for notifications to be sent to the
# Alertmanager.
# CLI flag: -ruler.notification-queue-capacity
[notification_queue_capacity: <int> | default = 10000]

# (advanced) Maximum number of notifications to send to Alertmanager in one
# request.
# CLI flag: -ruler.max-notification-batch-size
[max_notification_batch_size: <int> | default = 256]

# (advanced) HTTP timeout duration when sending notifications to the
# Alertmanager.
# CLI flag: -ruler.notification-timeout
[notification_timeout: <duration> | default = 10s]

# Deprecated, use limits.ruler_alertmanager_client_config instead.
alertmanager_client:
  # (advanced)
  [tls_enabled: <boolean> | default = ]

  # (advanced)
  [tls_cert_path: <string> | default = ""]

  # (advanced)
  [tls_key_path: <string> | default = ""]

  # (advanced)
  [tls_ca_path: <string> | default = ""]

  # (advanced)
  [tls_server_name: <string> | default = ""]

  # (advanced)
  [tls_insecure_skip_verify: <boolean> | default = ]

  # (advanced) Override the default cipher suite list (separated by commas).
  # Allowed values:
  #
  # Secure Ciphers:
  # - TLS_AES_128_GCM_SHA256
  # - TLS_AES_256_GCM_SHA384
  # - TLS_CHACHA20_POLY1305_SHA256
  # - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
  # - TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
  # - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
  # - TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
  # - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
  # - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
  # - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
  # - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
  # - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
  # - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
  #
  # Insecure Ciphers:
  # - TLS_RSA_WITH_RC4_128_SHA
  # - TLS_RSA_WITH_3DES_EDE_CBC_SHA
  # - TLS_RSA_WITH_AES_128_CBC_SHA
  # - TLS_RSA_WITH_AES_256_CBC_SHA
  # - TLS_RSA_WITH_AES_128_CBC_SHA256
  # - TLS_RSA_WITH_AES_128_GCM_SHA256
  # - TLS_RSA_WITH_AES_256_GCM_SHA384
  # - TLS_ECDHE_ECDSA_WITH_RC4_128_SHA
  # - TLS_ECDHE_RSA_WITH_RC4_128_SHA
  # - TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
  # - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
  # - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
  [tls_cipher_suites: <string> | default = ""]

  # (advanced)
  [tls_min_version: <string> | default = ""]

  [basic_auth_username: <string> | default = ""]

  [basic_auth_password: <string> | default = ""]

  oauth2:
    [client_id: <string> | default = ""]

    [client_secret: <string> | default = ""]

    [token_url: <string> | default = ""]

    [scopes: <string> | default = ""]

    # (advanced)
    [endpoint_params: <map of string to string> | default = ]

  # (advanced)
  [proxy_url: <string> | default = ""]

# (advanced) Max time to tolerate outage for restoring "for" state of alert.
# CLI flag: -ruler.for-outage-tolerance
[for_outage_tolerance: <duration> | default = 1h]

# (advanced) This grace period controls which alerts the ruler restores after a
# restart. Alerts with "for" duration lower than this grace period are not
# restored after a ruler restart. This means that if the alerts have been firing
# before the ruler restarted, they will now go to pending state and then to
# firing again after their "for" duration expires. Alerts with "for" duration
# greater than or equal to this grace period that have been pending before the
# ruler restart will remain in pending state for at least this grace period.
# Alerts with "for" duration greater than or equal to this grace period that
# have been firing before the ruler restart will continue to be firing after the
# restart.
# CLI flag: -ruler.for-grace-period
[for_grace_period: <duration> | default = 2m]

# (advanced) Minimum amount of time to wait before resending an alert to
# Alertmanager.
# CLI flag: -ruler.resend-delay
[resend_delay: <duration> | default = 1m]

ring:
  # The key-value store used to share the hash ring across multiple instances.
  kvstore:
    # Backend storage to use for the ring. Supported values are: consul, etcd,
    # inmemory, memberlist, multi.
    # CLI flag: -ruler.ring.store
    [store: <string> | default = "memberlist"]

    # (advanced) The prefix for the keys in the store. Should end with a /.
    # CLI flag: -ruler.ring.prefix
    [prefix: <string> | default = "rulers/"]

    # The consul block configures the consul client.
    # The CLI flags prefix for this block configuration is: ruler.ring
    [consul: <consul>]

    # The etcd block configures the etcd client.
    # The CLI flags prefix for this block configuration is: ruler.ring
    [etcd: <etcd>]

    multi:
      # (advanced) Primary backend storage used by multi-client.
      # CLI flag: -ruler.ring.multi.primary
      [primary: <string> | default = ""]

      # (advanced) Secondary backend storage used by multi-client.
      # CLI flag: -ruler.ring.multi.secondary
      [secondary: <string> | default = ""]

      # (advanced) Mirror writes to the secondary store.
      # CLI flag: -ruler.ring.multi.mirror-enabled
      [mirror_enabled: <boolean> | default = false]

      # (advanced) Timeout for storing a value to the secondary store.
      # CLI flag: -ruler.ring.multi.mirror-timeout
      [mirror_timeout: <duration> | default = 2s]

  # (advanced) Period at which to heartbeat to the ring.
  # CLI flag: -ruler.ring.heartbeat-period
  [heartbeat_period: <duration> | default = 15s]

  # (advanced) Heartbeat timeout after which Mimir marks rulers as unhealthy in
  # the ring.
  # CLI flag: -ruler.ring.heartbeat-timeout
  [heartbeat_timeout: <duration> | default = 1m]

  # (advanced) Instance ID to register in the ring.
  # CLI flag: -ruler.ring.instance-id
  [instance_id: <string> | default = "<hostname>"]

  # List of network interface names to look up when finding the instance IP
  # address.
  # CLI flag: -ruler.ring.instance-interface-names
  [instance_interface_names: <list of strings> | default = [<private network interfaces>]]

  # (advanced) Port to advertise in the ring (defaults to
  # -server.grpc-listen-port).
  # CLI flag: -ruler.ring.instance-port
  [instance_port: <int> | default = 0]

  # (advanced) IP address to advertise in the ring. Default is auto-detected.
  # CLI flag: -ruler.ring.instance-addr
  [instance_addr: <string> | default = ""]

  # (advanced) Enable using an IPv6 instance address.
  # CLI flag: -ruler.ring.instance-enable-ipv6
  [instance_enable_ipv6: <boolean> | default = false]

  # (advanced) Number of consecutive timeout periods after which Mimir
  # automatically removes an unhealthy instance in the ring. Set to 0 to disable
  # auto-forget.
  # CLI flag: -ruler.ring.auto-forget-unhealthy-periods
  [auto_forget_unhealthy_periods: <int> | default = 2]

  # (advanced) Number of tokens for each ruler.
  # CLI flag: -ruler.ring.num-tokens
  [num_tokens: <int> | default = 128]

# Enable the ruler config API.
# CLI flag: -ruler.enable-api
[enable_api: <boolean> | default = true]

# (advanced) Comma separated list of tenants whose rules this ruler can
# evaluate. If specified, only these tenants will be handled by ruler, otherwise
# this ruler can process rules from all tenants. Subject to sharding.
# CLI flag: -ruler.enabled-tenants
[enabled_tenants: <string> | default = ""]

# (advanced) Comma separated list of tenants whose rules this ruler cannot
# evaluate. If specified, a ruler that would normally pick the specified
# tenant(s) for processing will ignore them instead. Subject to sharding.
# CLI flag: -ruler.disabled-tenants
[disabled_tenants: <string> | default = ""]

# (advanced) Report the wall time for ruler queries to complete as a per-tenant
# metric and as an info level log message.
# CLI flag: -ruler.query-stats-enabled
[query_stats_enabled: <boolean> | default = false]

query_frontend:
  # Can be either the GRPC listen address of the query-frontend(s) or the
  # HTTP/HTTPS address of a Prometheus-compatible server. Must be a DNS address
  # (prefixed with dns:///) to enable GRPC client side load balancing.
  # CLI flag: -ruler.query-frontend.address
  [address: <string> | default = ""]

  # Configures the gRPC client used to communicate between the rulers and
  # query-frontends.
  # The CLI flags prefix for this block configuration is:
  # ruler.query-frontend.grpc-client-config
  [grpc_client_config: <grpc_client>]

  # Configures the HTTP client used to communicate between the rulers and
  # query-frontends.
  http_client_config:
    # (advanced) Timeout for establishing a connection to the query-frontend.
    # CLI flag: -ruler.query-frontend.http-client-config.connect-timeout
    [connect_timeout: <duration> | default = 30s]

  # Format to use when retrieving query results from query-frontends. Supported
  # values: json, protobuf
  # CLI flag: -ruler.query-frontend.query-result-response-format
  [query_result_response_format: <string> | default = "protobuf"]

  # Maximum number of retries for failed queries per second.
  # CLI flag: -ruler.query-frontend.max-retries-rate
  [max_retries_rate: <float> | default = 170]

tenant_federation:
  # Enable rule groups to query against multiple tenants. The tenant IDs
  # involved need to be in the rule group's 'source_tenants' field. If this flag
  # is set to 'false' when there are federated rule groups that already exist,
  # then these rules groups will be skipped during evaluations.
  # CLI flag: -ruler.tenant-federation.enabled
  [enabled: <boolean> | default = false]

# (experimental) Interval between sending queued rule sync requests to ruler
# replicas.
# CLI flag: -ruler.outbound-sync-queue-poll-interval
[outbound_sync_queue_poll_interval: <duration> | default = 10s]

# (experimental) Interval between applying queued incoming rule sync requests.
# CLI flag: -ruler.inbound-sync-queue-poll-interval
[inbound_sync_queue_poll_interval: <duration> | default = 10s]

# (experimental) Number of rules rules that don't have dependencies that we
# allow to be evaluated concurrently across all tenants. 0 to disable.
# CLI flag: -ruler.max-independent-rule-evaluation-concurrency
[max_independent_rule_evaluation_concurrency: <int> | default = 0]

# (experimental) Minimum threshold of the interval to last rule group runtime
# duration to allow a rule to be evaluated concurrency. By default, the rule
# group runtime duration must exceed 50.0% of the evaluation interval.
# CLI flag: -ruler.independent-rule-evaluation-concurrency-min-duration-percentage
[independent_rule_evaluation_concurrency_min_duration_percentage: <float> | default = 50]

# (experimental) Writes the results of rule evaluation to ingesters or ingest
# storage when enabled. Use this option for testing purposes. To disable, set to
# false.
# CLI flag: -ruler.rule-evaluation-write-enabled
[rule_evaluation_write_enabled: <boolean> | default = true]

ruler_storage

The ruler_storage block configures the ruler storage backend.

# Backend storage to use. Supported backends are: s3, gcs, azure, swift,
# filesystem, local.
# CLI flag: -ruler-storage.backend
[backend: <string> | default = "filesystem"]

# The s3_backend block configures the connection to Amazon S3 object storage
# backend.
# The CLI flags prefix for this block configuration is: ruler-storage
[s3: <s3_storage_backend>]

# The gcs_backend block configures the connection to Google Cloud Storage object
# storage backend.
# The CLI flags prefix for this block configuration is: ruler-storage
[gcs: <gcs_storage_backend>]

# The azure_storage_backend block configures the connection to Azure object
# storage backend.
# The CLI flags prefix for this block configuration is: ruler-storage
[azure: <azure_storage_backend>]

# The swift_storage_backend block configures the connection to OpenStack Object
# Storage (Swift) object storage backend.
# The CLI flags prefix for this block configuration is: ruler-storage
[swift: <swift_storage_backend>]

# The filesystem_storage_backend block configures the usage of local file system
# as object storage backend.
# The CLI flags prefix for this block configuration is: ruler-storage
[filesystem: <filesystem_storage_backend>]

# Prefix for all objects stored in the backend storage. For simplicity, it may
# only contain digits and English alphabet letters.
# CLI flag: -ruler-storage.storage-prefix
[storage_prefix: <string> | default = ""]

local:
  # Directory to scan for rules
  # CLI flag: -ruler-storage.local.directory
  [directory: <string> | default = ""]

cache:
  # Backend for ruler storage cache, if not empty. The cache is supported for
  # any storage backend except "local". Supported values: memcached.
  # CLI flag: -ruler-storage.cache.backend
  [backend: <string> | default = ""]

  # The memcached block configures the Memcached-based caching backend.
  # The CLI flags prefix for this block configuration is: ruler-storage.cache
  [memcached: <memcached>]

alertmanager

The alertmanager block configures the alertmanager.

# Directory to store Alertmanager state and temporarily configuration files. The
# content of this directory is not required to be persisted between restarts
# unless Alertmanager replication has been disabled.
# CLI flag: -alertmanager.storage.path
[data_dir: <string> | default = "./data-alertmanager/"]

# (advanced) How long should we store stateful data (notification logs and
# silences). For notification log entries, refers to how long should we keep
# entries before they expire and are deleted. For silences, refers to how long
# should tenants view silences after they expire and are deleted.
# CLI flag: -alertmanager.storage.retention
[retention: <duration> | default = 120h]

# The URL under which Alertmanager is externally reachable (eg. could be
# different than -http.alertmanager-http-prefix in case Alertmanager is served
# via a reverse proxy). This setting is used both to configure the internal
# requests router and to generate links in alert templates. If the external URL
# has a path portion, it will be used to prefix all HTTP endpoints served by
# Alertmanager, both the UI and API.
# CLI flag: -alertmanager.web.external-url
[external_url: <url> | default = http://localhost:8080/alertmanager]

# (advanced) How frequently to poll Alertmanager configs.
# CLI flag: -alertmanager.configs.poll-interval
[poll_interval: <duration> | default = 15s]

# (advanced) Maximum size (bytes) of an accepted HTTP request body.
# CLI flag: -alertmanager.max-recv-msg-size
[max_recv_msg_size: <int> | default = 104857600]

# (experimental) Timeout for reading the state from object storage during the
# initial sync. Set to `0` for no timeout.
# CLI flag: -alertmanager.storage.state-read-timeout
[state_read_timeout: <duration> | default = 15s]

sharding_ring:
  # The key-value store used to share the hash ring across multiple instances.
  kvstore:
    # Backend storage to use for the ring. Supported values are: consul, etcd,
    # inmemory, memberlist, multi.
    # CLI flag: -alertmanager.sharding-ring.store
    [store: <string> | default = "memberlist"]

    # (advanced) The prefix for the keys in the store. Should end with a /.
    # CLI flag: -alertmanager.sharding-ring.prefix
    [prefix: <string> | default = "alertmanagers/"]

    # The consul block configures the consul client.
    # The CLI flags prefix for this block configuration is:
    # alertmanager.sharding-ring
    [consul: <consul>]

    # The etcd block configures the etcd client.
    # The CLI flags prefix for this block configuration is:
    # alertmanager.sharding-ring
    [etcd: <etcd>]

    multi:
      # (advanced) Primary backend storage used by multi-client.
      # CLI flag: -alertmanager.sharding-ring.multi.primary
      [primary: <string> | default = ""]

      # (advanced) Secondary backend storage used by multi-client.
      # CLI flag: -alertmanager.sharding-ring.multi.secondary
      [secondary: <string> | default = ""]

      # (advanced) Mirror writes to the secondary store.
      # CLI flag: -alertmanager.sharding-ring.multi.mirror-enabled
      [mirror_enabled: <boolean> | default = false]

      # (advanced) Timeout for storing a value to the secondary store.
      # CLI flag: -alertmanager.sharding-ring.multi.mirror-timeout
      [mirror_timeout: <duration> | default = 2s]

  # (advanced) Period at which to heartbeat to the ring.
  # CLI flag: -alertmanager.sharding-ring.heartbeat-period
  [heartbeat_period: <duration> | default = 15s]

  # (advanced) Heartbeat timeout after which Mimir marks alertmanagers as
  # unhealthy in the ring.
  # CLI flag: -alertmanager.sharding-ring.heartbeat-timeout
  [heartbeat_timeout: <duration> | default = 1m]

  # (advanced) Instance ID to register in the ring.
  # CLI flag: -alertmanager.sharding-ring.instance-id
  [instance_id: <string> | default = "<hostname>"]

  # List of network interface names to look up when finding the instance IP
  # address.
  # CLI flag: -alertmanager.sharding-ring.instance-interface-names
  [instance_interface_names: <list of strings> | default = [<private network interfaces>]]

  # (advanced) Port to advertise in the ring (defaults to
  # -server.grpc-listen-port).
  # CLI flag: -alertmanager.sharding-ring.instance-port
  [instance_port: <int> | default = 0]

  # (advanced) IP address to advertise in the ring. Default is auto-detected.
  # CLI flag: -alertmanager.sharding-ring.instance-addr
  [instance_addr: <string> | default = ""]

  # (advanced) Enable using an IPv6 instance address.
  # CLI flag: -alertmanager.sharding-ring.instance-enable-ipv6
  [instance_enable_ipv6: <boolean> | default = false]

  # (advanced) The replication factor to use when sharding the alertmanager.
  # CLI flag: -alertmanager.sharding-ring.replication-factor
  [replication_factor: <int> | default = 3]

  # (advanced) True to enable zone-awareness and replicate alerts across
  # different availability zones.
  # CLI flag: -alertmanager.sharding-ring.zone-awareness-enabled
  [zone_awareness_enabled: <boolean> | default = false]

  # (advanced) The availability zone where this instance is running. Required if
  # zone-awareness is enabled.
  # CLI flag: -alertmanager.sharding-ring.instance-availability-zone
  [instance_availability_zone: <string> | default = ""]

# Filename of fallback config to use if none specified for instance.
# CLI flag: -alertmanager.configs.fallback
[fallback_config_file: <string> | default = ""]

# (advanced) Time to wait between peers to send notifications.
# CLI flag: -alertmanager.peer-timeout
[peer_timeout: <duration> | default = 15s]

# (advanced) Enable the alertmanager config API.
# CLI flag: -alertmanager.enable-api
[enable_api: <boolean> | default = true]

# (experimental) Duration to wait before shutting down an idle Alertmanager
# using a default/empty configuration when strict initialization is enabled.
# CLI flag: -alertmanager.strict-initialization-idle-grace-period
[strict_initialization_idle_grace_period: <duration> | default = 5m]

# (advanced) Maximum number of concurrent GET requests allowed per tenant. The
# zero value (and negative values) result in a limit of GOMAXPROCS or 8,
# whichever is larger. Status code 503 is served for GET requests that would
# exceed the concurrency limit.
# CLI flag: -alertmanager.max-concurrent-get-requests-per-tenant
[max_concurrent_get_requests_per_tenant: <int> | default = 0]

alertmanager_client:
  # (advanced) Timeout for downstream alertmanagers.
  # CLI flag: -alertmanager.alertmanager-client.remote-timeout
  [remote_timeout: <duration> | default = 2s]

  # (advanced) gRPC client max receive message size (bytes).
  # CLI flag: -alertmanager.alertmanager-client.grpc-max-recv-msg-size
  [max_recv_msg_size: <int> | default = 104857600]

  # (advanced) gRPC client max send message size (bytes).
  # CLI flag: -alertmanager.alertmanager-client.grpc-max-send-msg-size
  [max_send_msg_size: <int> | default = 104857600]

  # (advanced) Use compression when sending messages. Supported values are:
  # 'gzip', 'snappy', 's2' and '' (disable compression)
  # CLI flag: -alertmanager.alertmanager-client.grpc-compression
  [grpc_compression: <string> | default = ""]

  # (advanced) Rate limit for gRPC client; 0 means disabled.
  # CLI flag: -alertmanager.alertmanager-client.grpc-client-rate-limit
  [rate_limit: <float> | default = 0]

  # (advanced) Rate limit burst for gRPC client.
  # CLI flag: -alertmanager.alertmanager-client.grpc-client-rate-limit-burst
  [rate_limit_burst: <int> | default = 0]

  # (advanced) Enable backoff and retry when we hit rate limits.
  # CLI flag: -alertmanager.alertmanager-client.backoff-on-ratelimits
  [backoff_on_ratelimits: <boolean> | default = false]

  backoff_config:
    # (advanced) Minimum delay when backing off.
    # CLI flag: -alertmanager.alertmanager-client.backoff-min-period
    [min_period: <duration> | default = 100ms]

    # (advanced) Maximum delay when backing off.
    # CLI flag: -alertmanager.alertmanager-client.backoff-max-period
    [max_period: <duration> | default = 10s]

    # (advanced) Number of times to backoff and retry before failing.
    # CLI flag: -alertmanager.alertmanager-client.backoff-retries
    [max_retries: <int> | default = 10]

  # (experimental) Initial stream window size. Values less than the default are
  # not supported and are ignored. Setting this to a value other than the
  # default disables the BDP estimator.
  # CLI flag: -alertmanager.alertmanager-client.initial-stream-window-size
  [initial_stream_window_size: <int> | default = 63KiB1023B]

  # (experimental) Initial connection window size. Values less than the default
  # are not supported and are ignored. Setting this to a value other than the
  # default disables the BDP estimator.
  # CLI flag: -alertmanager.alertmanager-client.initial-connection-window-size
  [initial_connection_window_size: <int> | default = 63KiB1023B]

  # (advanced) Enable TLS in the gRPC client. This flag needs to be enabled when
  # any other TLS flag is set. If set to false, insecure connection to gRPC
  # server will be used.
  # CLI flag: -alertmanager.alertmanager-client.tls-enabled
  [tls_enabled: <boolean> | default = false]

  # (advanced) Path to the client certificate, which will be used for
  # authenticating with the server. Also requires the key path to be configured.
  # CLI flag: -alertmanager.alertmanager-client.tls-cert-path
  [tls_cert_path: <string> | default = ""]

  # (advanced) Path to the key for the client certificate. Also requires the
  # client certificate to be configured.
  # CLI flag: -alertmanager.alertmanager-client.tls-key-path
  [tls_key_path: <string> | default = ""]

  # (advanced) Path to the CA certificates to validate server certificate
  # against. If not set, the host's root CA certificates are used.
  # CLI flag: -alertmanager.alertmanager-client.tls-ca-path
  [tls_ca_path: <string> | default = ""]

  # (advanced) Override the expected name on the server certificate.
  # CLI flag: -alertmanager.alertmanager-client.tls-server-name
  [tls_server_name: <string> | default = ""]

  # (advanced) Skip validating server certificate.
  # CLI flag: -alertmanager.alertmanager-client.tls-insecure-skip-verify
  [tls_insecure_skip_verify: <boolean> | default = false]

  # (advanced) Override the default cipher suite list (separated by commas).
  # Allowed values:
  #
  # Secure Ciphers:
  # - TLS_AES_128_GCM_SHA256
  # - TLS_AES_256_GCM_SHA384
  # - TLS_CHACHA20_POLY1305_SHA256
  # - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
  # - TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
  # - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
  # - TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
  # - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
  # - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
  # - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
  # - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
  # - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
  # - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
  #
  # Insecure Ciphers:
  # - TLS_RSA_WITH_RC4_128_SHA
  # - TLS_RSA_WITH_3DES_EDE_CBC_SHA
  # - TLS_RSA_WITH_AES_128_CBC_SHA
  # - TLS_RSA_WITH_AES_256_CBC_SHA
  # - TLS_RSA_WITH_AES_128_CBC_SHA256
  # - TLS_RSA_WITH_AES_128_GCM_SHA256
  # - TLS_RSA_WITH_AES_256_GCM_SHA384
  # - TLS_ECDHE_ECDSA_WITH_RC4_128_SHA
  # - TLS_ECDHE_RSA_WITH_RC4_128_SHA
  # - TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
  # - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
  # - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
  # CLI flag: -alertmanager.alertmanager-client.tls-cipher-suites
  [tls_cipher_suites: <string> | default = ""]

  # (advanced) Override the default minimum TLS version. Allowed values:
  # VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13
  # CLI flag: -alertmanager.alertmanager-client.tls-min-version
  [tls_min_version: <string> | default = ""]

  # (advanced) The maximum amount of time to establish a connection. A value of
  # 0 means default gRPC client connect timeout and backoff.
  # CLI flag: -alertmanager.alertmanager-client.connect-timeout
  [connect_timeout: <duration> | default = 5s]

  # (advanced) Initial backoff delay after first connection failure. Only
  # relevant if ConnectTimeout > 0.
  # CLI flag: -alertmanager.alertmanager-client.connect-backoff-base-delay
  [connect_backoff_base_delay: <duration> | default = 1s]

  # (advanced) Maximum backoff delay when establishing a connection. Only
  # relevant if ConnectTimeout > 0.
  # CLI flag: -alertmanager.alertmanager-client.connect-backoff-max-delay
  [connect_backoff_max_delay: <duration> | default = 5s]

  cluster_validation:
    # (experimental) Primary cluster validation label.
    # CLI flag: -alertmanager.alertmanager-client.cluster-validation.label
    [label: <string> | default = ""]

  # (experimental) The grace period for health checks. If an alertmanager
  # connection consistently fails health checks for this period, any open
  # connections are closed. The alertmanager will attempt to reconnect to that
  # alertmanager if a subsequent request is made to that alertmanager. Set to 0
  # to immediately remove alertmanager connections on the first health check
  # failure.
  # CLI flag: -alertmanager.alertmanager-client.health-check-grace-period
  [health_check_grace_period: <duration> | default = 0s]

# (advanced) The interval between persisting the current alertmanager state
# (notification log and silences) to object storage. This is only used when
# sharding is enabled. This state is read when all replicas for a shard can not
# be contacted. In this scenario, having persisted the state more frequently
# will result in potentially fewer lost silences, and fewer duplicate
# notifications.
# CLI flag: -alertmanager.persist-interval
[persist_interval: <duration> | default = 15m]

# (advanced) Enables periodic cleanup of alertmanager stateful data
# (notification logs and silences) from object storage. When enabled, data is
# removed for any tenant that does not have a configuration.
# CLI flag: -alertmanager.enable-state-cleanup
[enable_state_cleanup: <boolean> | default = true]

# (experimental) Skip initializing Alertmanagers for tenants without a
# non-default, non-empty configuration. For Grafana Alertmanager tenants,
# configurations not marked as 'promoted' will also be skipped.
# CLI flag: -alertmanager.strict-initialization-enabled
[strict_initialization: <boolean> | default = false]

# (advanced) Enable UTF-8 strict mode. Allows UTF-8 characters in the matchers
# for routes and inhibition rules, in silences, and in the labels for alerts. It
# is recommended that all tenants run the `migrate-utf8` command in mimirtool
# before enabling this mode. Otherwise, some tenant configurations might fail to
# load. For more information, refer to [Enable
# UTF-8](https://grafana.com/docs/mimir/<MIMIR_VERSION>/references/architecture/components/alertmanager/#enable-utf-8).
# CLI flag: -alertmanager.utf8-strict-mode-enabled
[utf8_strict_mode: <boolean> | default = false]

# (experimental) Enable logging when parsing label matchers. This flag is
# intended to be used with -alertmanager.utf8-strict-mode-enabled to validate
# UTF-8 strict mode is working as intended.
# CLI flag: -alertmanager.log-parsing-label-matchers
[log_parsing_label_matchers: <boolean> | default = false]

# (experimental) Enable logging of tenant configurations that are incompatible
# with UTF-8 strict mode.
# CLI flag: -alertmanager.utf8-migration-logging-enabled
[utf8_migration_logging: <boolean> | default = false]

alertmanager_storage

The alertmanager_storage block configures the alertmanager storage backend.

# Backend storage to use. Supported backends are: s3, gcs, azure, swift,
# filesystem, local.
# CLI flag: -alertmanager-storage.backend
[backend: <string> | default = "filesystem"]

# The s3_backend block configures the connection to Amazon S3 object storage
# backend.
# The CLI flags prefix for this block configuration is: alertmanager-storage
[s3: <s3_storage_backend>]

# The gcs_backend block configures the connection to Google Cloud Storage object
# storage backend.
# The CLI flags prefix for this block configuration is: alertmanager-storage
[gcs: <gcs_storage_backend>]

# The azure_storage_backend block configures the connection to Azure object
# storage backend.
# The CLI flags prefix for this block configuration is: alertmanager-storage
[azure: <azure_storage_backend>]

# The swift_storage_backend block configures the connection to OpenStack Object
# Storage (Swift) object storage backend.
# The CLI flags prefix for this block configuration is: alertmanager-storage
[swift: <swift_storage_backend>]

# The filesystem_storage_backend block configures the usage of local file system
# as object storage backend.
# The CLI flags prefix for this block configuration is: alertmanager-storage
[filesystem: <filesystem_storage_backend>]

# Prefix for all objects stored in the backend storage. For simplicity, it may
# only contain digits and English alphabet letters.
# CLI flag: -alertmanager-storage.storage-prefix
[storage_prefix: <string> | default = ""]

local:
  # Path at which alertmanager configurations are stored.
  # CLI flag: -alertmanager-storage.local.path
  [path: <string> | default = ""]

ingester_client

The ingester_client block configures how the distributors connect to the ingesters.

# Configures the gRPC client used to communicate with ingesters from
# distributors, queriers and rulers.
# The CLI flags prefix for this block configuration is: ingester.client
[grpc_client_config: <grpc_client>]

grpc_client

The grpc_client block configures the gRPC client used to communicate between two Mimir components. The supported CLI flags <prefix> used to reference this configuration block are:

ingester.client
querier.scheduler-client
query-frontend.grpc-client-config
query-scheduler.grpc-client-config
ruler.query-frontend.grpc-client-config

# (advanced) gRPC client max receive message size (bytes).
# CLI flag: -<prefix>.grpc-max-recv-msg-size
[max_recv_msg_size: <int> | default = 104857600]

# (advanced) gRPC client max send message size (bytes).
# CLI flag: -<prefix>.grpc-max-send-msg-size
[max_send_msg_size: <int> | default = 104857600]

# (advanced) Use compression when sending messages. Supported values are:
# 'gzip', 'snappy', 's2' and '' (disable compression)
# CLI flag: -<prefix>.grpc-compression
[grpc_compression: <string> | default = ""]

# (advanced) Rate limit for gRPC client; 0 means disabled.
# CLI flag: -<prefix>.grpc-client-rate-limit
[rate_limit: <float> | default = 0]

# (advanced) Rate limit burst for gRPC client.
# CLI flag: -<prefix>.grpc-client-rate-limit-burst
[rate_limit_burst: <int> | default = 0]

# (advanced) Enable backoff and retry when we hit rate limits.
# CLI flag: -<prefix>.backoff-on-ratelimits
[backoff_on_ratelimits: <boolean> | default = false]

backoff_config:
  # (advanced) Minimum delay when backing off.
  # CLI flag: -<prefix>.backoff-min-period
  [min_period: <duration> | default = 100ms]

  # (advanced) Maximum delay when backing off.
  # CLI flag: -<prefix>.backoff-max-period
  [max_period: <duration> | default = 10s]

  # (advanced) Number of times to backoff and retry before failing.
  # CLI flag: -<prefix>.backoff-retries
  [max_retries: <int> | default = 10]

# (experimental) Initial stream window size. Values less than the default are
# not supported and are ignored. Setting this to a value other than the default
# disables the BDP estimator.
# CLI flag: -<prefix>.initial-stream-window-size
[initial_stream_window_size: <int> | default = 63KiB1023B]

# (experimental) Initial connection window size. Values less than the default
# are not supported and are ignored. Setting this to a value other than the
# default disables the BDP estimator.
# CLI flag: -<prefix>.initial-connection-window-size
[initial_connection_window_size: <int> | default = 63KiB1023B]

# (advanced) Enable TLS in the gRPC client. This flag needs to be enabled when
# any other TLS flag is set. If set to false, insecure connection to gRPC server
# will be used.
# CLI flag: -<prefix>.tls-enabled
[tls_enabled: <boolean> | default = false]

# (advanced) Path to the client certificate, which will be used for
# authenticating with the server. Also requires the key path to be configured.
# CLI flag: -<prefix>.tls-cert-path
[tls_cert_path: <string> | default = ""]

# (advanced) Path to the key for the client certificate. Also requires the
# client certificate to be configured.
# CLI flag: -<prefix>.tls-key-path
[tls_key_path: <string> | default = ""]

# (advanced) Path to the CA certificates to validate server certificate against.
# If not set, the host's root CA certificates are used.
# CLI flag: -<prefix>.tls-ca-path
[tls_ca_path: <string> | default = ""]

# (advanced) Override the expected name on the server certificate.
# CLI flag: -<prefix>.tls-server-name
[tls_server_name: <string> | default = ""]

# (advanced) Skip validating server certificate.
# CLI flag: -<prefix>.tls-insecure-skip-verify
[tls_insecure_skip_verify: <boolean> | default = false]

# (advanced) Override the default cipher suite list (separated by commas).
# Allowed values:
#
# Secure Ciphers:
# - TLS_AES_128_GCM_SHA256
# - TLS_AES_256_GCM_SHA384
# - TLS_CHACHA20_POLY1305_SHA256
# - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
# - TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
# - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
# - TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
# - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
# - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
# - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
# - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
# - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
# - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
#
# Insecure Ciphers:
# - TLS_RSA_WITH_RC4_128_SHA
# - TLS_RSA_WITH_3DES_EDE_CBC_SHA
# - TLS_RSA_WITH_AES_128_CBC_SHA
# - TLS_RSA_WITH_AES_256_CBC_SHA
# - TLS_RSA_WITH_AES_128_CBC_SHA256
# - TLS_RSA_WITH_AES_128_GCM_SHA256
# - TLS_RSA_WITH_AES_256_GCM_SHA384
# - TLS_ECDHE_ECDSA_WITH_RC4_128_SHA
# - TLS_ECDHE_RSA_WITH_RC4_128_SHA
# - TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
# - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
# - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
# CLI flag: -<prefix>.tls-cipher-suites
[tls_cipher_suites: <string> | default = ""]

# (advanced) Override the default minimum TLS version. Allowed values:
# VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13
# CLI flag: -<prefix>.tls-min-version
[tls_min_version: <string> | default = ""]

# (advanced) The maximum amount of time to establish a connection. A value of 0
# means default gRPC client connect timeout and backoff.
# CLI flag: -<prefix>.connect-timeout
[connect_timeout: <duration> | default = 5s]

# (advanced) Initial backoff delay after first connection failure. Only relevant
# if ConnectTimeout > 0.
# CLI flag: -<prefix>.connect-backoff-base-delay
[connect_backoff_base_delay: <duration> | default = 1s]

# (advanced) Maximum backoff delay when establishing a connection. Only relevant
# if ConnectTimeout > 0.
# CLI flag: -<prefix>.connect-backoff-max-delay
[connect_backoff_max_delay: <duration> | default = 5s]

cluster_validation:
  # (experimental) Primary cluster validation label.
  # CLI flag: -<prefix>.cluster-validation.label
  [label: <string> | default = ""]

frontend_worker

The frontend_worker block configures the worker running within the querier, picking up and executing queries enqueued by the query-frontend or the query-scheduler.

# Address of the query-scheduler component, in host:port format. The host should
# resolve to all query-scheduler instances. This option should be set only when
# query-scheduler component is in use and
# -query-scheduler.service-discovery-mode is set to 'dns'.
# CLI flag: -querier.scheduler-address
[scheduler_address: <string> | default = ""]

# (advanced) How often to query DNS for query-frontend or query-scheduler
# address.
# CLI flag: -querier.dns-lookup-period
[dns_lookup_duration: <duration> | default = 10s]

# (advanced) Querier ID, sent to the query-frontend to identify requests from
# the same querier. Defaults to hostname.
# CLI flag: -querier.id
[id: <string> | default = ""]

# Configures the gRPC client used to communicate between the querier and the
# query-frontend.
grpc_client_config:
  # (advanced) gRPC client max receive message size (bytes).
  # CLI flag: -querier.frontend-client.grpc-max-recv-msg-size
  [max_recv_msg_size: <int> | default = 104857600]

  # (advanced) gRPC client max send message size (bytes).
  # CLI flag: -querier.frontend-client.grpc-max-send-msg-size
  [max_send_msg_size: <int> | default = 104857600]

  # (advanced) Use compression when sending messages. Supported values are:
  # 'gzip', 'snappy', 's2' and '' (disable compression)
  # CLI flag: -querier.frontend-client.grpc-compression
  [grpc_compression: <string> | default = ""]

  # (advanced) Rate limit for gRPC client; 0 means disabled.
  # CLI flag: -querier.frontend-client.grpc-client-rate-limit
  [rate_limit: <float> | default = 0]

  # (advanced) Rate limit burst for gRPC client.
  # CLI flag: -querier.frontend-client.grpc-client-rate-limit-burst
  [rate_limit_burst: <int> | default = 0]

  # (advanced) Enable backoff and retry when we hit rate limits.
  # CLI flag: -querier.frontend-client.backoff-on-ratelimits
  [backoff_on_ratelimits: <boolean> | default = false]

  backoff_config:
    # (advanced) Minimum delay when backing off.
    # CLI flag: -querier.frontend-client.backoff-min-period
    [min_period: <duration> | default = 100ms]

    # (advanced) Maximum delay when backing off.
    # CLI flag: -querier.frontend-client.backoff-max-period
    [max_period: <duration> | default = 10s]

    # (advanced) Number of times to backoff and retry before failing.
    # CLI flag: -querier.frontend-client.backoff-retries
    [max_retries: <int> | default = 10]

  # (experimental) Initial stream window size. Values less than the default are
  # not supported and are ignored. Setting this to a value other than the
  # default disables the BDP estimator.
  # CLI flag: -querier.frontend-client.initial-stream-window-size
  [initial_stream_window_size: <int> | default = 63KiB1023B]

  # (experimental) Initial connection window size. Values less than the default
  # are not supported and are ignored. Setting this to a value other than the
  # default disables the BDP estimator.
  # CLI flag: -querier.frontend-client.initial-connection-window-size
  [initial_connection_window_size: <int> | default = 63KiB1023B]

  # (advanced) Enable TLS in the gRPC client. This flag needs to be enabled when
  # any other TLS flag is set. If set to false, insecure connection to gRPC
  # server will be used.
  # CLI flag: -querier.frontend-client.tls-enabled
  [tls_enabled: <boolean> | default = false]

  # (advanced) Path to the client certificate, which will be used for
  # authenticating with the server. Also requires the key path to be configured.
  # CLI flag: -querier.frontend-client.tls-cert-path
  [tls_cert_path: <string> | default = ""]

  # (advanced) Path to the key for the client certificate. Also requires the
  # client certificate to be configured.
  # CLI flag: -querier.frontend-client.tls-key-path
  [tls_key_path: <string> | default = ""]

  # (advanced) Path to the CA certificates to validate server certificate
  # against. If not set, the host's root CA certificates are used.
  # CLI flag: -querier.frontend-client.tls-ca-path
  [tls_ca_path: <string> | default = ""]

  # (advanced) Override the expected name on the server certificate.
  # CLI flag: -querier.frontend-client.tls-server-name
  [tls_server_name: <string> | default = ""]

  # (advanced) Skip validating server certificate.
  # CLI flag: -querier.frontend-client.tls-insecure-skip-verify
  [tls_insecure_skip_verify: <boolean> | default = false]

  # (advanced) Override the default cipher suite list (separated by commas).
  # Allowed values:
  #
  # Secure Ciphers:
  # - TLS_AES_128_GCM_SHA256
  # - TLS_AES_256_GCM_SHA384
  # - TLS_CHACHA20_POLY1305_SHA256
  # - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
  # - TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
  # - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
  # - TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
  # - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
  # - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
  # - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
  # - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
  # - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
  # - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
  #
  # Insecure Ciphers:
  # - TLS_RSA_WITH_RC4_128_SHA
  # - TLS_RSA_WITH_3DES_EDE_CBC_SHA
  # - TLS_RSA_WITH_AES_128_CBC_SHA
  # - TLS_RSA_WITH_AES_256_CBC_SHA
  # - TLS_RSA_WITH_AES_128_CBC_SHA256
  # - TLS_RSA_WITH_AES_128_GCM_SHA256
  # - TLS_RSA_WITH_AES_256_GCM_SHA384
  # - TLS_ECDHE_ECDSA_WITH_RC4_128_SHA
  # - TLS_ECDHE_RSA_WITH_RC4_128_SHA
  # - TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
  # - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
  # - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
  # CLI flag: -querier.frontend-client.tls-cipher-suites
  [tls_cipher_suites: <string> | default = ""]

  # (advanced) Override the default minimum TLS version. Allowed values:
  # VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13
  # CLI flag: -querier.frontend-client.tls-min-version
  [tls_min_version: <string> | default = ""]

  # (advanced) The maximum amount of time to establish a connection. A value of
  # 0 means default gRPC client connect timeout and backoff.
  # CLI flag: -querier.frontend-client.connect-timeout
  [connect_timeout: <duration> | default = 5s]

  # (advanced) Initial backoff delay after first connection failure. Only
  # relevant if ConnectTimeout > 0.
  # CLI flag: -querier.frontend-client.connect-backoff-base-delay
  [connect_backoff_base_delay: <duration> | default = 1s]

  # (advanced) Maximum backoff delay when establishing a connection. Only
  # relevant if ConnectTimeout > 0.
  # CLI flag: -querier.frontend-client.connect-backoff-max-delay
  [connect_backoff_max_delay: <duration> | default = 5s]

  cluster_validation:
    # (experimental) Primary cluster validation label.
    # CLI flag: -querier.frontend-client.cluster-validation.label
    [label: <string> | default = ""]

  # (experimental) The grace period for health checks. If a query-frontend
  # connection consistently fails health checks for this period, any open
  # connections are closed. The querier will attempt to reconnect to the
  # query-frontend if a subsequent request is received from it. Set to 0 to
  # immediately remove query-frontend connections on the first health check
  # failure.
  # CLI flag: -querier.frontend-client.health-check-grace-period
  [health_check_grace_period: <duration> | default = 0s]

# Configures the gRPC client used to communicate between the querier and the
# query-scheduler.
# The CLI flags prefix for this block configuration is: querier.scheduler-client
[query_scheduler_grpc_client_config: <grpc_client>]

etcd

The etcd block configures the etcd client. The supported CLI flags <prefix> used to reference this configuration block are:

alertmanager.sharding-ring
compactor.ring
distributor.ha-tracker
distributor.ring
ingester.partition-ring
ingester.ring
overrides-exporter.ring
querier.ring
query-scheduler.ring
ruler.ring
store-gateway.sharding-ring

# The etcd endpoints to connect to.
# CLI flag: -<prefix>.etcd.endpoints
[endpoints: <list of strings> | default = []]

# (advanced) The dial timeout for the etcd connection.
# CLI flag: -<prefix>.etcd.dial-timeout
[dial_timeout: <duration> | default = 10s]

# (advanced) The maximum number of retries to do for failed ops.
# CLI flag: -<prefix>.etcd.max-retries
[max_retries: <int> | default = 10]

# (advanced) Enable TLS.
# CLI flag: -<prefix>.etcd.tls-enabled
[tls_enabled: <boolean> | default = false]

# (advanced) Path to the client certificate, which will be used for
# authenticating with the server. Also requires the key path to be configured.
# CLI flag: -<prefix>.etcd.tls-cert-path
[tls_cert_path: <string> | default = ""]

# (advanced) Path to the key for the client certificate. Also requires the
# client certificate to be configured.
# CLI flag: -<prefix>.etcd.tls-key-path
[tls_key_path: <string> | default = ""]

# (advanced) Path to the CA certificates to validate server certificate against.
# If not set, the host's root CA certificates are used.
# CLI flag: -<prefix>.etcd.tls-ca-path
[tls_ca_path: <string> | default = ""]

# (advanced) Override the expected name on the server certificate.
# CLI flag: -<prefix>.etcd.tls-server-name
[tls_server_name: <string> | default = ""]

# (advanced) Skip validating server certificate.
# CLI flag: -<prefix>.etcd.tls-insecure-skip-verify
[tls_insecure_skip_verify: <boolean> | default = false]

# (advanced) Override the default cipher suite list (separated by commas).
# Allowed values:
#
# Secure Ciphers:
# - TLS_AES_128_GCM_SHA256
# - TLS_AES_256_GCM_SHA384
# - TLS_CHACHA20_POLY1305_SHA256
# - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
# - TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
# - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
# - TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
# - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
# - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
# - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
# - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
# - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
# - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
#
# Insecure Ciphers:
# - TLS_RSA_WITH_RC4_128_SHA
# - TLS_RSA_WITH_3DES_EDE_CBC_SHA
# - TLS_RSA_WITH_AES_128_CBC_SHA
# - TLS_RSA_WITH_AES_256_CBC_SHA
# - TLS_RSA_WITH_AES_128_CBC_SHA256
# - TLS_RSA_WITH_AES_128_GCM_SHA256
# - TLS_RSA_WITH_AES_256_GCM_SHA384
# - TLS_ECDHE_ECDSA_WITH_RC4_128_SHA
# - TLS_ECDHE_RSA_WITH_RC4_128_SHA
# - TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
# - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
# - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
# CLI flag: -<prefix>.etcd.tls-cipher-suites
[tls_cipher_suites: <string> | default = ""]

# (advanced) Override the default minimum TLS version. Allowed values:
# VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13
# CLI flag: -<prefix>.etcd.tls-min-version
[tls_min_version: <string> | default = ""]

# Etcd username.
# CLI flag: -<prefix>.etcd.username
[username: <string> | default = ""]

# Etcd password.
# CLI flag: -<prefix>.etcd.password
[password: <string> | default = ""]

consul

The consul block configures the consul client. The supported CLI flags <prefix> used to reference this configuration block are:

alertmanager.sharding-ring
compactor.ring
distributor.ha-tracker
distributor.ring
ingester.partition-ring
ingester.ring
overrides-exporter.ring
querier.ring
query-scheduler.ring
ruler.ring
store-gateway.sharding-ring

# Hostname and port of Consul.
# CLI flag: -<prefix>.consul.hostname
[host: <string> | default = "localhost:8500"]

# (advanced) ACL Token used to interact with Consul.
# CLI flag: -<prefix>.consul.acl-token
[acl_token: <string> | default = ""]

# (advanced) HTTP timeout when talking to Consul
# CLI flag: -<prefix>.consul.client-timeout
[http_client_timeout: <duration> | default = 20s]

# (advanced) Enable consistent reads to Consul.
# CLI flag: -<prefix>.consul.consistent-reads
[consistent_reads: <boolean> | default = false]

# (advanced) Rate limit when watching key or prefix in Consul, in requests per
# second. 0 disables the rate limit.
# CLI flag: -<prefix>.consul.watch-rate-limit
[watch_rate_limit: <float> | default = 1]

# (advanced) Burst size used in rate limit. Values less than 1 are treated as 1.
# CLI flag: -<prefix>.consul.watch-burst-size
[watch_burst_size: <int> | default = 1]

# (advanced) Maximum duration to wait before retrying a Compare And Swap (CAS)
# operation.
# CLI flag: -<prefix>.consul.cas-retry-delay
[cas_retry_delay: <duration> | default = 1s]

memberlist

The memberlist block configures the Gossip memberlist.

# (advanced) Name of the node in memberlist cluster. Defaults to hostname.
# CLI flag: -memberlist.nodename
[node_name: <string> | default = ""]

# (advanced) Add random suffix to the node name.
# CLI flag: -memberlist.randomize-node-name
[randomize_node_name: <boolean> | default = true]

# (advanced) The timeout for establishing a connection with a remote node, and
# for read/write operations.
# CLI flag: -memberlist.stream-timeout
[stream_timeout: <duration> | default = 2s]

# (advanced) Multiplication factor used when sending out messages (factor *
# log(N+1)).
# CLI flag: -memberlist.retransmit-factor
[retransmit_factor: <int> | default = 4]

# (advanced) How often to use pull/push sync.
# CLI flag: -memberlist.pullpush-interval
[pull_push_interval: <duration> | default = 30s]

# (advanced) How often to gossip.
# CLI flag: -memberlist.gossip-interval
[gossip_interval: <duration> | default = 200ms]

# (advanced) How many nodes to gossip to.
# CLI flag: -memberlist.gossip-nodes
[gossip_nodes: <int> | default = 3]

# (advanced) How long to keep gossiping to dead nodes, to give them chance to
# refute their death.
# CLI flag: -memberlist.gossip-to-dead-nodes-time
[gossip_to_dead_nodes_time: <duration> | default = 30s]

# (advanced) How soon can dead node's name be reclaimed with new address. 0 to
# disable.
# CLI flag: -memberlist.dead-node-reclaim-time
[dead_node_reclaim_time: <duration> | default = 0s]

# (advanced) Enable message compression. This can be used to reduce bandwidth
# usage at the cost of slightly more CPU utilization.
# CLI flag: -memberlist.compression-enabled
[compression_enabled: <boolean> | default = true]

# (advanced) How frequently to notify watchers when a key changes. Can reduce
# CPU activity in large memberlist deployments. 0 to notify without delay.
# CLI flag: -memberlist.notify-interval
[notify_interval: <duration> | default = 0s]

# (advanced) Size of the internal queue for messages received from other nodes.
# Increasing this value may help to avoid dropping messages when the node is
# processing a large number of messages from other nodes.
# CLI flag: -memberlist.received-messages-queue-size
[received_messages_queue_size: <int> | default = 1024]

# Gossip address to advertise to other members in the cluster. Used for NAT
# traversal.
# CLI flag: -memberlist.advertise-addr
[advertise_addr: <string> | default = ""]

# Gossip port to advertise to other members in the cluster. Used for NAT
# traversal.
# CLI flag: -memberlist.advertise-port
[advertise_port: <int> | default = 7946]

# (advanced) The cluster label is an optional string to include in outbound
# packets and gossip streams. Other members in the memberlist cluster will
# discard any message whose label doesn't match the configured one, unless the
# 'cluster-label-verification-disabled' configuration option is set to true.
# CLI flag: -memberlist.cluster-label
[cluster_label: <string> | default = ""]

# (advanced) When true, memberlist doesn't verify that inbound packets and
# gossip streams have the cluster label matching the configured one. This
# verification should be disabled while rolling out the change to the configured
# cluster label in a live memberlist cluster.
# CLI flag: -memberlist.cluster-label-verification-disabled
[cluster_label_verification_disabled: <boolean> | default = false]

# Other cluster members to join. Can be specified multiple times or as a
# comma-separated list. It can be an IP, hostname or an entry specified in the
# DNS Service Discovery format.
# CLI flag: -memberlist.join
[join_members: <list of strings> | default = ]

# (advanced) Min backoff duration to join other cluster members.
# CLI flag: -memberlist.min-join-backoff
[min_join_backoff: <duration> | default = 1s]

# (advanced) Max backoff duration to join other cluster members.
# CLI flag: -memberlist.max-join-backoff
[max_join_backoff: <duration> | default = 1m]

# (advanced) Max number of retries to join other cluster members.
# CLI flag: -memberlist.max-join-retries
[max_join_retries: <int> | default = 10]

# (advanced) Abort if this node fails the fast memberlist cluster joining
# procedure at startup. When enabled, it's guaranteed that other services,
# depending on memberlist, have an updated view over the cluster state when
# they're started.
# CLI flag: -memberlist.abort-if-fast-join-fails
[abort_if_cluster_fast_join_fails: <boolean> | default = false]

# (advanced) Minimum number of seed nodes that must be successfully joined
# during fast-join for it to succeed. Only applies when
# -memberlist.abort-if-fast-join-fails is enabled.
# CLI flag: -memberlist.abort-if-fast-join-fails-min-nodes
[abort_if_cluster_fast_join_fails_min_nodes: <int> | default = 1]

# Abort if this node fails to join memberlist cluster at startup. When enabled,
# it's not guaranteed that other services are started only after the cluster
# state has been successfully updated; use 'abort-if-fast-join-fails' instead.
# CLI flag: -memberlist.abort-if-join-fails
[abort_if_cluster_join_fails: <boolean> | default = false]

# (advanced) If not 0, how often to rejoin the cluster. Occasional rejoin can
# help to fix the cluster split issue, and is harmless otherwise. For example
# when using only few components as a seed nodes (via -memberlist.join), then
# it's recommended to use rejoin. If -memberlist.join points to dynamic service
# that resolves to all gossiping nodes (eg. Kubernetes headless service), then
# rejoin is not needed.
# CLI flag: -memberlist.rejoin-interval
[rejoin_interval: <duration> | default = 0s]

# (experimental) Seed nodes to use for periodic rejoin. Takes precedence over
# -memberlist.join for rejoining. If not specified, -memberlist.join is used.
# Can be specified multiple times or as a comma-separated list. Supports IP,
# hostname, or DNS Service Discovery format.
# CLI flag: -memberlist.rejoin-seed-nodes
[rejoin_seed_nodes: <list of strings> | default = ]

# (advanced) How long to keep LEFT ingesters in the ring.
# CLI flag: -memberlist.left-ingesters-timeout
[left_ingesters_timeout: <duration> | default = 5m]

# (experimental) How long to keep obsolete entries in the KV store.
# CLI flag: -memberlist.obsolete-entries-timeout
[obsolete_entries_timeout: <duration> | default = 30s]

# (advanced) Timeout for leaving memberlist cluster.
# CLI flag: -memberlist.leave-timeout
[leave_timeout: <duration> | default = 20s]

# (advanced) Timeout for broadcasting all remaining locally-generated updates to
# other nodes when shutting down. Only used if there are nodes left in the
# memberlist cluster, and only applies to locally-generated updates, not to
# broadcast messages that are result of incoming gossip updates. 0 = no timeout,
# wait until all locally-generated updates are sent.
# CLI flag: -memberlist.broadcast-timeout-for-local-updates-on-shutdown
[broadcast_timeout_for_local_updates_on_shutdown: <duration> | default = 10s]

# (advanced) How much space to use for keeping received and sent messages in
# memory for troubleshooting (two buffers). 0 to disable.
# CLI flag: -memberlist.message-history-buffer-bytes
[message_history_buffer_bytes: <int> | default = 0]

# (advanced) Size of the buffered channel for the WatchPrefix function.
# CLI flag: -memberlist.watch-prefix-buffer-size
[watch_prefix_buffer_size: <int> | default = 128]

# IP address to listen on for gossip messages. Multiple addresses may be
# specified. Defaults to 0.0.0.0
# CLI flag: -memberlist.bind-addr
[bind_addr: <list of strings> | default = []]

# Port to listen on for gossip messages.
# CLI flag: -memberlist.bind-port
[bind_port: <int> | default = 7946]

# (advanced) Timeout used when connecting to other nodes to send packet.
# CLI flag: -memberlist.packet-dial-timeout
[packet_dial_timeout: <duration> | default = 500ms]

# (advanced) Timeout for writing 'packet' data.
# CLI flag: -memberlist.packet-write-timeout
[packet_write_timeout: <duration> | default = 500ms]

# (advanced) Maximum number of concurrent writes to other nodes.
# CLI flag: -memberlist.max-concurrent-writes
[max_concurrent_writes: <int> | default = 5]

# (advanced) Timeout for acquiring one of the concurrent write slots. After this
# time, the message will be dropped.
# CLI flag: -memberlist.acquire-writer-timeout
[acquire_writer_timeout: <duration> | default = 1s]

# (advanced) Enable TLS on the memberlist transport layer.
# CLI flag: -memberlist.tls-enabled
[tls_enabled: <boolean> | default = false]

# (advanced) Path to the client certificate, which will be used for
# authenticating with the server. Also requires the key path to be configured.
# CLI flag: -memberlist.tls-cert-path
[tls_cert_path: <string> | default = ""]

# (advanced) Path to the key for the client certificate. Also requires the
# client certificate to be configured.
# CLI flag: -memberlist.tls-key-path
[tls_key_path: <string> | default = ""]

# (advanced) Path to the CA certificates to validate server certificate against.
# If not set, the host's root CA certificates are used.
# CLI flag: -memberlist.tls-ca-path
[tls_ca_path: <string> | default = ""]

# (advanced) Override the expected name on the server certificate.
# CLI flag: -memberlist.tls-server-name
[tls_server_name: <string> | default = ""]

# (advanced) Skip validating server certificate.
# CLI flag: -memberlist.tls-insecure-skip-verify
[tls_insecure_skip_verify: <boolean> | default = false]

# (advanced) Override the default cipher suite list (separated by commas).
# Allowed values:
#
# Secure Ciphers:
# - TLS_AES_128_GCM_SHA256
# - TLS_AES_256_GCM_SHA384
# - TLS_CHACHA20_POLY1305_SHA256
# - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
# - TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
# - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
# - TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
# - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
# - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
# - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
# - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
# - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
# - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
#
# Insecure Ciphers:
# - TLS_RSA_WITH_RC4_128_SHA
# - TLS_RSA_WITH_3DES_EDE_CBC_SHA
# - TLS_RSA_WITH_AES_128_CBC_SHA
# - TLS_RSA_WITH_AES_256_CBC_SHA
# - TLS_RSA_WITH_AES_128_CBC_SHA256
# - TLS_RSA_WITH_AES_128_GCM_SHA256
# - TLS_RSA_WITH_AES_256_GCM_SHA384
# - TLS_ECDHE_ECDSA_WITH_RC4_128_SHA
# - TLS_ECDHE_RSA_WITH_RC4_128_SHA
# - TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
# - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
# - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
# CLI flag: -memberlist.tls-cipher-suites
[tls_cipher_suites: <string> | default = ""]

# (advanced) Override the default minimum TLS version. Allowed values:
# VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13
# CLI flag: -memberlist.tls-min-version
[tls_min_version: <string> | default = ""]

zone_aware_routing:
  # (experimental) Enable zone-aware routing for memberlist gossip.
  # CLI flag: -memberlist.zone-aware-routing.enabled
  [enabled: <boolean> | default = false]

  # (experimental) Availability zone where this node is running.
  # CLI flag: -memberlist.zone-aware-routing.instance-availability-zone
  [instance_availability_zone: <string> | default = ""]

  # (experimental) Role of this node in the cluster. Valid values: member,
  # bridge.
  # CLI flag: -memberlist.zone-aware-routing.role
  [role: <string> | default = "member"]

propagation_delay_tracker:
  # (experimental) Enable the propagation delay tracker to measure gossip
  # propagation delay.
  # CLI flag: -memberlist.propagation-delay-tracker.enabled
  [enabled: <boolean> | default = false]

  # (experimental) How often to publish beacons for propagation tracking.
  # CLI flag: -memberlist.propagation-delay-tracker.beacon-interval
  [beacon_interval: <duration> | default = 1m]

  # (experimental) How long a beacon lives before being garbage collected.
  # CLI flag: -memberlist.propagation-delay-tracker.beacon-lifetime
  [beacon_lifetime: <duration> | default = 10m]

  # (experimental) Log warning when beacon propagation delay exceeds this
  # threshold. 0 disables logging.
  # CLI flag: -memberlist.propagation-delay-tracker.log-beacons-latency-longer-than
  [log_beacons_latency_longer_than: <duration> | default = 0s]

limits

The limits block configures default and per-tenant limits imposed by components.

# Per-tenant push request rate limit in requests per second. 0 to disable.
# CLI flag: -distributor.request-rate-limit
[request_rate: <float> | default = 0]

# Per-tenant allowed push request burst size. 0 to disable.
# CLI flag: -distributor.request-burst-size
[request_burst_size: <int> | default = 0]

# Per-tenant ingestion rate limit in samples per second.
# CLI flag: -distributor.ingestion-rate-limit
[ingestion_rate: <float> | default = 10000]

# Per-tenant allowed ingestion burst size (in number of samples).
# CLI flag: -distributor.ingestion-burst-size
[ingestion_burst_size: <int> | default = 200000]

# (experimental) Per-tenant burst factor which is the maximum burst size allowed
# as a multiple of the per-tenant ingestion rate, this burst-factor must be
# greater than or equal to 1. If this is set it will override the
# ingestion-burst-size option.
# CLI flag: -distributor.ingestion-burst-factor
[ingestion_burst_factor: <float> | default = 0]

# Flag to enable, for all tenants, handling of samples with external labels
# identifying replicas in an HA Prometheus setup.
# CLI flag: -distributor.ha-tracker.enable-for-all-users
[accept_ha_samples: <boolean> | default = false]

# (experimental) Experimental: evaluate HA deduplication per timeseries within a
# write request instead of applying the first series' decision to the whole
# request. Enables correct behavior for mixed-label requests such as Prometheus
# federation or metrics proxies.
# CLI flag: -distributor.ha-tracker.per-sample-dedupe
[ha_tracker_per_sample_dedupe: <boolean> | default = false]

# Prometheus label to look for in samples to identify a Prometheus HA cluster.
# CLI flag: -distributor.ha-tracker.cluster
[ha_cluster_label: <string> | default = "cluster"]

# Prometheus label to look for in samples to identify a Prometheus HA replica.
# CLI flag: -distributor.ha-tracker.replica
[ha_replica_label: <string> | default = "__replica__"]

# Maximum number of clusters that HA tracker will keep track of for a single
# tenant. 0 to disable the limit.
# CLI flag: -distributor.ha-tracker.max-clusters
[ha_max_clusters: <int> | default = 100]

# (advanced) Update the timestamp in the KV store for a given cluster/replica
# only after this amount of time has passed since the current stored timestamp.
# CLI flag: -distributor.ha-tracker.update-timeout
[ha_tracker_update_timeout: <duration> | default = 15s]

# (advanced) Maximum jitter applied to the update timeout, in order to spread
# the HA heartbeats over time.
# CLI flag: -distributor.ha-tracker.update-timeout-jitter-max
[ha_tracker_update_timeout_jitter_max: <duration> | default = 5s]

# (advanced) If we don't receive any samples from the accepted replica for a
# cluster in this amount of time we will failover to the next replica we receive
# a sample from. This value must be greater than the update timeout.
# CLI flag: -distributor.ha-tracker.failover-timeout
[ha_tracker_failover_timeout: <duration> | default = 30s]

# (advanced) Additional timeout to use for failover that uses the earliest
# sample time instead of the current time. Defaults to 0, which is disabled.
# This is useful to prevent samples from being too close together during
# failover when write requests are delayed so that the sample time is earlier
# than the current time.
# CLI flag: -distributor.ha-tracker.failover-sample-timeout
[ha_tracker_sample_failover_timeout: <duration> | default = 0s]

# (advanced) This flag can be used to specify label names that to drop during
# sample ingestion within the distributor and can be repeated in order to drop
# multiple labels.
# CLI flag: -distributor.drop-label
[drop_labels: <list of strings> | default = []]

# Maximum length accepted for label names
# CLI flag: -validation.max-length-label-name
[max_label_name_length: <int> | default = 1024]

# Maximum length accepted for label value. This setting also applies to the
# metric name
# CLI flag: -validation.max-length-label-value
[max_label_value_length: <int> | default = 2048]

# (experimental) What to do for label values over the length limit. Options are:
# 'error', 'truncate', 'drop'. For 'truncate', the hash of the full value
# replaces the end portion of the value. For 'drop', the hash fully replaces the
# value.
# CLI flag: -validation.label-value-length-over-limit-strategy
[label_value_length_over_limit_strategy: <string> | default = "error"]

# Maximum number of label names per series.
# CLI flag: -validation.max-label-names-per-series
[max_label_names_per_series: <int> | default = 30]

# Maximum number of label names per info series. Has no effect if less than the
# value of the maximum number of label names per series option
# (-validation.max-label-names-per-series)
# CLI flag: -validation.max-label-names-per-info-series
[max_label_names_per_info_series: <int> | default = 80]

# Maximum length accepted for metric metadata. Metadata refers to Metric Name,
# HELP and UNIT. Longer metadata is dropped except for HELP which is truncated.
# CLI flag: -validation.max-metadata-length
[max_metadata_length: <int> | default = 1024]

# Maximum number of buckets per native histogram sample. 0 to disable the limit.
# CLI flag: -validation.max-native-histogram-buckets
[max_native_histogram_buckets: <int> | default = 0]

# (experimental) Maximum number of exemplars per series per request. 0 to
# disable limit in request. The exceeding exemplars are dropped.
# CLI flag: -distributor.max-exemplars-per-series-per-request
[max_exemplars_per_series_per_request: <int> | default = 0]

# Whether to reduce or reject native histogram samples with more buckets than
# the configured limit.
# CLI flag: -validation.reduce-native-histogram-over-max-buckets
[reduce_native_histogram_over_max_buckets: <boolean> | default = true]

# (advanced) Controls how far into the future incoming samples and exemplars are
# accepted compared to the wall clock. Any sample or exemplar will be rejected
# if its timestamp is greater than '(now + creation_grace_period)'. This
# configuration is enforced in the distributor and ingester.
# CLI flag: -validation.create-grace-period
[creation_grace_period: <duration> | default = 10m]

# (advanced) Controls how far into the past incoming samples and exemplars are
# accepted compared to the wall clock. Any sample or exemplar will be rejected
# if its timestamp is lower than '(now - OOO window - past_grace_period)'. This
# configuration is enforced in the distributor and ingester. 0 to disable.
# CLI flag: -validation.past-grace-period
[past_grace_period: <duration> | default = 0s]

# (experimental) When enabled and past_grace_period is 0, the distributor
# rejects samples whose timestamp is older than '(now -
# out_of_order_time_window)'. This matches what the ingester will reject. Has no
# effect when past_grace_period is greater than 0.
# CLI flag: -validation.enforce-out-of-order-window-on-distributor
[enforce_out_of_order_window_on_distributor: <boolean> | default = false]

# (advanced) Enforce every metadata has a metric name.
# CLI flag: -validation.enforce-metadata-metric-name
[enforce_metadata_metric_name: <boolean> | default = true]

# The tenant's shard size used by shuffle-sharding. This value is the total size
# of the shard (ie. it is not the number of ingesters in the shard per zone, but
# the number of ingesters in the shard across all zones, if zone-awareness is
# enabled). Must be set both on ingesters and distributors. 0 disables shuffle
# sharding.
# CLI flag: -distributor.ingestion-tenant-shard-size
[ingestion_tenant_shard_size: <int> | default = 0]

# (experimental) List of metric relabel configurations. Note that in most
# situations, it is more effective to use metrics relabeling directly in the
# Prometheus server, e.g. remote_write.write_relabel_configs. Labels available
# during the relabeling phase and cleaned afterwards: __meta_tenant_id
[metric_relabel_configs: <relabel_config...> | default = ]

# The maximum number of in-memory series per tenant, across the cluster before
# replication. 0 to disable.
# CLI flag: -ingester.max-global-series-per-user
[max_global_series_per_user: <int> | default = 150000]

# The maximum number of in-memory series per metric name, across the cluster
# before replication. 0 to disable.
# CLI flag: -ingester.max-global-series-per-metric
[max_global_series_per_metric: <int> | default = 0]

# The maximum number of in-memory metrics with metadata per tenant, across the
# cluster. 0 to disable.
# CLI flag: -ingester.max-global-metadata-per-user
[max_global_metadata_per_user: <int> | default = 0]

# The maximum number of metadata per metric, across the cluster. 0 to disable.
# CLI flag: -ingester.max-global-metadata-per-metric
[max_global_metadata_per_metric: <int> | default = 0]

# (experimental) The maximum number of exemplars in memory, across the cluster.
# 0 to disable exemplars ingestion.
# CLI flag: -ingester.max-global-exemplars-per-user
[max_global_exemplars_per_user: <int> | default = 0]

# (experimental) Whether to ignore exemplars with out-of-order timestamps. If
# enabled, exemplars with out-of-order timestamps are silently dropped,
# otherwise they cause partial errors.
# CLI flag: -ingester.ignore-ooo-exemplars
[ignore_ooo_exemplars: <boolean> | default = false]

# (experimental) When the number of owned series for a tenant across the cluster
# exceeds this threshold, trigger early head compaction. 0 to disable.
# CLI flag: -ingester.early-head-compaction-owned-series-threshold
[early_head_compaction_owned_series_threshold: <int> | default = 0]

# (experimental) Minimum estimated series reduction percentage (0-100) required
# to trigger per-tenant early compaction.
# CLI flag: -ingester.early-head-compaction-min-estimated-series-reduction-percentage
[early_head_compaction_min_estimated_series_reduction_percentage: <int> | default = 15]

# (experimental) Enable ingestion of native histogram samples. If false, native
# histogram samples are ignored without an error. To query native histograms
# with query-sharding enabled make sure to set
# -query-frontend.query-result-response-format to 'protobuf'.
# CLI flag: -ingester.native-histograms-ingestion-enabled
[native_histograms_ingestion_enabled: <boolean> | default = true]

# (advanced) Custom trackers for active metrics. If there are active series
# matching a provided matcher (map value), the count is exposed in the custom
# trackers metric labeled using the tracker name (map key). Zero-valued counts
# are not exposed and are removed when they go back to zero.
# Example:
#   The following configuration counts the active series coming from dev and
#   prod namespaces for each tenant and labels them as {name="dev"} and
#   {name="prod"} in the cortex_ingester_active_series_custom_tracker metric.
#   active_series_custom_trackers:
#       dev: '{namespace=~"dev-.*"}'
#       prod: '{namespace=~"prod-.*"}'
# CLI flag: -ingester.active-series-custom-trackers
[active_series_custom_trackers: <map of tracker name (string) to matcher (string)> | default = ]

# (advanced) Additional custom trackers for active metrics merged on top of the
# base custom trackers. You can use this configuration option to define the base
# custom trackers globally for all tenants, and then use the additional trackers
# to add extra trackers on a per-tenant basis.
# Example:
#   The following configuration counts the active series coming from dev and
#   prod namespaces for each tenant and labels them as {name="dev"} and
#   {name="prod"} in the cortex_ingester_active_series_custom_tracker metric.
#   active_series_additional_custom_trackers:
#       dev: '{namespace=~"dev-.*"}'
#       prod: '{namespace=~"prod-.*"}'
[active_series_additional_custom_trackers: <map of tracker name (string) to matcher (string)> | default = ]

# (experimental) Maximum number of additional custom trackers for active series
# that you can configure per tenant. This limit only applies to additional
# custom trackers. Set to 0 to disable the limit.
# CLI flag: -validation.max-active-series-additional-custom-trackers
[max_active_series_additional_custom_trackers: <int> | default = 0]

# Non-zero value enables out-of-order support for most recent samples that are
# within the time window in relation to the TSDB's maximum time, i.e., within
# [db.maxTime-timeWindow, db.maxTime]). The ingester will need more memory as a
# factor of rate of out-of-order samples being ingested and the number of series
# that are getting out-of-order samples. If query falls into this window, cached
# results will use value from
# -query-frontend.results-cache-ttl-for-out-of-order-time-window option to
# specify TTL for resulting cache entry.
# CLI flag: -ingester.out-of-order-time-window
[out_of_order_time_window: <duration> | default = 0s]

# (advanced) Whether the shipper should label out-of-order blocks with an
# external label before uploading them. Setting this label will compact
# out-of-order blocks separately from non-out-of-order blocks
# CLI flag: -ingester.out-of-order-blocks-external-label-enabled
[out_of_order_blocks_external_label_enabled: <boolean> | default = false]

# (experimental) Label used to define the group label for metrics separation.
# For each write request, the group is obtained from the first non-empty group
# label from the first timeseries in the incoming list of timeseries. Specific
# distributor and ingester metrics will be further separated adding a 'group'
# label with group label's value. Currently applies to the following metrics:
# cortex_discarded_samples_total
# CLI flag: -validation.separate-metrics-group-label
[separate_metrics_group_label: <string> | default = ""]

# Maximum number of chunks that can be fetched in a single query from ingesters
# and store-gateways. This limit is enforced in the querier, ruler and
# store-gateway. 0 to disable.
# CLI flag: -querier.max-fetched-chunks-per-query
[max_fetched_chunks_per_query: <int> | default = 2000000]

# (advanced) Maximum number of chunks estimated to be fetched in a single query
# from ingesters and store-gateways, as a multiple of
# -querier.max-fetched-chunks-per-query. This limit is enforced in the querier.
# Must be greater than or equal to 1, or 0 to disable.
# CLI flag: -querier.max-estimated-fetched-chunks-per-query-multiplier
[max_estimated_fetched_chunks_per_query_multiplier: <float> | default = 0]

# The maximum number of unique series for which a query can fetch samples from
# ingesters and store-gateways. This limit is enforced in the querier, ruler and
# store-gateway. 0 to disable
# CLI flag: -querier.max-fetched-series-per-query
[max_fetched_series_per_query: <int> | default = 0]

# The maximum size of all chunks in bytes that a query can fetch from ingesters
# and store-gateways. This limit is enforced in the querier and ruler. 0 to
# disable.
# CLI flag: -querier.max-fetched-chunk-bytes-per-query
[max_fetched_chunk_bytes_per_query: <int> | default = 0]

# (experimental) The maximum estimated memory a single query can consume at
# once, in bytes. This limit is only enforced when Mimir's query engine is in
# use. This limit is enforced in the querier. 0 to disable.
# CLI flag: -querier.max-estimated-memory-consumption-per-query
[max_estimated_memory_consumption_per_query: <int> | default = 0]

# Limit how long back data (series and metadata) can be queried, up until
# <lookback> duration ago. This limit is enforced in the query-frontend, querier
# and ruler for instant, range and remote read queries. For metadata queries
# like series, label names, label values queries the limit is enforced in the
# querier and ruler. If the requested time range is outside the allowed range,
# the request will not fail but will be manipulated to only query data within
# the allowed time range. 0 to disable.
# CLI flag: -querier.max-query-lookback
[max_query_lookback: <duration> | default = 0s]

# Limit the time range for partial queries at the querier level.
# CLI flag: -querier.max-partial-query-length
[max_partial_query_length: <duration> | default = 0s]

# Maximum number of split (by time) or partial (by shard) queries that will be
# scheduled in parallel by the query-frontend for a single input query. This
# limit is introduced to have a fairer query scheduling and avoid a single query
# over a large time range saturating all available queriers.
# CLI flag: -querier.max-query-parallelism
[max_query_parallelism: <int> | default = 14]

# Limit the time range (end - start time) of series, label names and values
# queries. This limit is enforced in the querier. If the requested time range is
# outside the allowed range, the request will not fail but will be manipulated
# to only query data within the allowed time range. 0 to disable.
# CLI flag: -store.max-labels-query-length
[max_labels_query_length: <duration> | default = 0s]

# Maximum number of series, the series endpoint queries. This limit is enforced
# in the querier. If the requested limit is outside of the allowed value, the
# request doesn't fail, but is manipulated to only query data up to the allowed
# limit. Set to 0 to disable.
# CLI flag: -querier.max-series-query-limit
[max_series_query_limit: <int> | default = 0]

# Maximum number of names the label names endpoint returns. This limit is
# enforced in the querier. If the requested limit is outside of the allowed
# value, the request doesn't fail, but is manipulated to only query data up to
# the allowed limit. Set to 0 to disable.
# CLI flag: -querier.max-label-names-limit
[max_label_names_limit: <int> | default = 0]

# Maximum number of values the label values endpoint returns. This limit is
# enforced in the querier. If the requested limit is outside of the allowed
# value, the request doesn't fail, but is manipulated to only query data up to
# the allowed limit. Set to 0 to disable.
# CLI flag: -querier.max-label-values-limit
[max_label_values_limit: <int> | default = 0]

# (advanced) Most recent allowed cacheable result per-tenant, to prevent caching
# very recent results that might still be in flux.
# CLI flag: -query-frontend.max-cache-freshness
[max_cache_freshness: <duration> | default = 10m]

# Maximum number of queriers that can handle requests for a single tenant. If
# set to 0 or value higher than number of available queriers, *all* queriers
# will handle requests for the tenant. Each frontend (or query-scheduler, if
# used) will select the same set of queriers for the same tenant (given that all
# queriers are connected to all frontends / query-schedulers). This option only
# works with queriers connecting to the query-frontend / query-scheduler, not
# when using downstream URL.
# CLI flag: -query-frontend.max-queriers-per-tenant
[max_queriers_per_tenant: <int> | default = 0]

# The amount of shards to use when doing parallelisation via query sharding by
# tenant. 0 to disable query sharding for tenant. Query sharding implementation
# will adjust the number of query shards based on compactor shards. This allows
# querier to not search the blocks which cannot possibly have the series for
# given query shard.
# CLI flag: -query-frontend.query-sharding-total-shards
[query_sharding_total_shards: <int> | default = 16]

# The max number of sharded queries that can be run for a given received query.
# 0 to disable limit.
# CLI flag: -query-frontend.query-sharding-max-sharded-queries
[query_sharding_max_sharded_queries: <int> | default = 128]

# Disable query sharding for any query containing a regular expression matcher
# longer than the configured number of bytes. 0 to disable the limit.
# CLI flag: -query-frontend.query-sharding-max-regexp-size-bytes
[query_sharding_max_regexp_size_bytes: <int> | default = 4096]

# (advanced) Maximum lookback beyond which queries are not sent to ingester. 0
# means all queries are sent to ingester.
# CLI flag: -querier.query-ingesters-within
[query_ingesters_within: <duration> | default = 13h]

# (experimental) Enable the experimental Prometheus feature for delayed name
# removal within MQE, which only works if remote execution and running sharding
# within MQE is enabled.
# CLI flag: -querier.enable-delayed-name-removal
[enable_delayed_name_removal: <boolean> | default = false]

# Limit the total query time range (end - start time). This limit is enforced in
# the query-frontend on the received instant, range or remote read query.
# CLI flag: -query-frontend.max-total-query-length
[max_total_query_length: <duration> | default = 0s]

# Time to live duration for cached query results. If query falls into
# out-of-order time window,
# -query-frontend.results-cache-ttl-for-out-of-order-time-window is used
# instead.
# CLI flag: -query-frontend.results-cache-ttl
[results_cache_ttl: <duration> | default = 1w]

# Time to live duration for cached query results if query falls into
# out-of-order time window. This is lower than -query-frontend.results-cache-ttl
# so that incoming out-of-order samples are returned in the query results
# sooner.
# CLI flag: -query-frontend.results-cache-ttl-for-out-of-order-time-window
[results_cache_ttl_for_out_of_order_time_window: <duration> | default = 10m]

# Time to live duration for cached cardinality query results. The value 0
# disables the cache.
# CLI flag: -query-frontend.results-cache-ttl-for-cardinality-query
[results_cache_ttl_for_cardinality_query: <duration> | default = 0s]

# Time to live duration for cached label names and label values query results.
# The value 0 disables the cache.
# CLI flag: -query-frontend.results-cache-ttl-for-labels-query
[results_cache_ttl_for_labels_query: <duration> | default = 0s]

# Time to live duration for cached non-transient errors
# CLI flag: -query-frontend.results-cache-ttl-for-errors
[results_cache_ttl_for_errors: <duration> | default = 5m]

# (advanced) Cache requests that are not step-aligned.
# CLI flag: -query-frontend.cache-unaligned-requests
[cache_unaligned_requests: <boolean> | default = false]

# Max size of the raw query, in bytes. This limit is enforced by the
# query-frontend for instant, range and remote read queries. 0 to not apply a
# limit to the size of the query.
# CLI flag: -query-frontend.max-query-expression-size-bytes
[max_query_expression_size_bytes: <int> | default = 0]

# List of queries to block.
# Example:
#   The following configuration shows various ways to block queries: by pattern,
#   by time range, or by combining both. Rules are validated at configuration
#   load; an error is returned if the pattern is missing or, when regex: true,
#   the pattern is not a valid regular expression. Use pattern: ".*" with regex:
#   true to match all queries. Time range filtering blocks queries with
#   durations exceeding the specified threshold.
#   blocked_queries:
#       - pattern: rate(metric_counter[5m])
#         regex: false
#         reason: because the query is misconfigured
#       - pattern: .*expensive.*
#         regex: true
#         reason: expensive queries over 7 days are blocked
#         time_range_longer_than: 1w
#       - pattern: .*
#         regex: true
#         reason: queries longer than 21 days are blocked
#         time_range_longer_than: 3w
blocked_queries:
  - # PromQL expression pattern to match. Rules without a pattern are a
    # configuration error.
    [pattern: <string> | default = ""]

    # If true, the pattern is treated as a regular expression; an invalid
    # regular expression is a configuration error. If false, the pattern is
    # treated as a literal match.
    [regex: <boolean> | default = ]

    # Reason returned to clients when rejecting matching queries.
    [reason: <string> | default = ""]

    # If true, only block the query if the query time range is not aligned to
    # the step, meaning the query is not eligible for range query result
    # caching. If enabled, instant queries and remote read requests will not be
    # blocked.
    [unaligned_range_queries: <boolean> | default = ]

    # Block queries with time range longer than this duration. Set to 0 to
    # disable.
    [time_range_longer_than: <duration> | default = ]

    # Block queries where the step is shorter than this duration. Instant
    # queries and queries with no step are not blocked. Set to 0 to disable.
    [step_size_shorter_than: <duration> | default = ]

# (experimental) List of queries to limit and duration to limit them for.
# Example:
#   The following configuration limits the query "rate(metric_counter[5m])" to
#   running, at most, every minute.
#   limited_queries:
#       - query: rate(metric_counter[5m])
#         allowed_frequency: 1m0s
limited_queries:
  - # Literal PromQL expression to match.
    [query: <string> | default = ""]

    # Minimum duration between matching queries. If a matching query arrives
    # more often than this, it is rejected.
    [allowed_frequency: <duration> | default = ]

# (experimental) List of HTTP requests to block.
# Example:
#   The following configuration blocks all GET requests to /foo when the "limit"
#   parameter is set to 100.
#   blocked_requests:
#       - path: /foo
#         method: GET
#         query_params:
#           limit:
#               value: "100"
blocked_requests:
  - # Path to match, including leading slash (/). Leave blank to match all
    # paths.
    [path: <string> | default = ""]

    # HTTP method to match. Leave blank to match all methods.
    [method: <string> | default = ""]

    # Query parameters to match. Requests must have all of the provided query
    # parameters to be considered a match.
    [query_params:]
      <string>:
        # Value to match.
        [value: <string> | default = ""]

        # If true, the value is treated as a regular expression. If false, the
        # value is treated as a literal match.
        [is_regexp: <boolean> | default = ]

# Mutate incoming queries to align their start and end with their step to
# improve result caching.
# CLI flag: -query-frontend.align-queries-with-step
[align_queries_with_step: <boolean> | default = false]

# Enable certain experimental PromQL functions, which are subject to being
# changed or removed at any time, on a per-tenant basis. Defaults to empty which
# means all experimental functions are disabled. Set to 'all' to enable all
# experimental functions.
# CLI flag: -query-frontend.enabled-promql-experimental-functions
[enabled_promql_experimental_functions: <string> | default = ""]

# Enable certain experimental PromQL extended range selector modifiers, which
# are subject to being changed or removed at any time, on a per-tenant basis.
# Defaults to empty which means all experimental modifiers are disabled. Set to
# 'all' to enable all experimental modifiers.
# CLI flag: -query-frontend.enabled-promql-extended-range-selectors
[enabled_promql_extended_range_selectors: <string> | default = ""]

# (experimental) Rewrite queries using the same range selector and resolution
# [X:X] which don't work in Prometheus 3.0 to a nearly identical form that works
# with Prometheus 3.0 semantics
# CLI flag: -query-frontend.prom2-range-compat
[prom2_range_compat: <boolean> | default = false]

# (experimental) Enable spinning off subqueries from instant queries as range
# queries to optimize their performance.
# CLI flag: -query-frontend.subquery-spin-off-enabled
[subquery_spin_off_enabled: <boolean> | default = false]

# (advanced) Enable labels query optimizations. When enabled, the query-frontend
# may rewrite labels queries to improve their performance.
# CLI flag: -query-frontend.labels-query-optimizer-enabled
[labels_query_optimizer_enabled: <boolean> | default = true]

# Enables endpoints used for cardinality analysis.
# CLI flag: -querier.cardinality-analysis-enabled
[cardinality_analysis_enabled: <boolean> | default = false]

# Maximum size in bytes of distinct label names and values. When querier
# receives response from ingester, it merges the response with responses from
# other ingesters. This maximum size limit is applied to the merged(distinct)
# results. If the limit is reached, an error is returned.
# CLI flag: -querier.label-names-and-values-results-max-size-bytes
[label_names_and_values_results_max_size_bytes: <int> | default = 419430400]

# Maximum number of label names allowed to be queried in a single
# /api/v1/cardinality/label_values API call.
# CLI flag: -querier.label-values-max-cardinality-label-names-per-request
[label_values_max_cardinality_label_names_per_request: <int> | default = 100]

# (experimental) Maximum number of series that can be requested in a single
# cardinality API request.
# CLI flag: -querier.cardinality-api-max-series-limit
[cardinality_analysis_max_results: <int> | default = 500]

# (advanced) Maximum size of an active series or active native histogram series
# request result shard in bytes. 0 to disable.
# CLI flag: -querier.active-series-results-max-size-bytes
[active_series_results_max_size_bytes: <int> | default = 419430400]

# (experimental)
cost_attribution_labels_structured:
  -     [input: <string> | default = ""]

    [output: <string> | default = ""]

# (experimental) Maximum cardinality of cost attribution labels allowed per
# user.
# CLI flag: -validation.max-cost-attribution-cardinality
[max_cost_attribution_cardinality: <int> | default = 2000]

# (experimental) Defines how long cost attribution stays in overflow before
# attempting a reset, with received/discarded samples extending the cooldown if
# overflow persists, while active series reset and restart tracking after the
# cooldown.
# CLI flag: -validation.cost-attribution-cooldown
[cost_attribution_cooldown: <duration> | default = 0s]

# Duration to delay the evaluation of rules to ensure the underlying metrics
# have been pushed.
# CLI flag: -ruler.evaluation-delay-duration
[ruler_evaluation_delay_duration: <duration> | default = 1m]

# (experimental) The maximum tolerated ingestion delay for eventually consistent
# rule evaluations. Set to 0 to disable the enforcement.
# CLI flag: -ruler.evaluation-consistency-max-delay
[ruler_evaluation_consistency_max_delay: <duration> | default = 0s]

# The tenant's shard size when sharding is used by ruler. Value of 0 disables
# shuffle sharding for the tenant, and tenant rules will be sharded across all
# ruler replicas.
# CLI flag: -ruler.tenant-shard-size
[ruler_tenant_shard_size: <int> | default = 0]

# Maximum number of rules per rule group per-tenant. 0 to disable.
# CLI flag: -ruler.max-rules-per-rule-group
[ruler_max_rules_per_rule_group: <int> | default = 20]

# Maximum number of rule groups per-tenant. 0 to disable.
# CLI flag: -ruler.max-rule-groups-per-tenant
[ruler_max_rule_groups_per_tenant: <int> | default = 70]

# Controls whether recording rules evaluation is enabled. This configuration
# option can be used to forcefully disable recording rules evaluation on a
# per-tenant basis.
# CLI flag: -ruler.recording-rules-evaluation-enabled
[ruler_recording_rules_evaluation_enabled: <boolean> | default = true]

# Controls whether alerting rules evaluation is enabled. This configuration
# option can be used to forcefully disable alerting rules evaluation on a
# per-tenant basis.
# CLI flag: -ruler.alerting-rules-evaluation-enabled
[ruler_alerting_rules_evaluation_enabled: <boolean> | default = true]

# (advanced) True to enable a re-sync of the configured rule groups as soon as
# they're changed via ruler's config API. This re-sync is in addition of the
# periodic syncing. When enabled, it may take up to few tens of seconds before a
# configuration change triggers the re-sync.
# CLI flag: -ruler.sync-rules-on-changes-enabled
[ruler_sync_rules_on_changes_enabled: <boolean> | default = true]

# (experimental) Maximum number of rules per rule group by namespace. Value is a
# map, where each key is the namespace and value is the number of rules allowed
# in the namespace (int). On the command line, this map is given in a JSON
# format. The number of rules specified has the same meaning as
# -ruler.max-rules-per-rule-group, but only applies for the specific namespace.
# If specified, it supersedes -ruler.max-rules-per-rule-group.
# CLI flag: -ruler.max-rules-per-rule-group-by-namespace
[ruler_max_rules_per_rule_group_by_namespace: <map of string to int> | default = {}]

# (experimental) Maximum number of rule groups per tenant by namespace. Value is
# a map, where each key is the namespace and value is the number of rule groups
# allowed in the namespace (int). On the command line, this map is given in a
# JSON format. The number of rule groups specified has the same meaning as
# -ruler.max-rule-groups-per-tenant, but only applies for the specific
# namespace. If specified, it supersedes -ruler.max-rule-groups-per-tenant.
# CLI flag: -ruler.max-rule-groups-per-tenant-by-namespace
[ruler_max_rule_groups_per_tenant_by_namespace: <map of string to int> | default = {}]

# (experimental) List of namespaces that are protected from modification unless
# a special HTTP header is used. If a namespace is protected, it can only be
# read, not modified via the ruler's configuration API. The value is a list of
# strings, where each string is a namespace name. On the command line, this list
# is given as a comma-separated list.
# CLI flag: -ruler.protected-namespaces
[ruler_protected_namespaces: <string> | default = ""]

# (experimental) Maximum number of independent rules that can run concurrently
# for each tenant. Depends on ruler.max-independent-rule-evaluation-concurrency
# being greater than 0. Ideally this flag should be a lower value. 0 to disable.
# CLI flag: -ruler.max-independent-rule-evaluation-concurrency-per-tenant
[ruler_max_independent_rule_evaluation_concurrency_per_tenant: <int> | default = 4]

# Per-tenant Alertmanager client configuration. If not supplied, the tenant's
# notifications are sent to the ruler-wide default.
ruler_alertmanager_client_config:
  # Comma-separated list of URL(s) of the Alertmanager(s) to send notifications
  # to. Each URL is treated as a separate group. Multiple Alertmanagers in HA
  # per group can be supported by using DNS service discovery format,
  # comprehensive of the scheme. Basic auth is supported as part of the URL.
  # CLI flag: -ruler.alertmanager-url
  [alertmanager_url: <string> | default = ""]

  # (advanced) Enable TLS for gRPC client connecting to alertmanager.
  # CLI flag: -ruler.alertmanager-client.tls-enabled
  [tls_enabled: <boolean> | default = true]

  # (advanced) Path to the client certificate, which will be used for
  # authenticating with the server. Also requires the key path to be configured.
  # CLI flag: -ruler.alertmanager-client.tls-cert-path
  [tls_cert_path: <string> | default = ""]

  # (advanced) Path to the key for the client certificate. Also requires the
  # client certificate to be configured.
  # CLI flag: -ruler.alertmanager-client.tls-key-path
  [tls_key_path: <string> | default = ""]

  # (advanced) Path to the CA certificates to validate server certificate
  # against. If not set, the host's root CA certificates are used.
  # CLI flag: -ruler.alertmanager-client.tls-ca-path
  [tls_ca_path: <string> | default = ""]

  # (advanced) Override the expected name on the server certificate.
  # CLI flag: -ruler.alertmanager-client.tls-server-name
  [tls_server_name: <string> | default = ""]

  # (advanced) Skip validating server certificate.
  # CLI flag: -ruler.alertmanager-client.tls-insecure-skip-verify
  [tls_insecure_skip_verify: <boolean> | default = false]

  # (advanced) Override the default cipher suite list (separated by commas).
  # Allowed values:
  #
  # Secure Ciphers:
  # - TLS_AES_128_GCM_SHA256
  # - TLS_AES_256_GCM_SHA384
  # - TLS_CHACHA20_POLY1305_SHA256
  # - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
  # - TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
  # - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
  # - TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
  # - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
  # - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
  # - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
  # - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
  # - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
  # - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
  #
  # Insecure Ciphers:
  # - TLS_RSA_WITH_RC4_128_SHA
  # - TLS_RSA_WITH_3DES_EDE_CBC_SHA
  # - TLS_RSA_WITH_AES_128_CBC_SHA
  # - TLS_RSA_WITH_AES_256_CBC_SHA
  # - TLS_RSA_WITH_AES_128_CBC_SHA256
  # - TLS_RSA_WITH_AES_128_GCM_SHA256
  # - TLS_RSA_WITH_AES_256_GCM_SHA384
  # - TLS_ECDHE_ECDSA_WITH_RC4_128_SHA
  # - TLS_ECDHE_RSA_WITH_RC4_128_SHA
  # - TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
  # - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
  # - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
  # CLI flag: -ruler.alertmanager-client.tls-cipher-suites
  [tls_cipher_suites: <string> | default = ""]

  # (advanced) Override the default minimum TLS version. Allowed values:
  # VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13
  # CLI flag: -ruler.alertmanager-client.tls-min-version
  [tls_min_version: <string> | default = ""]

  # HTTP Basic authentication username. It overrides the username set in the URL
  # (if any).
  # CLI flag: -ruler.alertmanager-client.basic-auth-username
  [basic_auth_username: <string> | default = ""]

  # HTTP Basic authentication password. It overrides the password set in the URL
  # (if any).
  # CLI flag: -ruler.alertmanager-client.basic-auth-password
  [basic_auth_password: <string> | default = ""]

  oauth2:
    # OAuth2 client ID. Enables the use of OAuth2 for authenticating with
    # Alertmanager.
    # CLI flag: -ruler.alertmanager-client.oauth.client_id
    [client_id: <string> | default = ""]

    # OAuth2 client secret.
    # CLI flag: -ruler.alertmanager-client.oauth.client_secret
    [client_secret: <string> | default = ""]

    # Endpoint used to fetch access token.
    # CLI flag: -ruler.alertmanager-client.oauth.token_url
    [token_url: <string> | default = ""]

    # Optional scopes to include with the token request.
    # CLI flag: -ruler.alertmanager-client.oauth.scopes
    [scopes: <string> | default = ""]

    # (advanced) Optional additional URL parameters to send to the token URL.
    # CLI flag: -ruler.alertmanager-client.oauth.endpoint-params
    [endpoint_params: <map of string to string> | default = {}]

  # (advanced) Optional HTTP, HTTPS via CONNECT, or SOCKS5 proxy URL to route
  # requests through. Applies to all requests, including auxiliary traffic, such
  # as OAuth token requests.
  # CLI flag: -ruler.alertmanager-client.proxy-url
  [proxy_url: <string> | default = ""]

# (experimental) Minimum allowable evaluation interval for rule groups.
# CLI flag: -ruler.min-rule-evaluation-interval
[ruler_min_rule_evaluation_interval: <duration> | default = 0s]

# (experimental) Maximum number of alerts or series one alerting rule or one
# recording rule respectively can produce. 0 is no limit.
# CLI flag: -ruler.max-rule-evaluation-results
[ruler_max_rule_evaluation_results: <int> | default = 0]

# The tenant's shard size, used when store-gateway sharding is enabled. Value of
# 0 disables shuffle sharding for the tenant, that is all tenant blocks are
# sharded across all store-gateway replicas.
# CLI flag: -store-gateway.tenant-shard-size
[store_gateway_tenant_shard_size: <int> | default = 0]

# (experimental) The tenant's shard size per availability zone when
# zone-awareness is enabled, used when store-gateway sharding is enabled. The
# total shard size is computed as this value multiplied by the number of zones.
# This option takes precedence over -store-gateway.tenant-shard-size.
# CLI flag: -store-gateway.tenant-shard-size-per-zone
[store_gateway_tenant_shard_size_per_zone: <int> | default = 0]

# Delete blocks containing samples older than the specified retention period.
# Also used by query-frontend to avoid querying beyond the retention period by
# instant, range or remote read queries. 0 to disable.
# CLI flag: -compactor.blocks-retention-period
[compactor_blocks_retention_period: <duration> | default = 0s]

# The number of shards to use when splitting blocks. 0 to disable splitting.
# CLI flag: -compactor.split-and-merge-shards
[compactor_split_and_merge_shards: <int> | default = 0]

# The number of shards to use when splitting out-of-order blocks. 0 to use the
# value of -compactor.split-and-merge-shards. Only applies to blocks with the
# out-of-order external label, see
# -ingester.out-of-order-blocks-external-label-enabled.
# CLI flag: -compactor.ooo-split-and-merge-shards
[compactor_ooo_split_and_merge_shards: <int> | default = 0]

# Number of groups that blocks for splitting should be grouped into. Each group
# of blocks is then split separately. Number of output split shards is
# controlled by -compactor.split-and-merge-shards.
# CLI flag: -compactor.split-groups
[compactor_split_groups: <int> | default = 1]

# Max number of compactors that can compact blocks for single tenant. 0 to
# disable the limit and use all compactors.
# CLI flag: -compactor.compactor-tenant-shard-size
[compactor_tenant_shard_size: <int> | default = 0]

# If a partial block (unfinished block without meta.json file) hasn't been
# modified for this time, it will be marked for deletion. The minimum accepted
# value is 4h0m0s: a lower value will be ignored and the feature disabled. 0 to
# disable.
# CLI flag: -compactor.partial-block-deletion-delay
[compactor_partial_block_deletion_delay: <duration> | default = 1d]

# Enable block upload API for the tenant.
# CLI flag: -compactor.block-upload-enabled
[compactor_block_upload_enabled: <boolean> | default = false]

# Enable block upload validation for the tenant.
# CLI flag: -compactor.block-upload-validation-enabled
[compactor_block_upload_validation_enabled: <boolean> | default = true]

# Verify chunks when uploading blocks via the upload API for the tenant.
# CLI flag: -compactor.block-upload-verify-chunks
[compactor_block_upload_verify_chunks: <boolean> | default = true]

# (advanced) Maximum size in bytes of a block that is allowed to be uploaded or
# validated. 0 = no limit.
# CLI flag: -compactor.block-upload-max-block-size-bytes
[compactor_block_upload_max_block_size_bytes: <int> | default = 0]

# (experimental) Blocks uploaded before the lookback aren't considered in
# compactor cycles. If set, this value should be larger than all values in
# `-blocks-storage.tsdb.block-ranges-period`. A value of 0s means that all
# blocks are considered regardless of their upload time.
# CLI flag: -compactor.max-lookback
[compactor_max_lookback: <duration> | default = 0s]

# (advanced) Maximum number of TSDB segment files that the compactor can upload
# concurrently per block.
# CLI flag: -compactor.max-per-block-upload-concurrency
[compactor_max_per_block_upload_concurrency: <int> | default = 8]

# S3 server-side encryption type. Required to enable server-side encryption
# overrides for a specific tenant. If not set, the default S3 client settings
# are used.
[s3_sse_type: <string> | default = ""]

# S3 server-side encryption KMS Key ID. Ignored if the SSE type override is not
# set.
[s3_sse_kms_key_id: <string> | default = ""]

# S3 server-side encryption KMS encryption context. If unset and the key ID
# override is set, the encryption context will not be provided to S3. Ignored if
# the SSE type override is not set.
[s3_sse_kms_encryption_context: <string> | default = ""]

# Comma-separated list of network CIDRs to block in Alertmanager receiver
# integrations.
# CLI flag: -alertmanager.receivers-firewall-block-cidr-networks
[alertmanager_receivers_firewall_block_cidr_networks: <string> | default = ""]

# True to block private and local addresses in Alertmanager receiver
# integrations. It blocks private addresses defined by  RFC 1918 (IPv4
# addresses) and RFC 4193 (IPv6 addresses), as well as loopback, local unicast
# and local multicast addresses.
# CLI flag: -alertmanager.receivers-firewall-block-private-addresses
[alertmanager_receivers_firewall_block_private_addresses: <boolean> | default = false]

# Per-tenant rate limit for sending notifications from Alertmanager in
# notifications/sec. 0 = rate limit disabled. Negative value = no notifications
# are allowed.
# CLI flag: -alertmanager.notification-rate-limit
[alertmanager_notification_rate_limit: <float> | default = 0]

# Per-integration notification rate limits. Value is a map, where each key is
# integration name and value is a rate-limit (float). On command line, this map
# is given in JSON format. Rate limit has the same meaning as
# -alertmanager.notification-rate-limit, but only applies for specific
# integration. Allowed integration names: webhook, email, pagerduty, opsgenie,
# wechat, slack, victorops, pushover, sns, webex, telegram, discord, msteams,
# msteamsv2.
# CLI flag: -alertmanager.notification-rate-limit-per-integration
[alertmanager_notification_rate_limit_per_integration: <map of string to float64> | default = {}]

# Maximum size of the Alertmanager configuration for a tenant. 0 = no limit.
# CLI flag: -alertmanager.max-config-size-bytes
[alertmanager_max_config_size_bytes: <int> | default = 0]

# Maximum number of silences, including expired silences, that a tenant can have
# at once. 0 = no limit.
# CLI flag: -alertmanager.max-silences-count
[alertmanager_max_silences_count: <int> | default = 0]

# Maximum silence size in bytes. 0 = no limit.
# CLI flag: -alertmanager.max-silence-size-bytes
[alertmanager_max_silence_size_bytes: <int> | default = 0]

# Maximum number of templates in tenant's Alertmanager configuration uploaded
# via Alertmanager API. 0 = no limit.
# CLI flag: -alertmanager.max-templates-count
[alertmanager_max_templates_count: <int> | default = 0]

# Maximum size of single template in tenant's Alertmanager configuration
# uploaded via Alertmanager API. 0 = no limit.
# CLI flag: -alertmanager.max-template-size-bytes
[alertmanager_max_template_size_bytes: <int> | default = 0]

# Maximum number of aggregation groups in Alertmanager's dispatcher that a
# tenant can have. Each active aggregation group uses single goroutine. When the
# limit is reached, dispatcher will not dispatch alerts that belong to
# additional aggregation groups, but existing groups will keep working properly.
# 0 = no limit.
# CLI flag: -alertmanager.max-dispatcher-aggregation-groups
[alertmanager_max_dispatcher_aggregation_groups: <int> | default = 0]

# Maximum number of alerts that a single tenant can have. Inserting more alerts
# will fail with a log message and metric increment. 0 = no limit.
# CLI flag: -alertmanager.max-alerts-count
[alertmanager_max_alerts_count: <int> | default = 0]

# Maximum total size of alerts that a single tenant can have, alert size is the
# sum of the bytes of its labels, annotations and generatorURL. Inserting more
# alerts will fail with a log message and metric increment. 0 = no limit.
# CLI flag: -alertmanager.max-alerts-size-bytes
[alertmanager_max_alerts_size_bytes: <int> | default = 0]

# (advanced) Whether to enable automatic suffixes to names of metrics ingested
# through OTLP.
# CLI flag: -distributor.otel-metric-suffixes-enabled
[otel_metric_suffixes_enabled: <boolean> | default = false]

# (experimental) Whether to enable translation of OTel start timestamps to
# Prometheus zero samples in the OTLP endpoint.
# CLI flag: -distributor.otel-created-timestamp-zero-ingestion-enabled
[otel_created_timestamp_zero_ingestion_enabled: <boolean> | default = false]

# (experimental) Optionally specify a comma-separated list of OTel resource
# attributes to promote to labels. E.g.
# 'k8s.cluster.name,host.name,cloud.region'
# CLI flag: -distributor.otel-promote-resource-attributes
[promote_otel_resource_attributes: <string> | default = ""]

# (experimental) Whether to keep identifying OTel resource attributes in the
# target_info metric on top of converting to job and instance labels.
# CLI flag: -distributor.otel-keep-identifying-resource-attributes
[otel_keep_identifying_resource_attributes: <boolean> | default = false]

# (experimental) Whether to convert OTel explicit histograms into native
# histograms with custom buckets.
# CLI flag: -distributor.otel-convert-histograms-to-nhcb
[otel_convert_histograms_to_nhcb: <boolean> | default = false]

# (experimental) Whether to promote OTel scope metadata (scope name, version,
# schema URL, attributes) to corresponding metric labels, prefixed with
# otel_scope_.
# CLI flag: -distributor.otel-promote-scope-metadata
[otel_promote_scope_metadata: <boolean> | default = false]

# (experimental) Whether to enable native ingestion of delta OTLP metrics, which
# will store the raw delta sample values without conversion. If disabled, delta
# metrics will be rejected. Delta support is in an early stage of development.
# The ingestion and querying process is likely to change over time.
# CLI flag: -distributor.otel-native-delta-ingestion
[otel_native_delta_ingestion: <boolean> | default = false]

# (experimental) Translation strategy to apply in OTLP endpoint for metric and
# label names. If unspecified (the default), the strategy is derived from
# -validation.name-validation-scheme and
# -distributor.otel-metric-suffixes-enabled. Supported values: "",
# UnderscoreEscapingWithSuffixes, UnderscoreEscapingWithoutSuffixes,
# NoUTF8EscapingWithSuffixes, NoTranslation.
# CLI flag: -distributor.otel-translation-strategy
[otel_translation_strategy: <string> | default = ""]

# (advanced) If enabled, prefixes label names starting with a single underscore
# with `key_` when translating OTel attribute names. Defaults to true.
# CLI flag: -distributor.otel-label-name-underscore-sanitization
[otel_label_name_underscore_sanitization: <boolean> | default = true]

# (advanced) If enabled, keeps multiple consecutive underscores in label names
# when translating OTel attribute names. Defaults to true.
# CLI flag: -distributor.otel-label-name-preserve-underscores
[otel_label_name_preserve_multiple_underscores: <boolean> | default = true]

# (experimental) The default consistency level to enforce for queries when using
# the ingest storage. Supports values: strong, eventual.
# CLI flag: -ingest-storage.read-consistency
[ingest_storage_read_consistency: <string> | default = "eventual"]

# (experimental) The number of partitions a tenant's data should be sharded to
# when using the ingest storage. Tenants are sharded across partitions using
# shuffle-sharding. 0 disables shuffle sharding and tenant is sharded across all
# partitions.
# CLI flag: -ingest-storage.ingestion-partition-tenant-shard-size
[ingestion_partitions_tenant_shard_size: <int> | default = 0]

# (experimental) The maximum number of partitions a tenant's data should be
# written to when using the ingest storage. When set to a value > 0 and less
# than -ingest-storage.ingestion-partition-tenant-shard-size, writes use fewer
# partitions while reads continue using the full shard size. This allows safely
# reducing the shard size without losing query coverage during the migration. 0
# means the write shard size equals the read shard size.
# CLI flag: -ingest-storage.ingestion-partition-tenant-write-shard-size
[ingestion_partitions_tenant_write_shard_size: <int> | default = 0]

# (experimental) Validation scheme to use for metric and label names.
# Distributors reject time series that do not adhere to this scheme. Rulers
# reject rules with unsupported metric or label names. Supported values: legacy,
# utf8.
# CLI flag: -validation.name-validation-scheme
[name_validation_scheme: <int> | default = legacy]

ingest_storage

The ingest_storage block configures the Kafka-based ingest storage.

# True to enable the ingestion via object storage.
# CLI flag: -ingest-storage.enabled
[enabled: <boolean> | default = false]

kafka:
  # The Kafka seed broker address, or a comma-separated list of seed broker
  # addresses.
  # CLI flag: -ingest-storage.kafka.address
  [address: <string> | default = ""]

  # The Kafka topic name.
  # CLI flag: -ingest-storage.kafka.topic
  [topic: <string> | default = ""]

  # The Kafka client ID.
  # CLI flag: -ingest-storage.kafka.client-id
  [client_id: <string> | default = ""]

  # The rack identifier for this Kafka client. Corresponds to the Kafka
  # client.rack setting.
  # CLI flag: -ingest-storage.kafka.client-rack
  [client_rack: <string> | default = ""]

  # The maximum time allowed to open a connection to a Kafka broker.
  # CLI flag: -ingest-storage.kafka.dial-timeout
  [dial_timeout: <duration> | default = 2s]

  # How long to wait for an incoming write request to be successfully committed
  # to the Kafka backend.
  # CLI flag: -ingest-storage.kafka.write-timeout
  [write_timeout: <duration> | default = 10s]

  # (deprecated) The number of Kafka clients used by producers. When the
  # configured number of clients is greater than 1, partitions are sharded among
  # Kafka clients. A higher number of clients may provide higher write
  # throughput at the cost of additional Metadata requests pressure to Kafka.
  # Deprecated: has no effect (Mimir always uses a single Kafka write client).
  # CLI flag: -ingest-storage.kafka.write-clients
  [write_clients: <int> | default = 1]

  # The SASL mechanism used to authenticate to Kafka. Supported values: PLAIN,
  # SCRAM-SHA-256, SCRAM-SHA-512, OAUTHBEARER, AWS_MSK_IAM. For
  # backwards-compatibility, PLAIN with no username nor password disables SASL.
  # CLI flag: -ingest-storage.kafka.sasl-mechanism
  [sasl_mechanism: <string> | default = "PLAIN"]

  # The username used to authenticate to Kafka using SASL. To enable SASL,
  # configure both the username and password.
  # CLI flag: -ingest-storage.kafka.sasl-username
  [sasl_username: <string> | default = ""]

  # The password used to authenticate to Kafka using SASL. To enable SASL,
  # configure both the username and password.
  # CLI flag: -ingest-storage.kafka.sasl-password
  [sasl_password: <string> | default = ""]

  # The OAuth token to use to authenticate to Kafka. Consider
  # ingest-storage.kafka.sasl-oauthbearer-file-path instead.
  # CLI flag: -ingest-storage.kafka.sasl-oauthbearer-token
  [sasl_oauthbearer_token: <string> | default = ""]

  # Optional authorization ID to use when authenticating to Kafka using SASL
  # OAUTHBEARER.
  # CLI flag: -ingest-storage.kafka.sasl-oauthbearer-zid
  [sasl_oauthbearer_zid: <string> | default = ""]

  # Optional additional OAuth extensions to include when authenticating to Kafka
  # using SASL OAUTHBEARER as a JSON object.
  # CLI flag: -ingest-storage.kafka.sasl-oauthbearer-extensions
  [sasl_oauthbearer_extensions: <map of string to string> | default = {}]

  # Path to a file containing an OAuth token to authenticate to Kafka. Mutually
  # exclusive with ingest-storage.kafka.sasl-oauthbearer-http-socket-path. The
  # file is read anew on every reauthentication, so it can be updated with fresh
  # tokens. The file must be in JSON format, adhering to this JSON schema:
  # {"type": "object", "required": ["token"], "properties": {"token": {"type":
  # "string"}, "zid": {"type": "string"}, "extensions": {"type": "object",
  # "additionalProperties": {"type": "string"}}}
  # CLI flag: -ingest-storage.kafka.sasl-oauthbearer-file-path
  [sasl_oauthbearer_file_path: <string> | default = ""]

  # Path to a Unix domain socket to fetch an OAuth token from via HTTP. Mutually
  # exclusive with ingest-storage.kafka.sasl-oauthbearer-file-path. On every
  # authentication or reauthentication, an HTTP GET / request is made to the
  # socket and the response body is read as JSON. The JSON schema is the same as
  # for ingest-storage.kafka.sasl-oauthbearer-file-path.
  # CLI flag: -ingest-storage.kafka.sasl-oauthbearer-http-socket-path
  [sasl_oauthbearer_http_socket_path: <string> | default = ""]

  # Timeout for requesting the token from the HTTP socket. Effective when
  # ingest-storage.kafka.sasl-oauthbearer-http-socket-path is set.
  # CLI flag: -ingest-storage.kafka.sasl-oauthbearer-http-socket-timeout
  [sasl_oauthbearer_http_socket_timeout: <duration> | default = 10s]

  # The AWS access key ID to authenticate to Kafka using SASL AWS_MSK_IAM.
  # Consider ingest-storage.kafka.sasl-msk-iam-file-path instead.
  # CLI flag: -ingest-storage.kafka.sasl-msk-iam-access-key
  [sasl_msk_iam_access_key: <string> | default = ""]

  # The AWS secret access key to authenticate to Kafka using SASL AWS_MSK_IAM.
  # Consider ingest-storage.kafka.sasl-msk-iam-file-path instead.
  # CLI flag: -ingest-storage.kafka.sasl-msk-iam-secret-key
  [sasl_msk_iam_secret_key: <string> | default = ""]

  # Optional AWS session token to authenticate to Kafka using SASL AWS_MSK_IAM.
  # CLI flag: -ingest-storage.kafka.sasl-msk-iam-session-token
  [sasl_msk_iam_session_token: <string> | default = ""]

  # Optional user agent to use when authenticating to Kafka using SASL
  # AWS_MSK_IAM.
  # CLI flag: -ingest-storage.kafka.sasl-msk-iam-user-agent
  [sasl_msk_iam_user_agent: <string> | default = ""]

  # Path to a file containing AWS credentials to authenticate to Kafka using
  # SASL AWS_MSK_IAM. Mutually exclusive with
  # ingest-storage.kafka.sasl-msk-iam-http-socket-path. The file is read anew on
  # every reauthentication, so it can be updated with fresh credentials. The
  # file must be in JSON format, adhering to this JSON schema: {"type":
  # "object", "required": ["AccessKey", "SecretKey"], "properties":
  # {"AccessKey": {"type": "string"}, "SecretKey": {"type": "string"},
  # "SessionToken": {"type": "string"}, "UserAgent": {"type": "string"}}}
  # CLI flag: -ingest-storage.kafka.sasl-msk-iam-file-path
  [sasl_msk_iam_file_path: <string> | default = ""]

  # Path to a Unix domain socket to fetch AWS credentials from via HTTP.
  # Mutually exclusive with ingest-storage.kafka.sasl-msk-iam-file-path. On
  # every authentication or reauthentication, an HTTP GET / request is made to
  # the socket and the response body is read as JSON. The JSON schema is the
  # same as for ingest-storage.kafka.sasl-msk-iam-file-path.
  # CLI flag: -ingest-storage.kafka.sasl-msk-iam-http-socket-path
  [sasl_msk_iam_http_socket_path: <string> | default = ""]

  # Timeout for requesting AWS credentials from the HTTP socket. Effective when
  # ingest-storage.kafka.sasl-msk-iam-http-socket-path is set.
  # CLI flag: -ingest-storage.kafka.sasl-msk-iam-http-socket-timeout
  [sasl_msk_iam_http_socket_timeout: <duration> | default = 10s]

  # Enable TLS for the Kafka client connection.
  # CLI flag: -ingest-storage.kafka.tls-enabled
  [tls_enabled: <boolean> | default = false]

  # (advanced) Path to the client certificate, which will be used for
  # authenticating with the server. Also requires the key path to be configured.
  # CLI flag: -ingest-storage.kafka.tls-cert-path
  [tls_cert_path: <string> | default = ""]

  # (advanced) Path to the key for the client certificate. Also requires the
  # client certificate to be configured.
  # CLI flag: -ingest-storage.kafka.tls-key-path
  [tls_key_path: <string> | default = ""]

  # (advanced) Path to the CA certificates to validate server certificate
  # against. If not set, the host's root CA certificates are used.
  # CLI flag: -ingest-storage.kafka.tls-ca-path
  [tls_ca_path: <string> | default = ""]

  # (advanced) Override the expected name on the server certificate.
  # CLI flag: -ingest-storage.kafka.tls-server-name
  [tls_server_name: <string> | default = ""]

  # (advanced) Skip validating server certificate.
  # CLI flag: -ingest-storage.kafka.tls-insecure-skip-verify
  [tls_insecure_skip_verify: <boolean> | default = false]

  # (advanced) Override the default cipher suite list (separated by commas).
  # Allowed values:
  #
  # Secure Ciphers:
  # - TLS_AES_128_GCM_SHA256
  # - TLS_AES_256_GCM_SHA384
  # - TLS_CHACHA20_POLY1305_SHA256
  # - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
  # - TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
  # - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
  # - TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
  # - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
  # - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
  # - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
  # - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
  # - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
  # - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
  #
  # Insecure Ciphers:
  # - TLS_RSA_WITH_RC4_128_SHA
  # - TLS_RSA_WITH_3DES_EDE_CBC_SHA
  # - TLS_RSA_WITH_AES_128_CBC_SHA
  # - TLS_RSA_WITH_AES_256_CBC_SHA
  # - TLS_RSA_WITH_AES_128_CBC_SHA256
  # - TLS_RSA_WITH_AES_128_GCM_SHA256
  # - TLS_RSA_WITH_AES_256_GCM_SHA384
  # - TLS_ECDHE_ECDSA_WITH_RC4_128_SHA
  # - TLS_ECDHE_RSA_WITH_RC4_128_SHA
  # - TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
  # - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
  # - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
  # CLI flag: -ingest-storage.kafka.tls-cipher-suites
  [tls_cipher_suites: <string> | default = ""]

  # (advanced) Override the default minimum TLS version. Allowed values:
  # VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13
  # CLI flag: -ingest-storage.kafka.tls-min-version
  [tls_min_version: <string> | default = ""]

  # The consumer group used by the consumer to track the last consumed offset.
  # The consumer group must be different for each ingester. If the configured
  # consumer group contains the '<partition>' placeholder, it is replaced with
  # the actual partition ID owned by the ingester. When empty (recommended),
  # Mimir uses the ingester instance ID to guarantee uniqueness.
  # CLI flag: -ingest-storage.kafka.consumer-group
  [consumer_group: <string> | default = ""]

  # How frequently a consumer should commit the consumed offset to Kafka. The
  # last committed offset is used at startup to continue the consumption from
  # where it was left.
  # CLI flag: -ingest-storage.kafka.consumer-group-offset-commit-interval
  [consumer_group_offset_commit_interval: <duration> | default = 1s]

  # (experimental) When true, the file-based offset stored in the TSDB directory
  # is enforced at startup, taking precedence over Kafka consumer group offset.
  # When false, offsets are still written to the file (in the TSDB directory)
  # but the Kafka consumer group offset is used at startup.
  # CLI flag: -ingest-storage.kafka.consumer-group-offset-commit-file-enforced
  [consumer_group_offset_commit_file_enforced: <boolean> | default = false]

  # How frequently to poll the last produced offset, used to enforce strong read
  # consistency.
  # CLI flag: -ingest-storage.kafka.last-produced-offset-poll-interval
  [last_produced_offset_poll_interval: <duration> | default = 1s]

  # How long to retry a failed request to get the last produced offset.
  # CLI flag: -ingest-storage.kafka.last-produced-offset-retry-timeout
  [last_produced_offset_retry_timeout: <duration> | default = 10s]

  # From which position to start consuming the partition at startup. Supported
  # options: last-offset, start, end, timestamp.
  # CLI flag: -ingest-storage.kafka.consume-from-position-at-startup
  [consume_from_position_at_startup: <string> | default = "last-offset"]

  # Milliseconds timestamp after which the consumption of the partition starts
  # at startup. Only applies when consume-from-position-at-startup is timestamp
  # CLI flag: -ingest-storage.kafka.consume-from-timestamp-at-startup
  [consume_from_timestamp_at_startup: <int> | default = 0]

  # The best-effort maximum lag a consumer tries to achieve at startup. Set both
  # -ingest-storage.kafka.target-consumer-lag-at-startup and
  # -ingest-storage.kafka.max-consumer-lag-at-startup to 0 to disable waiting
  # for maximum consumer lag being honored at startup.
  # CLI flag: -ingest-storage.kafka.target-consumer-lag-at-startup
  [target_consumer_lag_at_startup: <duration> | default = 2s]

  # The guaranteed maximum lag before a consumer is considered to have caught up
  # reading from a partition at startup, becomes ACTIVE in the hash ring and
  # passes the readiness check. Set both
  # -ingest-storage.kafka.target-consumer-lag-at-startup and
  # -ingest-storage.kafka.max-consumer-lag-at-startup to 0 to disable waiting
  # for maximum consumer lag being honored at startup.
  # CLI flag: -ingest-storage.kafka.max-consumer-lag-at-startup
  [max_consumer_lag_at_startup: <duration> | default = 15s]

  # Enable auto-creation of Kafka topic on startup if it doesn't exist. If
  # creating the topic fails and the topic doesn't already exist, Mimir fails to
  # start.
  # CLI flag: -ingest-storage.kafka.auto-create-topic-enabled
  [auto_create_topic_enabled: <boolean> | default = true]

  # When auto-creation of Kafka topic is enabled and this value is positive,
  # Mimir creates the topic with this number of partitions. When the value is -1
  # the Kafka broker uses the default number of partitions (num.partitions
  # configuration).
  # CLI flag: -ingest-storage.kafka.auto-create-topic-default-partitions
  [auto_create_topic_default_partitions: <int> | default = -1]

  # The maximum size of a Kafka record data that should be generated by the
  # producer. An incoming write request larger than this size is split into
  # multiple Kafka records. We strongly recommend to not change this setting
  # unless for testing purposes.
  # CLI flag: -ingest-storage.kafka.producer-max-record-size-bytes
  [producer_max_record_size_bytes: <int> | default = 15983616]

  # The maximum size of (uncompressed) buffered and unacknowledged produced
  # records sent to Kafka. The produce request fails once this limit is reached.
  # This limit is per Kafka client. 0 to disable the limit.
  # CLI flag: -ingest-storage.kafka.producer-max-buffered-bytes
  [producer_max_buffered_bytes: <int> | default = 1073741824]

  # The maximum allowed for a read requests processed by an ingester to wait
  # until strong read consistency is enforced. 0 to disable the timeout.
  # CLI flag: -ingest-storage.kafka.wait-strong-read-consistency-timeout
  [wait_strong_read_consistency_timeout: <duration> | default = 20s]

  # The record version that this producer sends.
  # CLI flag: -ingest-storage.kafka.producer-record-version
  [producer_record_version: <int> | default = 2]

  # The maximum amount of time a Kafka broker waits for some records before a
  # Fetch response is returned.
  # CLI flag: -ingest-storage.kafka.fetch-max-wait
  [fetch_max_wait: <duration> | default = 5s]

  # The maximum number of concurrent fetch requests that the ingester makes when
  # reading data from Kafka during startup. Concurrent fetch requests are issued
  # only when there is sufficient backlog of records to consume. Set to 0 to
  # disable.
  # CLI flag: -ingest-storage.kafka.fetch-concurrency-max
  [fetch_concurrency_max: <int> | default = 12]

  # When enabled, the fetch request MaxBytes field is computed using the
  # compressed size of previous records. When disabled, MaxBytes is computed
  # using uncompressed bytes. Different Kafka implementations interpret MaxBytes
  # differently.
  # CLI flag: -ingest-storage.kafka.use-compressed-bytes-as-fetch-max-bytes
  [use_compressed_bytes_as_fetch_max_bytes: <boolean> | default = true]

  # The maximum number of buffered records ready to be processed. This limit
  # applies to the sum of all inflight requests. Set to 0 to disable the limit.
  # CLI flag: -ingest-storage.kafka.max-buffered-bytes
  [max_buffered_bytes: <int> | default = 1000000000]

  # The maximum number of concurrent ingestion streams to the TSDB head. Every
  # tenant has their own set of streams. 0 to disable.
  # CLI flag: -ingest-storage.kafka.ingestion-concurrency-max
  [ingestion_concurrency_max: <int> | default = 8]

  # The number of timeseries to batch together before ingesting to the TSDB
  # head. Only use this setting when
  # -ingest-storage.kafka.ingestion-concurrency-max is greater than 0.
  # CLI flag: -ingest-storage.kafka.ingestion-concurrency-batch-size
  [ingestion_concurrency_batch_size: <int> | default = 150]

  # The number of batches to prepare and queue to ingest to the TSDB head. Only
  # use this setting when -ingest-storage.kafka.ingestion-concurrency-max is
  # greater than 0.
  # CLI flag: -ingest-storage.kafka.ingestion-concurrency-queue-capacity
  [ingestion_concurrency_queue_capacity: <int> | default = 3]

  # The expected number of times to ingest timeseries to the TSDB head after
  # batching. With fewer flushes, the overhead of splitting up the work is
  # higher than the benefit of parallelization. Only use this setting when
  # -ingest-storage.kafka.ingestion-concurrency-max is greater than 0.
  # CLI flag: -ingest-storage.kafka.ingestion-concurrency-target-flushes-per-shard
  [ingestion_concurrency_target_flushes_per_shard: <int> | default = 40]

  # The estimated number of bytes a sample has at time of ingestion. This value
  # is used to estimate the timeseries without decompressing them. Only use this
  # setting when -ingest-storage.kafka.ingestion-concurrency-max is greater than
  # 0.
  # CLI flag: -ingest-storage.kafka.ingestion-concurrency-estimated-bytes-per-sample
  [ingestion_concurrency_estimated_bytes_per_sample: <int> | default = 200]

migration:
  # When both this option and ingest storage are enabled, distributors write to
  # both Kafka and ingesters. A write request is considered successful only when
  # written to both backends.
  # CLI flag: -ingest-storage.migration.distributor-send-to-ingesters-enabled
  [distributor_send_to_ingesters_enabled: <boolean> | default = false]

  # When enabled, errors writing to ingest storage are logged but do not affect
  # write success or quorum. When disabled, write requests fail if ingest
  # storage write fails.
  # CLI flag: -ingest-storage.migration.ignore-ingest-storage-errors
  [ignore_ingest_storage_errors: <boolean> | default = false]

  # The maximum time a write request that goes through the ingest storage waits
  # before it times out. Set to `0` to disable the timeout.
  # CLI flag: -ingest-storage.migration.ingest-storage-max-wait-time
  [ingest_storage_max_wait_time: <duration> | default = 0s]

# (advanced) Number of tenants to concurrently fsync WAL and WBL before Kafka
# offsets are committed, must be at least 1.
# CLI flag: -ingest-storage.write-logs-fsync-before-kafka-commit-concurrency
[write_logs_fsync_before_kafka_commit_concurrency: <int> | default = 4]

blocks_storage

The blocks_storage block configures the blocks storage.

# Backend storage to use. Supported backends are: s3, gcs, azure, swift,
# filesystem.
# CLI flag: -blocks-storage.backend
[backend: <string> | default = "filesystem"]

# The s3_backend block configures the connection to Amazon S3 object storage
# backend.
# The CLI flags prefix for this block configuration is: blocks-storage
[s3: <s3_storage_backend>]

# The gcs_backend block configures the connection to Google Cloud Storage object
# storage backend.
# The CLI flags prefix for this block configuration is: blocks-storage
[gcs: <gcs_storage_backend>]

# The azure_storage_backend block configures the connection to Azure object
# storage backend.
# The CLI flags prefix for this block configuration is: blocks-storage
[azure: <azure_storage_backend>]

# The swift_storage_backend block configures the connection to OpenStack Object
# Storage (Swift) object storage backend.
# The CLI flags prefix for this block configuration is: blocks-storage
[swift: <swift_storage_backend>]

# The filesystem_storage_backend block configures the usage of local file system
# as object storage backend.
# The CLI flags prefix for this block configuration is: blocks-storage
[filesystem: <filesystem_storage_backend>]

# Prefix for all objects stored in the backend storage. For simplicity, it may
# only contain digits and English alphabet letters.
# CLI flag: -blocks-storage.storage-prefix
[storage_prefix: <string> | default = ""]

# This configures how the querier and store-gateway discover and synchronize
# blocks stored in the bucket.
bucket_store:
  # Directory to store synchronized TSDB index headers. This directory is not
  # required to be persisted between restarts, but it's highly recommended in
  # order to improve the store-gateway startup time.
  # CLI flag: -blocks-storage.bucket-store.sync-dir
  [sync_dir: <string> | default = "./tsdb-sync/"]

  # (advanced) How frequently to scan the bucket, or to refresh the bucket index
  # (if enabled), in order to look for changes (new blocks shipped by ingesters
  # and blocks deleted by retention or compaction).
  # CLI flag: -blocks-storage.bucket-store.sync-interval
  [sync_interval: <duration> | default = 15m]

  # (advanced) Max number of concurrent queries to execute against the long-term
  # storage. The limit is shared across all tenants.
  # CLI flag: -blocks-storage.bucket-store.max-concurrent
  [max_concurrent: <int> | default = 200]

  # (advanced) Timeout for the queue of queries waiting for execution. If the
  # queue is full and the timeout is reached, the query will be retried on
  # another store-gateway. 0 means no timeout and all queries will wait
  # indefinitely for their turn.
  # CLI flag: -blocks-storage.bucket-store.max-concurrent-queue-timeout
  [max_concurrent_queue_timeout: <duration> | default = 5s]

  # (advanced) Maximum number of concurrent tenants synching blocks.
  # CLI flag: -blocks-storage.bucket-store.tenant-sync-concurrency
  [tenant_sync_concurrency: <int> | default = 1]

  # (advanced) Maximum number of concurrent blocks synching per tenant.
  # CLI flag: -blocks-storage.bucket-store.block-sync-concurrency
  [block_sync_concurrency: <int> | default = 4]

  # (advanced) Number of goroutines to use when syncing block meta files from
  # object storage per tenant.
  # CLI flag: -blocks-storage.bucket-store.meta-sync-concurrency
  [meta_sync_concurrency: <int> | default = 20]

  index_cache:
    # The index cache backend type. Supported values: inmemory, memcached.
    # CLI flag: -blocks-storage.bucket-store.index-cache.backend
    [backend: <string> | default = "inmemory"]

    # The memcached block configures the Memcached-based caching backend.
    # The CLI flags prefix for this block configuration is:
    # blocks-storage.bucket-store.index-cache
    [memcached: <memcached>]

    inmemory:
      # Maximum size in bytes of in-memory index cache used to speed up blocks
      # index lookups (shared between all tenants).
      # CLI flag: -blocks-storage.bucket-store.index-cache.inmemory.max-size-bytes
      [max_size_bytes: <int> | default = 1073741824]

  chunks_cache:
    # Backend for chunks cache, if not empty. Supported values: memcached.
    # CLI flag: -blocks-storage.bucket-store.chunks-cache.backend
    [backend: <string> | default = ""]

    # The memcached block configures the Memcached-based caching backend.
    # The CLI flags prefix for this block configuration is:
    # blocks-storage.bucket-store.chunks-cache
    [memcached: <memcached>]

    # (advanced) Maximum number of sub-GetRange requests that a single GetRange
    # request can be split into when fetching chunks. Zero or negative value =
    # unlimited number of sub-requests.
    # CLI flag: -blocks-storage.bucket-store.chunks-cache.max-get-range-requests
    [max_get_range_requests: <int> | default = 3]

    # (advanced) TTL for caching object attributes for chunks. If the metadata
    # cache is configured, attributes will be stored under this cache backend,
    # otherwise attributes are stored in the chunks cache backend.
    # CLI flag: -blocks-storage.bucket-store.chunks-cache.attributes-ttl
    [attributes_ttl: <duration> | default = 168h]

    # (advanced) Maximum number of object attribute items to keep in a first
    # level in-memory LRU cache. Metadata will be stored and fetched in-memory
    # before hitting the cache backend. 0 to disable the in-memory cache.
    # CLI flag: -blocks-storage.bucket-store.chunks-cache.attributes-in-memory-max-items
    [attributes_in_memory_max_items: <int> | default = 50000]

    # (advanced) TTL for caching individual chunks subranges.
    # CLI flag: -blocks-storage.bucket-store.chunks-cache.subrange-ttl
    [subrange_ttl: <duration> | default = 24h]

  metadata_cache:
    # Backend for metadata cache, if not empty. Supported values: memcached.
    # CLI flag: -blocks-storage.bucket-store.metadata-cache.backend
    [backend: <string> | default = ""]

    # The memcached block configures the Memcached-based caching backend.
    # The CLI flags prefix for this block configuration is:
    # blocks-storage.bucket-store.metadata-cache
    [memcached: <memcached>]

    # (advanced) How long to cache list of tenants in the bucket.
    # CLI flag: -blocks-storage.bucket-store.metadata-cache.tenants-list-ttl
    [tenants_list_ttl: <duration> | default = 15m]

    # (advanced) How long to cache list of blocks for each tenant.
    # CLI flag: -blocks-storage.bucket-store.metadata-cache.tenant-blocks-list-ttl
    [tenant_blocks_list_ttl: <duration> | default = 5m]

    # (advanced) How long to cache list of chunks for a block.
    # CLI flag: -blocks-storage.bucket-store.metadata-cache.chunks-list-ttl
    [chunks_list_ttl: <duration> | default = 24h]

    # (advanced) How long to cache information that block metafile exists. Also
    # used for tenant deletion mark file.
    # CLI flag: -blocks-storage.bucket-store.metadata-cache.metafile-exists-ttl
    [metafile_exists_ttl: <duration> | default = 2h]

    # (advanced) How long to cache information that block metafile doesn't
    # exist. Also used for tenant deletion mark file.
    # CLI flag: -blocks-storage.bucket-store.metadata-cache.metafile-doesnt-exist-ttl
    [metafile_doesnt_exist_ttl: <duration> | default = 5m]

    # (advanced) How long to cache content of the metafile.
    # CLI flag: -blocks-storage.bucket-store.metadata-cache.metafile-content-ttl
    [metafile_content_ttl: <duration> | default = 24h]

    # (advanced) Maximum size of metafile content to cache in bytes. Caching
    # will be skipped if the content exceeds this size. This is useful to avoid
    # network round trip for large content if the configured caching backend has
    # an hard limit on cached items size (in this case, you should set this
    # limit to the same limit in the caching backend).
    # CLI flag: -blocks-storage.bucket-store.metadata-cache.metafile-max-size-bytes
    [metafile_max_size_bytes: <int> | default = 1048576]

    # (advanced) How long to cache attributes of the block metafile.
    # CLI flag: -blocks-storage.bucket-store.metadata-cache.metafile-attributes-ttl
    [metafile_attributes_ttl: <duration> | default = 168h]

    # (advanced) How long to cache attributes of the block index.
    # CLI flag: -blocks-storage.bucket-store.metadata-cache.block-index-attributes-ttl
    [block_index_attributes_ttl: <duration> | default = 168h]

    # (advanced) How long to cache content of the bucket index.
    # CLI flag: -blocks-storage.bucket-store.metadata-cache.bucket-index-content-ttl
    [bucket_index_content_ttl: <duration> | default = 5m]

    # (advanced) Maximum size of bucket index content to cache in bytes. Caching
    # will be skipped if the content exceeds this size. This is useful to avoid
    # network round trip for large content if the configured caching backend has
    # an hard limit on cached items size (in this case, you should set this
    # limit to the same limit in the caching backend).
    # CLI flag: -blocks-storage.bucket-store.metadata-cache.bucket-index-max-size-bytes
    [bucket_index_max_size_bytes: <int> | default = 1048576]

  # (advanced) Duration after which the blocks marked for deletion will be
  # filtered out while fetching blocks. The idea of ignore-deletion-marks-delay
  # is to ignore blocks that are marked for deletion with some delay. This
  # ensures store can still serve blocks that are meant to be deleted but do not
  # have a replacement yet.
  # CLI flag: -blocks-storage.bucket-store.ignore-deletion-marks-delay
  [ignore_deletion_mark_delay: <duration> | default = 1h]

  # (experimental) Duration after which blocks marked for deletion will still be
  # queried. This ensures queriers still query blocks that are meant to be
  # deleted but do not have a replacement yet.
  # CLI flag: -blocks-storage.bucket-store.ignore-deletion-marks-while-querying-delay
  [ignore_deletion_mark_while_querying_delay: <duration> | default = 50m]

  bucket_index:
    # (advanced) How frequently a bucket index, which previously failed to load,
    # should be tried to load again. This option is used only by querier.
    # CLI flag: -blocks-storage.bucket-store.bucket-index.update-on-error-interval
    [update_on_error_interval: <duration> | default = 1m]

    # (advanced) How long a unused bucket index should be cached. Once this
    # timeout expires, the unused bucket index is removed from the in-memory
    # cache. This option is used only by querier.
    # CLI flag: -blocks-storage.bucket-store.bucket-index.idle-timeout
    [idle_timeout: <duration> | default = 1h]

    # (advanced) The maximum allowed age of a bucket index (last updated) before
    # queries start failing because the bucket index is too old. The bucket
    # index is periodically updated by the compactor, and this check is enforced
    # in the querier (at query time).
    # CLI flag: -blocks-storage.bucket-store.bucket-index.max-stale-period
    [max_stale_period: <duration> | default = 1h]

  # (advanced) Blocks with minimum time within this duration are ignored, and
  # not loaded by store-gateway. Useful when used together with
  # -querier.query-store-after to prevent loading young blocks, because there
  # are usually many of them (depending on number of ingesters) and they are not
  # yet compacted. Negative values or 0 disable the filter.
  # CLI flag: -blocks-storage.bucket-store.ignore-blocks-within
  [ignore_blocks_within: <duration> | default = 10h]

  # (advanced) Max size - in bytes - of the in-memory series hash cache. The
  # cache is shared across all tenants and it's used only when query sharding is
  # enabled.
  # CLI flag: -blocks-storage.bucket-store.series-hash-cache-max-size-bytes
  [series_hash_cache_max_size_bytes: <int> | default = 1073741824]

  # (advanced) Max size - in bytes - of a gap for which the partitioner
  # aggregates together two bucket GET object requests.
  # CLI flag: -blocks-storage.bucket-store.partitioner-max-gap-bytes
  [partitioner_max_gap_bytes: <int> | default = 524288]

  # (experimental) Max size - in bytes - of a gap for which the partitioner
  # aggregates together two bucket GET object requests. Overrides the
  # 'bucket-store.partitioner-max-gap-bytes' when requesting chunks
  # CLI flag: -blocks-storage.bucket-store.partitioner-max-gap-bytes-chunks
  [partitioner_max_gap_bytes_chunks: <int> | default = 0]

  # (advanced) Controls what is the ratio of postings offsets that the store
  # will hold in memory.
  # CLI flag: -blocks-storage.bucket-store.posting-offsets-in-mem-sampling
  [postings_offsets_in_mem_sampling: <int> | default = 32]

  index_header:
    # (advanced) Maximum number of idle file handles the store-gateway keeps
    # open for each index-header file.
    # CLI flag: -blocks-storage.bucket-store.index-header.max-idle-file-handles
    [max_idle_file_handles: <int> | default = 1]

    # (advanced) If enabled, store-gateway will lazy load an index-header only
    # once required by a query.
    # CLI flag: -blocks-storage.bucket-store.index-header.lazy-loading-enabled
    [lazy_loading_enabled: <boolean> | default = true]

    # (advanced) If index-header lazy loading is enabled and this setting is >
    # 0, the store-gateway will offload unused index-headers after 'idle
    # timeout' inactivity.
    # CLI flag: -blocks-storage.bucket-store.index-header.lazy-loading-idle-timeout
    [lazy_loading_idle_timeout: <duration> | default = 1h]

    # (advanced) Maximum number of concurrent index header loads across all
    # tenants. If set to 0, concurrency is unlimited.
    # CLI flag: -blocks-storage.bucket-store.index-header.lazy-loading-concurrency
    [lazy_loading_concurrency: <int> | default = 4]

    # (advanced) Timeout for the queue of index header loads. If the queue is
    # full and the timeout is reached, the load will return an error. 0 means no
    # timeout and the load will wait indefinitely.
    # CLI flag: -blocks-storage.bucket-store.index-header.lazy-loading-concurrency-queue-timeout
    [lazy_loading_concurrency_queue_timeout: <duration> | default = 5s]

    # (advanced) If true, verify the checksum of index headers upon loading them
    # (either on startup or lazily when lazy loading is enabled). Setting to
    # true helps detect disk corruption at the cost of slowing down index header
    # loading.
    # CLI flag: -blocks-storage.bucket-store.index-header.verify-on-load
    [verify_on_load: <boolean> | default = false]

    bucket_reader:
      # (experimental) Enable reading TSDB index-header sections from object
      # storage. When enabled, the configured
      # -blocks-storage.bucket-store.index-header.bucket-reader.index-sections
      # are not downloaded to local disk.
      # CLI flag: -blocks-storage.bucket-store.index-header.bucket-reader.enabled
      [enabled: <boolean> | default = false]

      # (experimental) Index sections to read from object storage instead of
      # local disk. Valid sections: postings-offsets-table
      # CLI flag: -blocks-storage.bucket-store.index-header.bucket-reader.index-sections
      [index_sections: <string> | default = "postings-offsets-table"]

  # (advanced) This option controls how many series to fetch per batch. The
  # batch size must be greater than 0.
  # CLI flag: -blocks-storage.bucket-store.batch-series-size
  [streaming_series_batch_size: <int> | default = 5000]

  # (advanced) This parameter controls the trade-off in fetching series versus
  # fetching postings to fulfill a series request. Increasing the series
  # preference results in fetching more series and reducing the volume of
  # postings fetched. Reducing the series preference results in the opposite.
  # Increase this parameter to reduce the rate of fetched series bytes (see
  # "Mimir / Queries" dashboard) or API calls to the object store. Must be a
  # positive floating point number.
  # CLI flag: -blocks-storage.bucket-store.series-fetch-preference
  [series_fetch_preference: <float> | default = 0.75]

tsdb:
  # Directory to store TSDBs (including WAL) in the ingesters. This directory is
  # required to be persisted between restarts.
  # CLI flag: -blocks-storage.tsdb.dir
  [dir: <string> | default = "./tsdb/"]

  # TSDB blocks retention in the ingester before a block is removed. If shipping
  # is enabled, the retention will be relative to the time when the block was
  # uploaded to storage. If shipping is disabled then its relative to the
  # creation time of the block. This should be larger than the
  # -blocks-storage.tsdb.block-ranges-period, -querier.query-store-after and
  # large enough to give store-gateways and queriers enough time to discover
  # newly uploaded blocks.
  # CLI flag: -blocks-storage.tsdb.retention-period
  [retention_period: <duration> | default = 13h]

  # (advanced) How frequently the TSDB blocks are scanned and new ones are
  # shipped to the storage. 0 means shipping is disabled.
  # CLI flag: -blocks-storage.tsdb.ship-interval
  [ship_interval: <duration> | default = 1m]

  # (advanced) Maximum number of tenants concurrently shipping blocks to the
  # storage.
  # CLI flag: -blocks-storage.tsdb.ship-concurrency
  [ship_concurrency: <int> | default = 10]

  # (advanced) How frequently the ingester checks whether the TSDB head should
  # be compacted and, if so, triggers the compaction. Mimir applies a jitter to
  # the first check, and subsequent checks will happen at the configured
  # interval. A block is only created if data covers the smallest block range.
  # The configured interval must be between 0 and 15 minutes.
  # CLI flag: -blocks-storage.tsdb.head-compaction-interval
  [head_compaction_interval: <duration> | default = 1m]

  # (advanced) Maximum number of tenants concurrently compacting TSDB head into
  # a new block
  # CLI flag: -blocks-storage.tsdb.head-compaction-concurrency
  [head_compaction_concurrency: <int> | default = 1]

  # (advanced) If TSDB head is idle for this duration, it is compacted. Note
  # that up to 25% jitter is added to the value to avoid ingesters compacting
  # concurrently. 0 means disabled.
  # CLI flag: -blocks-storage.tsdb.head-compaction-idle-timeout
  [head_compaction_idle_timeout: <duration> | default = 1h]

  # (advanced) The write buffer size used by the head chunks mapper. Lower
  # values reduce memory utilisation on clusters with a large number of tenants
  # at the cost of increased disk I/O operations. The configured buffer size
  # must be between 65536 and 8388608.
  # CLI flag: -blocks-storage.tsdb.head-chunks-write-buffer-size-bytes
  [head_chunks_write_buffer_size_bytes: <int> | default = 4194304]

  # (experimental) How much variance (as percentage between 0 and 1) should be
  # applied to the chunk end time, to spread chunks writing across time. Doesn't
  # apply to the last chunk of the chunk range. 0 means no variance.
  # CLI flag: -blocks-storage.tsdb.head-chunks-end-time-variance
  [head_chunks_end_time_variance: <float> | default = 0]

  # (advanced) The number of shards of series to use in TSDB (must be a power of
  # 2). Reducing this will decrease memory footprint, but can negatively impact
  # performance.
  # CLI flag: -blocks-storage.tsdb.stripe-size
  [stripe_size: <int> | default = 16384]

  # (advanced) True to enable TSDB WAL compression.
  # CLI flag: -blocks-storage.tsdb.wal-compression-enabled
  [wal_compression_enabled: <boolean> | default = false]

  # (advanced) TSDB WAL segments files max size (bytes).
  # CLI flag: -blocks-storage.tsdb.wal-segment-size-bytes
  [wal_segment_size_bytes: <int> | default = 134217728]

  # (advanced) Maximum number of CPUs that can simultaneously processes WAL
  # replay. If it is set to 0, then each TSDB is replayed with a concurrency
  # equal to the number of CPU cores available on the machine.
  # CLI flag: -blocks-storage.tsdb.wal-replay-concurrency
  [wal_replay_concurrency: <int> | default = 0]

  # (advanced) True to flush blocks to storage on shutdown. If false, incomplete
  # blocks will be reused after restart.
  # CLI flag: -blocks-storage.tsdb.flush-blocks-on-shutdown
  [flush_blocks_on_shutdown: <boolean> | default = false]

  # (advanced) If TSDB has not received any data for this duration, and all
  # blocks from TSDB have been shipped, TSDB is closed and deleted from local
  # disk. If set to positive value, this value should be equal or higher than
  # -querier.query-ingesters-within flag to make sure that TSDB is not closed
  # prematurely, which could cause partial query results. 0 or negative value
  # disables closing of idle TSDB.
  # CLI flag: -blocks-storage.tsdb.close-idle-tsdb-timeout
  [close_idle_tsdb_timeout: <duration> | default = 13h]

  # (advanced) True to allow closing of idle TSDBs even when block shipping is
  # disabled. When enabled, idle TSDBs with an empty head will be closed and
  # deleted from local disk regardless of shipping status. This helps prevent
  # accumulation of idle users with 0 series.
  # CLI flag: -blocks-storage.tsdb.close-idle-tsdb-when-shipping-disabled
  [close_idle_tsdb_when_shipping_disabled: <boolean> | default = false]

  # (experimental) True to enable snapshotting of in-memory TSDB data on disk
  # when shutting down.
  # CLI flag: -blocks-storage.tsdb.memory-snapshot-on-shutdown
  [memory_snapshot_on_shutdown: <boolean> | default = false]

  # (advanced) The size of the write queue used by the head chunks mapper. Lower
  # values reduce memory utilisation at the cost of potentially higher ingest
  # latency. Value of 0 switches chunks mapper to implementation without a
  # queue.
  # CLI flag: -blocks-storage.tsdb.head-chunks-write-queue-size
  [head_chunks_write_queue_size: <int> | default = 1000000]

  # (experimental) When enabled, ingester produces 24h blocks for out-of-order
  # data that is before the current day, instead of the usual 2h blocks.
  # CLI flag: -blocks-storage.tsdb.bigger-out-of-order-blocks-for-old-samples
  [bigger_out_of_order_blocks_for_old_samples: <boolean> | default = false]

  # (advanced) Max size - in bytes - of the in-memory series hash cache. The
  # cache is shared across all tenants and it's used only when query sharding is
  # enabled.
  # CLI flag: -blocks-storage.tsdb.series-hash-cache-max-size-bytes
  [series_hash_cache_max_size_bytes: <int> | default = 367001600]

  # (experimental) Maximum capacity for out of order chunks, in samples between
  # 1 and 255.
  # CLI flag: -blocks-storage.tsdb.out-of-order-capacity-max
  [out_of_order_capacity_max: <int> | default = 32]

  # (experimental) Whether postings for matchers cache should be shared across
  # blocks, as opposed to instantiated per block. With a shared cache, one cache
  # is created for head blocks, and one for compacted blocks.
  # CLI flag: -blocks-storage.tsdb.shared-postings-for-matchers-cache
  [shared_postings_for_matchers_cache: <boolean> | default = false]

  # (experimental) Whether head block postings should be tracked and invalidated
  # when they change, allowing higher TTLs to be used. When not using
  # invalidation, cache entries will be used until removed.
  # CLI flag: -blocks-storage.tsdb.head-postings-for-matchers-cache-invalidation
  [head_postings_for_matchers_cache_invalidation: <boolean> | default = false]

  # (experimental) The size of the metric versions cache in each ingester when
  # invalidation is enabled.
  # CLI flag: -blocks-storage.tsdb.head-postings-for-matchers-cache-versions
  [head_postings_for_matchers_cache_versions: <int> | default = 2097152]

  # (advanced) How long to cache postings for matchers in the Head and OOOHead.
  # Set to 0 to disable the cache and only deduplicate in-flight calls.
  # CLI flag: -blocks-storage.tsdb.head-postings-for-matchers-cache-ttl
  [head_postings_for_matchers_cache_ttl: <duration> | default = 10s]

  # (deprecated) Maximum number of entries in the cache for postings for
  # matchers in the Head and OOOHead when TTL is greater than 0.
  # CLI flag: -blocks-storage.tsdb.head-postings-for-matchers-cache-size
  [head_postings_for_matchers_cache_size: <int> | default = 100]

  # (advanced) Maximum size, in bytes, of the cache for postings for matchers in
  # each compacted block when the TTL is greater than 0.
  # CLI flag: -blocks-storage.tsdb.head-postings-for-matchers-cache-max-bytes
  [head_postings_for_matchers_cache_max_bytes: <int> | default = 104857600]

  # (advanced) Force the cache to be used for postings for matchers in the Head
  # and OOOHead, even if it's not a concurrent (query-sharding) call.
  # CLI flag: -blocks-storage.tsdb.head-postings-for-matchers-cache-force
  [head_postings_for_matchers_cache_force: <boolean> | default = false]

  # (advanced) How long to cache postings for matchers in each compacted block
  # queried from the ingester. 0 disables the cache and just deduplicates the
  # in-flight calls.
  # CLI flag: -blocks-storage.tsdb.block-postings-for-matchers-cache-ttl
  [block_postings_for_matchers_cache_ttl: <duration> | default = 10s]

  # (deprecated) Maximum number of entries in the cache for postings for
  # matchers in each compacted block when TTL is greater than 0.
  # CLI flag: -blocks-storage.tsdb.block-postings-for-matchers-cache-size
  [block_postings_for_matchers_cache_size: <int> | default = 100]

  # (advanced) Maximum size in bytes of the cache for postings for matchers in
  # each compacted block when TTL is greater than 0.
  # CLI flag: -blocks-storage.tsdb.block-postings-for-matchers-cache-max-bytes
  [block_postings_for_matchers_cache_max_bytes: <int> | default = 104857600]

  # (advanced) Force the cache to be used for postings for matchers in compacted
  # blocks, even if it's not a concurrent (query-sharding) call.
  # CLI flag: -blocks-storage.tsdb.block-postings-for-matchers-cache-force
  [block_postings_for_matchers_cache_force: <boolean> | default = false]

  # (experimental) When the number of in-memory series in the ingester is equal
  # to or greater than this setting, the ingester tries to compact the TSDB
  # Head. The early compaction removes from the memory all samples and inactive
  # series up until -ingester.active-series-metrics-idle-timeout time ago. After
  # an early compaction, the ingester will not accept any sample with a
  # timestamp older than -ingester.active-series-metrics-idle-timeout time ago
  # (unless out of order ingestion is enabled). The ingester checks every
  # -blocks-storage.tsdb.head-compaction-interval whether an early compaction is
  # required. Use 0 to disable it.
  # CLI flag: -blocks-storage.tsdb.early-head-compaction-min-in-memory-series
  [early_head_compaction_min_in_memory_series: <int> | default = 0]

  # (experimental) When the early compaction is enabled, the early compaction is
  # triggered only if the estimated series reduction is at least the configured
  # percentage (0-100).
  # CLI flag: -blocks-storage.tsdb.early-head-compaction-min-estimated-series-reduction-percentage
  [early_head_compaction_min_estimated_series_reduction_percentage: <int> | default = 15]

  # (experimental) Allows head compaction to happen when the min block range can
  # no longer be appended, without requiring 1.5x the chunk range worth of data
  # in the head.
  # CLI flag: -blocks-storage.tsdb.timely-head-compaction-enabled
  [timely_head_compaction_enabled: <boolean> | default = false]

  index_lookup_planning:
    # (advanced) Cost for iterating postings that have been retrieved from the
    # index.
    # CLI flag: -blocks-storage.tsdb.index-lookup-planning.retrieved-posting-cost
    [retrieved_posting_cost: <float> | default = 0.01]

    # (advanced) Cost for retrieving series from the index and checking if a
    # series belongs to the query's shard.
    # CLI flag: -blocks-storage.tsdb.index-lookup-planning.retrieved-series-cost
    [retrieved_series_cost: <float> | default = 15]

    # (advanced) Cost for retrieving the posting list from disk or from memory.
    # CLI flag: -blocks-storage.tsdb.index-lookup-planning.retrieved-posting-list-cost
    [retrieved_posting_list_cost: <float> | default = 10]

    # (advanced) Minimum number of series a block must have for query planning
    # to be used.
    # CLI flag: -blocks-storage.tsdb.index-lookup-planning.min-series-per-block-for-query-planning
    [min_series_per_block_for_query_planning: <int> | default = 10000]

    # (advanced) Number of series for a label name above which larger count-min
    # sketches are used for that label.
    # CLI flag: -blocks-storage.tsdb.index-lookup-planning.label-cardinality-for-larger-sketch
    [label_cardinality_for_larger_sketch: <int> | default = 1000000]

    # (advanced) Number of series for a label name below which smaller count-min
    # sketches are used for that label.
    # CLI flag: -blocks-storage.tsdb.index-lookup-planning.label-cardinality-for-smaller-sketch
    [label_cardinality_for_smaller_sketch: <int> | default = 1000]

    # (experimental) Controls the collection of statistics and whether to defer
    # some vector selector matchers to sequential scans. This leads to better
    # performance.
    # CLI flag: -blocks-storage.tsdb.index-lookup-planning.enabled
    [index_lookup_planning_enabled: <boolean> | default = false]

    # (experimental) Portion of queries where a mirrored chunk querier compares
    # results with and without index lookup planning. Value between 0 (disabled)
    # and 1 (all queries).
    # CLI flag: -blocks-storage.tsdb.index-lookup-planning.comparison-portion
    [index_lookup_planning_comparison_portion: <float> | default = 0]

    # (experimental) How frequently to collect block statistics, which are used
    # in query execution optimization. 0 to disable.
    # CLI flag: -blocks-storage.tsdb.index-lookup-planning.statistics-collection-frequency
    [statistics_collection_frequency: <duration> | default = 1h]

  offset_catalogue:
    # (experimental) Controls the maintaining of kafka offset catalogue per
    # block.
    # CLI flag: -blocks-storage.tsdb.offset-catalogue.enabled
    [enabled: <boolean> | default = false]

    # (experimental) Maximum number of tenants concurrently syncing offset
    # catalogue to disk.
    # CLI flag: -blocks-storage.tsdb.offset-catalogue.sync-concurrency
    [sync_concurrency: <int> | default = 10]

compactor

The compactor block configures the compactor component.

# (advanced) List of compaction time ranges.
# CLI flag: -compactor.block-ranges
[block_ranges: <list of durations> | default = 2h0m0s,12h0m0s,24h0m0s]

# (advanced) Number of goroutines to use when downloading blocks for compaction
# and uploading resulting blocks.
# CLI flag: -compactor.block-sync-concurrency
[block_sync_concurrency: <int> | default = 8]

# (advanced) Number of goroutines to use when syncing block meta files from the
# long term storage.
# CLI flag: -compactor.meta-sync-concurrency
[meta_sync_concurrency: <int> | default = 20]

# Directory to temporarily store blocks during compaction. This directory is not
# required to be persisted between restarts.
# CLI flag: -compactor.data-dir
[data_dir: <string> | default = "./data-compactor/"]

# (advanced) The frequency at which the compaction runs
# CLI flag: -compactor.compaction-interval
[compaction_interval: <duration> | default = 1h]

# (advanced) How many times to retry a failed compaction within a single
# compaction run.
# CLI flag: -compactor.compaction-retries
[compaction_retries: <int> | default = 3]

# (advanced) Max number of concurrent compactions running.
# CLI flag: -compactor.compaction-concurrency
[compaction_concurrency: <int> | default = 1]

# How long the compactor waits before compacting first-level blocks that are
# uploaded by the ingesters or block-builders. This configuration option allows
# for the reduction of cases where the compactor begins to compact blocks before
# all ingesters have uploaded their blocks to the storage. Does not apply to
# out-of-order blocks.
# CLI flag: -compactor.first-level-compaction-wait-period
[first_level_compaction_wait_period: <duration> | default = 25m]

# (experimental) How long the compactor waits before compacting first-level
# blocks containing out-of-order samples. When set to 0 (default), out-of-order
# blocks do not delay compaction.
# CLI flag: -compactor.first-level-compaction-ooo-wait-period
[first_level_compaction_ooo_wait_period: <duration> | default = 0s]

# (experimental) When enabled, the compactor skips first-level compaction jobs
# if any source block has a MaxTime more recent than the wait period threshold.
# This prevents premature compaction of blocks that may still receive
# late-arriving data.
# CLI flag: -compactor.first-level-compaction-skip-future-max-time
[first_level_compaction_skip_future_max_time: <boolean> | default = false]

# (advanced) How frequently the compactor should run blocks cleanup and
# maintenance, as well as update the bucket index.
# CLI flag: -compactor.cleanup-interval
[cleanup_interval: <duration> | default = 15m]

# (advanced) Max number of tenants for which blocks cleanup and maintenance
# should run concurrently.
# CLI flag: -compactor.cleanup-concurrency
[cleanup_concurrency: <int> | default = 20]

# (advanced) Time before a block marked for deletion is deleted from bucket. If
# not 0, blocks will be marked for deletion and the compactor component will
# permanently delete blocks marked for deletion from the bucket. If 0, blocks
# will be deleted straight away. Note that deleting blocks immediately can cause
# query failures.
# CLI flag: -compactor.deletion-delay
[deletion_delay: <duration> | default = 12h]

# (advanced) For tenants marked for deletion, this is the time between deletion
# of the last block, and doing final cleanup (marker files, debug files) of the
# tenant.
# CLI flag: -compactor.tenant-cleanup-delay
[tenant_cleanup_delay: <duration> | default = 6h]

# (advanced) Max time for starting compactions for a single tenant. After this
# time no new compactions for the tenant are started before next compaction
# cycle. This can help in multi-tenant environments to avoid single tenant using
# all compaction time, but also in single-tenant environments to force new
# discovery of blocks more often. 0 = disabled.
# CLI flag: -compactor.max-compaction-time
[max_compaction_time: <duration> | default = 1h]

# (advanced) Number of goroutines opening blocks before compaction.
# CLI flag: -compactor.max-opening-blocks-concurrency
[max_opening_blocks_concurrency: <int> | default = 1]

# (advanced) Max number of blocks that can be closed concurrently during split
# compaction. Note that closing a newly compacted block uses a lot of memory for
# writing the index.
# CLI flag: -compactor.max-closing-blocks-concurrency
[max_closing_blocks_concurrency: <int> | default = 1]

# (advanced) Number of symbols flushers used when doing split compaction.
# CLI flag: -compactor.symbols-flushers-concurrency
[symbols_flushers_concurrency: <int> | default = 1]

# (advanced) Max number of uploaded blocks that can be validated concurrently. 0
# = no limit.
# CLI flag: -compactor.max-block-upload-validation-concurrency
[max_block_upload_validation_concurrency: <int> | default = 1]

# (advanced) Number of goroutines to use when updating blocks metadata during
# bucket index updates.
# CLI flag: -compactor.update-blocks-concurrency
[update_blocks_concurrency: <int> | default = 1]

# (advanced) Comma separated list of tenants that can be compacted. If
# specified, only these tenants will be compacted by the compactor, otherwise
# all tenants can be compacted. Subject to sharding.
# CLI flag: -compactor.enabled-tenants
[enabled_tenants: <string> | default = ""]

# (advanced) Comma separated list of tenants that cannot be compacted by the
# compactor. If specified, and the compactor would normally pick a given tenant
# for compaction (via -compactor.enabled-tenants or sharding), it will be
# ignored instead.
# CLI flag: -compactor.disabled-tenants
[disabled_tenants: <string> | default = ""]

sharding_ring:
  # The key-value store used to share the hash ring across multiple instances.
  kvstore:
    # Backend storage to use for the ring. Supported values are: consul, etcd,
    # inmemory, memberlist, multi.
    # CLI flag: -compactor.ring.store
    [store: <string> | default = "memberlist"]

    # (advanced) The prefix for the keys in the store. Should end with a /.
    # CLI flag: -compactor.ring.prefix
    [prefix: <string> | default = "collectors/"]

    # The consul block configures the consul client.
    # The CLI flags prefix for this block configuration is: compactor.ring
    [consul: <consul>]

    # The etcd block configures the etcd client.
    # The CLI flags prefix for this block configuration is: compactor.ring
    [etcd: <etcd>]

    multi:
      # (advanced) Primary backend storage used by multi-client.
      # CLI flag: -compactor.ring.multi.primary
      [primary: <string> | default = ""]

      # (advanced) Secondary backend storage used by multi-client.
      # CLI flag: -compactor.ring.multi.secondary
      [secondary: <string> | default = ""]

      # (advanced) Mirror writes to the secondary store.
      # CLI flag: -compactor.ring.multi.mirror-enabled
      [mirror_enabled: <boolean> | default = false]

      # (advanced) Timeout for storing a value to the secondary store.
      # CLI flag: -compactor.ring.multi.mirror-timeout
      [mirror_timeout: <duration> | default = 2s]

  # (advanced) Period at which to heartbeat to the ring.
  # CLI flag: -compactor.ring.heartbeat-period
  [heartbeat_period: <duration> | default = 15s]

  # (advanced) Heartbeat timeout after which Mimir marks compactors as unhealthy
  # in the ring.
  # CLI flag: -compactor.ring.heartbeat-timeout
  [heartbeat_timeout: <duration> | default = 1m]

  # (advanced) Instance ID to register in the ring.
  # CLI flag: -compactor.ring.instance-id
  [instance_id: <string> | default = "<hostname>"]

  # List of network interface names to look up when finding the instance IP
  # address.
  # CLI flag: -compactor.ring.instance-interface-names
  [instance_interface_names: <list of strings> | default = [<private network interfaces>]]

  # (advanced) Port to advertise in the ring (defaults to
  # -server.grpc-listen-port).
  # CLI flag: -compactor.ring.instance-port
  [instance_port: <int> | default = 0]

  # (advanced) IP address to advertise in the ring. Default is auto-detected.
  # CLI flag: -compactor.ring.instance-addr
  [instance_addr: <string> | default = ""]

  # (advanced) Enable using an IPv6 instance address.
  # CLI flag: -compactor.ring.instance-enable-ipv6
  [instance_enable_ipv6: <boolean> | default = false]

  # (advanced) Minimum time to wait for ring stability at startup. 0 to disable.
  # CLI flag: -compactor.ring.wait-stability-min-duration
  [wait_stability_min_duration: <duration> | default = 0s]

  # (advanced) Maximum time to wait for ring stability at startup. If the
  # compactor ring keeps changing after this period of time, the compactor will
  # start anyway.
  # CLI flag: -compactor.ring.wait-stability-max-duration
  [wait_stability_max_duration: <duration> | default = 5m]

  # (advanced) Timeout for waiting on compactor to become ACTIVE in the ring.
  # CLI flag: -compactor.ring.wait-active-instance-timeout
  [wait_active_instance_timeout: <duration> | default = 10m]

  # (advanced) Number of consecutive timeout periods after which Mimir
  # automatically removes an unhealthy instance in the ring. Set to 0 to disable
  # auto-forget.
  # CLI flag: -compactor.ring.auto-forget-unhealthy-periods
  [auto_forget_unhealthy_periods: <int> | default = 10]

# (advanced) The sorting to use when deciding which compaction jobs should run
# first for a given tenant. Supported values are:
# smallest-range-oldest-blocks-first, newest-blocks-first.
# CLI flag: -compactor.compaction-jobs-order
[compaction_jobs_order: <string> | default = "smallest-range-oldest-blocks-first"]

store_gateway

The store_gateway block configures the store-gateway component.

# The hash ring configuration.
sharding_ring:
  # The key-value store used to share the hash ring across multiple instances.
  # This option needs be set both on the store-gateway, querier and ruler when
  # running in microservices mode.
  kvstore:
    # Backend storage to use for the ring. Supported values are: consul, etcd,
    # inmemory, memberlist, multi.
    # CLI flag: -store-gateway.sharding-ring.store
    [store: <string> | default = "memberlist"]

    # (advanced) The prefix for the keys in the store. Should end with a /.
    # CLI flag: -store-gateway.sharding-ring.prefix
    [prefix: <string> | default = "collectors/"]

    # The consul block configures the consul client.
    # The CLI flags prefix for this block configuration is:
    # store-gateway.sharding-ring
    [consul: <consul>]

    # The etcd block configures the etcd client.
    # The CLI flags prefix for this block configuration is:
    # store-gateway.sharding-ring
    [etcd: <etcd>]

    multi:
      # (advanced) Primary backend storage used by multi-client.
      # CLI flag: -store-gateway.sharding-ring.multi.primary
      [primary: <string> | default = ""]

      # (advanced) Secondary backend storage used by multi-client.
      # CLI flag: -store-gateway.sharding-ring.multi.secondary
      [secondary: <string> | default = ""]

      # (advanced) Mirror writes to the secondary store.
      # CLI flag: -store-gateway.sharding-ring.multi.mirror-enabled
      [mirror_enabled: <boolean> | default = false]

      # (advanced) Timeout for storing a value to the secondary store.
      # CLI flag: -store-gateway.sharding-ring.multi.mirror-timeout
      [mirror_timeout: <duration> | default = 2s]

  # (advanced) Period at which to heartbeat to the ring.
  # CLI flag: -store-gateway.sharding-ring.heartbeat-period
  [heartbeat_period: <duration> | default = 15s]

  # (advanced) The heartbeat timeout after which store gateways are considered
  # unhealthy within the ring. This option needs be set both on the
  # store-gateway, querier and ruler when running in microservices mode.
  # CLI flag: -store-gateway.sharding-ring.heartbeat-timeout
  [heartbeat_timeout: <duration> | default = 1m]

  # (deprecated) When enabled, a store-gateway is automatically removed from the
  # ring after failing to heartbeat the ring for a period longer than the
  # configured -store-gateway.sharding-ring.auto-forget-unhealthy-periods times
  # the configured -store-gateway.sharding-ring.heartbeat-timeout. This setting
  # is deprecated. Set
  # -store-gateway.sharding-ring.auto-forget-unhealthy-periods to 0 to disable
  # auto-forget.
  # CLI flag: -store-gateway.sharding-ring.auto-forget-enabled
  [auto_forget_enabled: <boolean> | default = true]

  # (advanced) Number of consecutive timeout periods after which Mimir
  # automatically removes an unhealthy instance in the ring. Set to 0 to disable
  # auto-forget.
  # CLI flag: -store-gateway.sharding-ring.auto-forget-unhealthy-periods
  [auto_forget_unhealthy_periods: <int> | default = 10]

  # (advanced) The replication factor to use when sharding blocks. This option
  # needs be set both on the store-gateway, querier and ruler when running in
  # microservices mode.
  # CLI flag: -store-gateway.sharding-ring.replication-factor
  [replication_factor: <int> | default = 3]

  # File path where tokens are stored. If empty, tokens are not stored at
  # shutdown and restored at startup.
  # CLI flag: -store-gateway.sharding-ring.tokens-file-path
  [tokens_file_path: <string> | default = ""]

  # (advanced) Number of tokens for each store-gateway.
  # CLI flag: -store-gateway.sharding-ring.num-tokens
  [num_tokens: <int> | default = 512]

  # True to enable zone-awareness and replicate blocks across different
  # availability zones. This option needs be set both on the store-gateway,
  # querier and ruler when running in microservices mode.
  # CLI flag: -store-gateway.sharding-ring.zone-awareness-enabled
  [zone_awareness_enabled: <boolean> | default = false]

  # (advanced) Comma-separated list of zones to exclude from the ring. Instances
  # in excluded zones will be filtered out from the ring. This option needs be
  # set both on the store-gateway, querier and ruler when running in
  # microservices mode.
  # CLI flag: -store-gateway.sharding-ring.excluded-zones
  [excluded_zones: <string> | default = ""]

  # (advanced) Minimum time to wait for ring stability at startup, if set to
  # positive value.
  # CLI flag: -store-gateway.sharding-ring.wait-stability-min-duration
  [wait_stability_min_duration: <duration> | default = 0s]

  # (advanced) Maximum time to wait for ring stability at startup. If the
  # store-gateway ring keeps changing after this period of time, the
  # store-gateway will start anyway.
  # CLI flag: -store-gateway.sharding-ring.wait-stability-max-duration
  [wait_stability_max_duration: <duration> | default = 5m]

  # (advanced) Instance ID to register in the ring.
  # CLI flag: -store-gateway.sharding-ring.instance-id
  [instance_id: <string> | default = "<hostname>"]

  # List of network interface names to look up when finding the instance IP
  # address.
  # CLI flag: -store-gateway.sharding-ring.instance-interface-names
  [instance_interface_names: <list of strings> | default = [<private network interfaces>]]

  # (advanced) Port to advertise in the ring (defaults to
  # -server.grpc-listen-port).
  # CLI flag: -store-gateway.sharding-ring.instance-port
  [instance_port: <int> | default = 0]

  # (advanced) IP address to advertise in the ring. Default is auto-detected.
  # CLI flag: -store-gateway.sharding-ring.instance-addr
  [instance_addr: <string> | default = ""]

  # (advanced) Enable using a IPv6 instance address. (default false)
  # CLI flag: -store-gateway.sharding-ring.instance-enable-ipv6
  [instance_enable_ipv6: <boolean> | default = false]

  # The availability zone where this instance is running. Required if
  # zone-awareness is enabled.
  # CLI flag: -store-gateway.sharding-ring.instance-availability-zone
  [instance_availability_zone: <string> | default = ""]

  # Unregister from the ring upon clean shutdown.
  # CLI flag: -store-gateway.sharding-ring.unregister-on-shutdown
  [unregister_on_shutdown: <boolean> | default = true]

# Experimental dynamic replication configuration.
dynamic_replication:
  # (experimental) Use a higher number of replicas for recent blocks. Useful to
  # spread query load more evenly at the cost of slightly higher disk usage.
  # CLI flag: -store-gateway.dynamic-replication.enabled
  [enabled: <boolean> | default = false]

  # (experimental) Threshold of the most recent sample in a block used to
  # determine it is eligible for higher than default replication. If a block has
  # samples within this amount of time, it is considered recent and will be
  # owned by more replicas.
  # CLI flag: -store-gateway.dynamic-replication.max-time-threshold
  [max_time_threshold: <duration> | default = 25h]

  # (experimental) Multiple of the default replication factor that should be
  # used for recent blocks. Minimum value is 2
  # CLI flag: -store-gateway.dynamic-replication.multiple
  [multiple: <int> | default = 5]

# (advanced) Comma separated list of tenants that can be loaded by the
# store-gateway. If specified, only blocks for these tenants will be loaded by
# the store-gateway, otherwise all tenants can be loaded. Subject to sharding.
# CLI flag: -store-gateway.enabled-tenants
[enabled_tenants: <string> | default = ""]

# (advanced) Comma separated list of tenants that cannot be loaded by the
# store-gateway. If specified, and the store-gateway would normally load a given
# tenant for (via -store-gateway.enabled-tenants or sharding), it will be
# ignored instead.
# CLI flag: -store-gateway.disabled-tenants
[disabled_tenants: <string> | default = ""]

memcached

The memcached block configures the Memcached-based caching backend. The supported CLI flags <prefix> used to reference this configuration block are:

blocks-storage.bucket-store.chunks-cache
blocks-storage.bucket-store.index-cache
blocks-storage.bucket-store.metadata-cache
querier.mimir-query-engine.range-vector-splitting
query-frontend.results-cache
ruler-storage.cache

# Comma-separated list of memcached addresses. Each address can be an IP
# address, hostname, or an entry specified in the DNS Service Discovery format.
# CLI flag: -<prefix>.memcached.addresses
[addresses: <string> | default = ""]

# The socket read/write timeout.
# CLI flag: -<prefix>.memcached.timeout
[timeout: <duration> | default = 200ms]

# The connection timeout.
# CLI flag: -<prefix>.memcached.connect-timeout
[connect_timeout: <duration> | default = 200ms]

# (advanced) The minimum number of idle connections to keep open as a percentage
# (0-100) of the number of recently used idle connections. If negative, idle
# connections are kept open indefinitely.
# CLI flag: -<prefix>.memcached.min-idle-connections-headroom-percentage
[min_idle_connections_headroom_percentage: <float> | default = -1]

# (advanced) The maximum number of idle connections that will be maintained per
# address.
# CLI flag: -<prefix>.memcached.max-idle-connections
[max_idle_connections: <int> | default = 100]

# (advanced) The maximum number of concurrent asynchronous operations can occur.
# CLI flag: -<prefix>.memcached.max-async-concurrency
[max_async_concurrency: <int> | default = 50]

# (advanced) The maximum number of enqueued asynchronous operations allowed.
# CLI flag: -<prefix>.memcached.max-async-buffer-size
[max_async_buffer_size: <int> | default = 25000]

# (advanced) The maximum number of concurrent connections running get
# operations. If set to 0, concurrency is unlimited.
# CLI flag: -<prefix>.memcached.max-get-multi-concurrency
[max_get_multi_concurrency: <int> | default = 100]

# (advanced) The maximum number of keys a single underlying get operation should
# run. If more keys are specified, internally keys are split into multiple
# batches and fetched concurrently, honoring the max concurrency. If set to 0,
# the max batch size is unlimited.
# CLI flag: -<prefix>.memcached.max-get-multi-batch-size
[max_get_multi_batch_size: <int> | default = 100]

# (advanced) The maximum size of an item stored in memcached, in bytes. Bigger
# items are not stored. If set to 0, no maximum size is enforced.
# CLI flag: -<prefix>.memcached.max-item-size
[max_item_size: <int> | default = 1048576]

# (advanced) Enable connecting to Memcached with TLS.
# CLI flag: -<prefix>.memcached.tls-enabled
[tls_enabled: <boolean> | default = false]

# (advanced) Path to the client certificate, which will be used for
# authenticating with the server. Also requires the key path to be configured.
# CLI flag: -<prefix>.memcached.tls-cert-path
[tls_cert_path: <string> | default = ""]

# (advanced) Path to the key for the client certificate. Also requires the
# client certificate to be configured.
# CLI flag: -<prefix>.memcached.tls-key-path
[tls_key_path: <string> | default = ""]

# (advanced) Path to the CA certificates to validate server certificate against.
# If not set, the host's root CA certificates are used.
# CLI flag: -<prefix>.memcached.tls-ca-path
[tls_ca_path: <string> | default = ""]

# (advanced) Override the expected name on the server certificate.
# CLI flag: -<prefix>.memcached.tls-server-name
[tls_server_name: <string> | default = ""]

# (advanced) Skip validating server certificate.
# CLI flag: -<prefix>.memcached.tls-insecure-skip-verify
[tls_insecure_skip_verify: <boolean> | default = false]

# (advanced) Override the default cipher suite list (separated by commas).
# Allowed values:
#
# Secure Ciphers:
# - TLS_AES_128_GCM_SHA256
# - TLS_AES_256_GCM_SHA384
# - TLS_CHACHA20_POLY1305_SHA256
# - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
# - TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
# - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
# - TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
# - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
# - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
# - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
# - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
# - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
# - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
#
# Insecure Ciphers:
# - TLS_RSA_WITH_RC4_128_SHA
# - TLS_RSA_WITH_3DES_EDE_CBC_SHA
# - TLS_RSA_WITH_AES_128_CBC_SHA
# - TLS_RSA_WITH_AES_256_CBC_SHA
# - TLS_RSA_WITH_AES_128_CBC_SHA256
# - TLS_RSA_WITH_AES_128_GCM_SHA256
# - TLS_RSA_WITH_AES_256_GCM_SHA384
# - TLS_ECDHE_ECDSA_WITH_RC4_128_SHA
# - TLS_ECDHE_RSA_WITH_RC4_128_SHA
# - TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
# - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
# - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
# CLI flag: -<prefix>.memcached.tls-cipher-suites
[tls_cipher_suites: <string> | default = ""]

# (advanced) Override the default minimum TLS version. Allowed values:
# VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13
# CLI flag: -<prefix>.memcached.tls-min-version
[tls_min_version: <string> | default = ""]

s3_storage_backend

The s3_backend block configures the connection to Amazon S3 object storage backend. The supported CLI flags <prefix> used to reference this configuration block are:

alertmanager-storage
blocks-storage
common.storage
ruler-storage

# The S3 bucket endpoint. It could be an AWS S3 endpoint listed at
# https://docs.aws.amazon.com/general/latest/gr/s3.html or the address of an
# S3-compatible service in hostname:port format.
# CLI flag: -<prefix>.s3.endpoint
[endpoint: <string> | default = ""]

# S3 region. If unset, the client will issue a S3 GetBucketLocation API call to
# autodetect it.
# CLI flag: -<prefix>.s3.region
[region: <string> | default = ""]

# S3 bucket name
# CLI flag: -<prefix>.s3.bucket-name
[bucket_name: <string> | default = ""]

# S3 secret access key
# CLI flag: -<prefix>.s3.secret-access-key
[secret_access_key: <string> | default = ""]

# S3 access key ID
# CLI flag: -<prefix>.s3.access-key-id
[access_key_id: <string> | default = ""]

# S3 session token
# CLI flag: -<prefix>.s3.session-token
[session_token: <string> | default = ""]

# (advanced) If enabled, use http:// for the S3 endpoint instead of https://.
# This could be useful in local dev/test environments while using an
# S3-compatible backend storage, like Minio.
# CLI flag: -<prefix>.s3.insecure
[insecure: <boolean> | default = false]

# (advanced) The signature version to use for authenticating against S3.
# Supported values are: v4, v2.
# CLI flag: -<prefix>.s3.signature-version
[signature_version: <string> | default = "v4"]

# (advanced) Use a specific version of the S3 list object API. Supported values
# are v1 or v2. Default is unset.
# CLI flag: -<prefix>.s3.list-objects-version
[list_objects_version: <string> | default = ""]

# (advanced) Bucket lookup style type, used to access bucket in S3-compatible
# service. Default is auto. Supported values are: auto, path, virtual-hosted.
# CLI flag: -<prefix>.s3.bucket-lookup-type
[bucket_lookup_type: <string> | default = "auto"]

# (experimental) When enabled, direct all AWS S3 requests to the dual-stack
# IPv4/IPv6 endpoint for the configured region.
# CLI flag: -<prefix>.s3.dualstack-enabled
[dualstack_enabled: <boolean> | default = true]

# (experimental) The S3 storage class to use, not set by default. Details can be
# found at https://aws.amazon.com/s3/storage-classes/. Supported values are:
# STANDARD, REDUCED_REDUNDANCY, STANDARD_IA, ONEZONE_IA, INTELLIGENT_TIERING,
# GLACIER, DEEP_ARCHIVE, OUTPOSTS, GLACIER_IR, SNOW, EXPRESS_ONEZONE,
# FSX_OPENZFS, FSX_ONTAP
# CLI flag: -<prefix>.s3.storage-class
[storage_class: <string> | default = ""]

# (experimental) If enabled, it will use the default authentication methods of
# the AWS SDK for go based on known environment variables and known AWS config
# files.
# CLI flag: -<prefix>.s3.native-aws-auth-enabled
[native_aws_auth_enabled: <boolean> | default = false]

# (experimental) The minimum file size in bytes used for multipart uploads. If
# 0, the value is optimally computed for each object.
# CLI flag: -<prefix>.s3.part-size
[part_size: <int> | default = 0]

# (experimental) If enabled, a Content-MD5 header is sent with S3 Put Object
# requests. Consumes more resources to compute the MD5, but may improve
# compatibility with object storage services that do not support checksums.
# CLI flag: -<prefix>.s3.send-content-md5
[send_content_md5: <boolean> | default = false]

# Accessing S3 resources using temporary, secure credentials provided by AWS
# Security Token Service.
# CLI flag: -<prefix>.s3.sts-endpoint
[sts_endpoint: <string> | default = ""]

sse:
  # Enable AWS Server Side Encryption. Supported values: SSE-KMS, SSE-S3.
  # CLI flag: -<prefix>.s3.sse.type
  [type: <string> | default = ""]

  # KMS Key ID used to encrypt objects in S3
  # CLI flag: -<prefix>.s3.sse.kms-key-id
  [kms_key_id: <string> | default = ""]

  # KMS Encryption Context used for object encryption. It expects JSON formatted
  # string.
  # CLI flag: -<prefix>.s3.sse.kms-encryption-context
  [kms_encryption_context: <string> | default = ""]

http:
  # (advanced) The time an idle connection remains idle before closing.
  # CLI flag: -<prefix>.s3.http.idle-conn-timeout
  [idle_conn_timeout: <duration> | default = 1m30s]

  # (advanced) The amount of time the client waits for a server's response
  # headers.
  # CLI flag: -<prefix>.s3.http.response-header-timeout
  [response_header_timeout: <duration> | default = 2m]

  # (advanced) If the client connects to object storage via HTTPS and this
  # option is enabled, the client accepts any certificate and hostname.
  # CLI flag: -<prefix>.s3.http.insecure-skip-verify
  [insecure_skip_verify: <boolean> | default = false]

  # (advanced) Maximum time to wait for a TLS handshake. Set to 0 for no limit.
  # CLI flag: -<prefix>.s3.tls-handshake-timeout
  [tls_handshake_timeout: <duration> | default = 10s]

  # (advanced) The time to wait for a server's first response headers after
  # fully writing the request headers if the request has an Expect header. Set
  # to 0 to send the request body immediately.
  # CLI flag: -<prefix>.s3.expect-continue-timeout
  [expect_continue_timeout: <duration> | default = 1s]

  # (advanced) Maximum number of idle (keep-alive) connections across all hosts.
  # Set to 0 for no limit.
  # CLI flag: -<prefix>.s3.max-idle-connections
  [max_idle_connections: <int> | default = 100]

  # (advanced) Maximum number of idle (keep-alive) connections to keep per-host.
  # Set to 0 to use a built-in default value of 2.
  # CLI flag: -<prefix>.s3.max-idle-connections-per-host
  [max_idle_connections_per_host: <int> | default = 100]

  # (advanced) Maximum number of connections per host. Set to 0 for no limit.
  # CLI flag: -<prefix>.s3.max-connections-per-host
  [max_connections_per_host: <int> | default = 0]

  # (advanced) Path to the Certificate Authority (CA) certificates to validate
  # the server certificate. If not set, the host's root CA certificates are
  # used.
  # CLI flag: -<prefix>.s3.http.tls-ca-path
  [tls_ca_path: <string> | default = ""]

  # (advanced) Path to the client certificate, which is used for authenticating
  # with the server. This setting also requires you to configure the key path.
  # CLI flag: -<prefix>.s3.http.tls-cert-path
  [tls_cert_path: <string> | default = ""]

  # (advanced) Path to the key for the client certificate. This setting also
  # requires you to configure the client certificate.
  # CLI flag: -<prefix>.s3.http.tls-key-path
  [tls_key_path: <string> | default = ""]

  # (advanced) Override the expected name on the server certificate.
  # CLI flag: -<prefix>.s3.http.tls-server-name
  [tls_server_name: <string> | default = ""]

trace:
  # (advanced) When enabled, low-level S3 HTTP operation information is logged
  # at the debug level.
  # CLI flag: -<prefix>.s3.trace.enabled
  [enabled: <boolean> | default = false]

gcs_storage_backend

The gcs_backend block configures the connection to Google Cloud Storage object storage backend. The supported CLI flags <prefix> used to reference this configuration block are:

alertmanager-storage
blocks-storage
common.storage
ruler-storage

# GCS bucket name
# CLI flag: -<prefix>.gcs.bucket-name
[bucket_name: <string> | default = ""]

# JSON either from a Google Developers Console client_credentials.json file, or
# a Google Developers service account key. Needs to be valid JSON, not a
# filesystem path. If empty, fallback to Google default logic:
# 1. A JSON file whose path is specified by the GOOGLE_APPLICATION_CREDENTIALS
# environment variable. For workload identity federation, refer to
# https://cloud.google.com/iam/docs/how-to#using-workload-identity-federation on
# how to generate the JSON configuration file for on-prem/non-Google cloud
# platforms.
# 2. A JSON file in a location known to the gcloud command-line tool:
# $HOME/.config/gcloud/application_default_credentials.json.
# 3. On Google Compute Engine it fetches credentials from the metadata server.
# CLI flag: -<prefix>.gcs.service-account
[service_account: <string> | default = ""]

# (advanced) Enable automatic retries for GCS uploads using the RetryAlways
# policy. Uploads will be retried on transient errors. Note: this does not
# guarantee idempotency.
# CLI flag: -<prefix>.gcs.enable-upload-retries
[enable_upload_retries: <boolean> | default = true]

# (advanced) Maximum number of attempts for GCS operations (0 = unlimited, 1 =
# no retries). Applies to both regular and upload retry modes.
# CLI flag: -<prefix>.gcs.max-retries
[max_retries: <int> | default = 20]

http:
  # (advanced) The time an idle connection remains idle before closing.
  # CLI flag: -<prefix>.gcs.http.idle-conn-timeout
  [idle_conn_timeout: <duration> | default = 1m30s]

  # (advanced) The amount of time the client waits for a server's response
  # headers.
  # CLI flag: -<prefix>.gcs.http.response-header-timeout
  [response_header_timeout: <duration> | default = 2m]

  # (advanced) If the client connects to object storage via HTTPS and this
  # option is enabled, the client accepts any certificate and hostname.
  # CLI flag: -<prefix>.gcs.http.insecure-skip-verify
  [insecure_skip_verify: <boolean> | default = false]

  # (advanced) Maximum time to wait for a TLS handshake. Set to 0 for no limit.
  # CLI flag: -<prefix>.gcs.tls-handshake-timeout
  [tls_handshake_timeout: <duration> | default = 10s]

  # (advanced) The time to wait for a server's first response headers after
  # fully writing the request headers if the request has an Expect header. Set
  # to 0 to send the request body immediately.
  # CLI flag: -<prefix>.gcs.expect-continue-timeout
  [expect_continue_timeout: <duration> | default = 1s]

  # (advanced) Maximum number of idle (keep-alive) connections across all hosts.
  # Set to 0 for no limit.
  # CLI flag: -<prefix>.gcs.max-idle-connections
  [max_idle_connections: <int> | default = 100]

  # (advanced) Maximum number of idle (keep-alive) connections to keep per-host.
  # Set to 0 to use a built-in default value of 2.
  # CLI flag: -<prefix>.gcs.max-idle-connections-per-host
  [max_idle_connections_per_host: <int> | default = 100]

  # (advanced) Maximum number of connections per host. Set to 0 for no limit.
  # CLI flag: -<prefix>.gcs.max-connections-per-host
  [max_connections_per_host: <int> | default = 0]

  # (advanced) Path to the Certificate Authority (CA) certificates to validate
  # the server certificate. If not set, the host's root CA certificates are
  # used.
  # CLI flag: -<prefix>.gcs.http.tls-ca-path
  [tls_ca_path: <string> | default = ""]

  # (advanced) Path to the client certificate, which is used for authenticating
  # with the server. This setting also requires you to configure the key path.
  # CLI flag: -<prefix>.gcs.http.tls-cert-path
  [tls_cert_path: <string> | default = ""]

  # (advanced) Path to the key for the client certificate. This setting also
  # requires you to configure the client certificate.
  # CLI flag: -<prefix>.gcs.http.tls-key-path
  [tls_key_path: <string> | default = ""]

  # (advanced) Override the expected name on the server certificate.
  # CLI flag: -<prefix>.gcs.http.tls-server-name
  [tls_server_name: <string> | default = ""]

azure_storage_backend

The azure_storage_backend block configures the connection to Azure object storage backend. The supported CLI flags <prefix> used to reference this configuration block are:

alertmanager-storage
blocks-storage
common.storage
ruler-storage

# Azure storage account name
# CLI flag: -<prefix>.azure.account-name
[account_name: <string> | default = ""]

# Azure storage account key. If unset, Azure managed identities will be used for
# authentication instead.
# CLI flag: -<prefix>.azure.account-key
[account_key: <string> | default = ""]

# If `connection-string` is set, the value of `endpoint-suffix` will not be
# used. Use this method over `account-key` if you need to authenticate via a SAS
# token. Or if you use the Azurite emulator.
# CLI flag: -<prefix>.azure.connection-string
[connection_string: <string> | default = ""]

# Azure storage container name
# CLI flag: -<prefix>.azure.container-name
[container_name: <string> | default = ""]

# Azure storage endpoint suffix without schema. The account name will be
# prefixed to this value to create the FQDN. If set to empty string, default
# endpoint suffix is used.
# CLI flag: -<prefix>.azure.endpoint-suffix
[endpoint_suffix: <string> | default = ""]

# (advanced) Number of retries for recoverable errors
# CLI flag: -<prefix>.azure.max-retries
[max_retries: <int> | default = 20]

# (advanced) User assigned managed identity. If empty, then System assigned
# identity is used.
# CLI flag: -<prefix>.azure.user-assigned-id
[user_assigned_id: <string> | default = ""]

http:
  # (advanced) The time an idle connection remains idle before closing.
  # CLI flag: -<prefix>.azure.http.idle-conn-timeout
  [idle_conn_timeout: <duration> | default = 1m30s]

  # (advanced) The amount of time the client waits for a server's response
  # headers.
  # CLI flag: -<prefix>.azure.http.response-header-timeout
  [response_header_timeout: <duration> | default = 2m]

  # (advanced) If the client connects to object storage via HTTPS and this
  # option is enabled, the client accepts any certificate and hostname.
  # CLI flag: -<prefix>.azure.http.insecure-skip-verify
  [insecure_skip_verify: <boolean> | default = false]

  # (advanced) Maximum time to wait for a TLS handshake. Set to 0 for no limit.
  # CLI flag: -<prefix>.azure.tls-handshake-timeout
  [tls_handshake_timeout: <duration> | default = 10s]

  # (advanced) The time to wait for a server's first response headers after
  # fully writing the request headers if the request has an Expect header. Set
  # to 0 to send the request body immediately.
  # CLI flag: -<prefix>.azure.expect-continue-timeout
  [expect_continue_timeout: <duration> | default = 1s]

  # (advanced) Maximum number of idle (keep-alive) connections across all hosts.
  # Set to 0 for no limit.
  # CLI flag: -<prefix>.azure.max-idle-connections
  [max_idle_connections: <int> | default = 100]

  # (advanced) Maximum number of idle (keep-alive) connections to keep per-host.
  # Set to 0 to use a built-in default value of 2.
  # CLI flag: -<prefix>.azure.max-idle-connections-per-host
  [max_idle_connections_per_host: <int> | default = 100]

  # (advanced) Maximum number of connections per host. Set to 0 for no limit.
  # CLI flag: -<prefix>.azure.max-connections-per-host
  [max_connections_per_host: <int> | default = 0]

  # (advanced) Path to the Certificate Authority (CA) certificates to validate
  # the server certificate. If not set, the host's root CA certificates are
  # used.
  # CLI flag: -<prefix>.azure.http.tls-ca-path
  [tls_ca_path: <string> | default = ""]

  # (advanced) Path to the client certificate, which is used for authenticating
  # with the server. This setting also requires you to configure the key path.
  # CLI flag: -<prefix>.azure.http.tls-cert-path
  [tls_cert_path: <string> | default = ""]

  # (advanced) Path to the key for the client certificate. This setting also
  # requires you to configure the client certificate.
  # CLI flag: -<prefix>.azure.http.tls-key-path
  [tls_key_path: <string> | default = ""]

  # (advanced) Override the expected name on the server certificate.
  # CLI flag: -<prefix>.azure.http.tls-server-name
  [tls_server_name: <string> | default = ""]

swift_storage_backend

The swift_storage_backend block configures the connection to OpenStack Object Storage (Swift) object storage backend. The supported CLI flags <prefix> used to reference this configuration block are:

alertmanager-storage
blocks-storage
common.storage
ruler-storage

# OpenStack Swift application credential id
# CLI flag: -<prefix>.swift.application-credential-id
[application_credential_id: <string> | default = ""]

# OpenStack Swift application credential name
# CLI flag: -<prefix>.swift.application-credential-name
[application_credential_name: <string> | default = ""]

# OpenStack Swift application credential secret
# CLI flag: -<prefix>.swift.application-credential-secret
[application_credential_secret: <string> | default = ""]

# OpenStack Swift authentication API version. 0 to autodetect.
# CLI flag: -<prefix>.swift.auth-version
[auth_version: <int> | default = 0]

# OpenStack Swift authentication URL
# CLI flag: -<prefix>.swift.auth-url
[auth_url: <string> | default = ""]

# OpenStack Swift username.
# CLI flag: -<prefix>.swift.username
[username: <string> | default = ""]

# OpenStack Swift user's domain name.
# CLI flag: -<prefix>.swift.user-domain-name
[user_domain_name: <string> | default = ""]

# OpenStack Swift user's domain ID.
# CLI flag: -<prefix>.swift.user-domain-id
[user_domain_id: <string> | default = ""]

# OpenStack Swift user ID.
# CLI flag: -<prefix>.swift.user-id
[user_id: <string> | default = ""]

# OpenStack Swift API key.
# CLI flag: -<prefix>.swift.password
[password: <string> | default = ""]

# OpenStack Swift user's domain ID.
# CLI flag: -<prefix>.swift.domain-id
[domain_id: <string> | default = ""]

# OpenStack Swift user's domain name.
# CLI flag: -<prefix>.swift.domain-name
[domain_name: <string> | default = ""]

# OpenStack Swift project ID (v2,v3 auth only).
# CLI flag: -<prefix>.swift.project-id
[project_id: <string> | default = ""]

# OpenStack Swift project name (v2,v3 auth only).
# CLI flag: -<prefix>.swift.project-name
[project_name: <string> | default = ""]

# ID of the OpenStack Swift project's domain (v3 auth only), only needed if it
# differs the from user domain.
# CLI flag: -<prefix>.swift.project-domain-id
[project_domain_id: <string> | default = ""]

# Name of the OpenStack Swift project's domain (v3 auth only), only needed if it
# differs from the user domain.
# CLI flag: -<prefix>.swift.project-domain-name
[project_domain_name: <string> | default = ""]

# OpenStack Swift Region to use (v2,v3 auth only).
# CLI flag: -<prefix>.swift.region-name
[region_name: <string> | default = ""]

# Name of the OpenStack Swift container to put chunks in.
# CLI flag: -<prefix>.swift.container-name
[container_name: <string> | default = ""]

# (advanced) Max retries on requests error.
# CLI flag: -<prefix>.swift.max-retries
[max_retries: <int> | default = 3]

# (advanced) Time after which a connection attempt is aborted.
# CLI flag: -<prefix>.swift.connect-timeout
[connect_timeout: <duration> | default = 10s]

# (advanced) Time after which an idle request is aborted. The timeout watchdog
# is reset each time some data is received, so the timeout triggers after X time
# no data is received on a request.
# CLI flag: -<prefix>.swift.request-timeout
[request_timeout: <duration> | default = 5s]

filesystem_storage_backend

The filesystem_storage_backend block configures the usage of local file system as object storage backend. The supported CLI flags <prefix> used to reference this configuration block are:

alertmanager-storage
blocks-storage
common.storage
ruler-storage

# Local filesystem storage directory.
# CLI flag: -<prefix>.filesystem.dir
[dir: <string> | default = ""]

Configure Grafana Mimir shuffle sharding

Wed, 03 Jun 2026 09:01:40 +0200

Configure Grafana Mimir shuffle sharding

Grafana Mimir leverages sharding to horizontally scale both single- and multi-tenant clusters beyond the capacity of a single node.

Background

Grafana Mimir uses a sharding strategy that distributes the workload across a subset of the instances or partitions that run a given component. For example, on the write path, each tenant’s series are sharded across a subset of the partitions. The size of this subset, which is the number of partitions, is configured using the shard size parameter, which by default is 0. This default value means that each tenant uses all available instances to fairly balance resources, such as CPU and memory, and to maximize the use of these resources across the cluster.

In a multi-tenant cluster this default (0) value introduces the following downsides:

An outage affects all tenants.
A misbehaving tenant, for example, a tenant that causes an out-of-memory error, can negatively affect all other tenants.

Configuring a shard size value higher than 0 enables shuffle sharding. The goal of shuffle sharding is to reduce the blast radius of an outage and better isolate tenants.

About shuffle sharding

Shuffle sharding is a technique that isolates different tenant’s workloads and gives each tenant a single-tenant experience, even if they’re running in a shared cluster. For more information about how AWS describes shuffle sharding, refer to What is shuffle sharding?.

Shuffle sharding assigns each tenant a shard that is composed of a subset of the Grafana Mimir instances or a subset of Kafka partitions. This technique minimizes the number of overlapping instances between two tenants. Shuffle sharding provides the following benefits:

An outage on some Grafana Mimir cluster instances, Kafka partitions, or nodes only affects a subset of tenants.
A misbehaving tenant only affects its shard instances or partitions. Assuming that each tenant shard is relatively small compared to the total number of instances or partitions in the cluster, it’s likely that any other tenant runs on different instances or partitions, or that only a subset of instances match the affected instances.

Using shuffle sharding doesn’t require more resources, but can result in unbalanced instances or partitions.

Low overlapping instances or partition probability

For example, in a Grafana Mimir cluster that runs 50 Kafka partitions and assigns each tenant four out of 50 partitions, shuffling partitions between each tenant creates 230,000 possible combinations.

Randomly picking two tenants yields the following probabilities:

71% chance that they don’t share any partitions
26% chance that they share only 1 partition
2.7% chance that they share 2 partitions
0.08% chance that they share 3 partitions
0.0004% chance that their partitions fully overlap

Grafana Mimir shuffle sharding

Grafana Mimir supports shuffle sharding in the following components:

When you run Grafana Mimir with the default configuration, shuffle sharding is disabled and you need to explicitly enable it by increasing the shard size either globally or for a given tenant.

Note
If the shard size value is equal to or higher than the number of available instances or partitions, for example where -ingest-storage.ingestion-partition-tenant-shard-size is higher than the number of active partitions, then shuffle sharding is disabled and all partitions are used.

Guaranteed properties

The Grafana Mimir shuffle sharding implementation provides the following benefits:

Stability
Given a consistent state of the hash ring, the shuffle sharding algorithm always selects the same instances for a given tenant, even across different machines.
Consistency
Adding or removing an instance from the hash ring leads to, at most, only one instance changed in each tenant’s shard.
Shuffling
Probabilistically and for a large enough cluster, shuffle sharding ensures that every tenant receives a different set of instances with a reduced number of overlapping instances between two tenants, which improves failure isolation.
Zone-awareness
When you enable zone-aware replication, the subset of instances selected for each tenant contains a balanced number of instances for each availability zone.

Partitions shuffle sharding in ingest storage architecture

Note
This guidance applies to ingest storage architecture. For more information about the supported architectures in Grafana Mimir, refer to Grafana Mimir architecture.

Without shuffle sharding when using ingest storage, the Grafana Mimir distributor divides the received series among all active partitions. With shuffle sharding, the distributor divides each tenant’s series among multiple partitions.

Configuring shuffle sharding on the write path automatically enables it on the read path too.

You can configure the default shard size for all tenants by setting - -ingest-storage.ingestion-partition-tenant-shard-size=<size> or its YAML equavalent. You can override the shard size on a per-tenant basis by setting ingestion_partitions_tenant_shard_size in the overrides section of the runtime configuration.

Partitions shuffle sharding on the write path

With partition shuffle sharding, the distributor divides each tenant’s series among the number of partitions specified in -ingest-storage.ingestion-partition-tenant-shard-size.

Partitions shuffle sharding read path

When shuffle sharding is enabled for the write path, the read path automatically uses the same configuration to query only the relevant partition owners (ingesters). The querier uses the ShuffleShardWithLookback algorithm to:

Include all partitions in the tenant’s current shard
Include partitions that were recently part of the tenant’s shard (within the lookback period)
Include INACTIVE partitions that are transitioning to a read-only state
Exclude PENDING partitions that have not yet received any traffic

This ensures query consistency during partition lifecycle transitions such as adding or removing ingesters. For more information about the hash ring and the state of partitions, refer to Hash ring.

Rollout strategy

If you’re running a Grafana Mimir cluster with shuffle sharding disabled, and you want to enable it for the ingesters, use the following rollout strategy to avoid missing queries for any series currently in the ingesters:

Explicitly disable shuffle sharding on the ingester read path via -querier.shuffle-sharding-ingesters-enabled=false, since this is enabled by default.
Enable shuffle sharding on the distributor write path.
Wait for at least the amount of time specified via -blocks-storage.tsdb.retention-period.
Re-enable shuffle sharding on the ingester read path via -querier.shuffle-sharding-ingesters-enabled=true.

Limitation: Decreasing the tenant shard size

The shuffle sharding implementation in Grafana Mimir has a limitation that prevents you from abruptly decreasing the tenant shard size when shuffle sharding is enabled for distributors on the read path. This is because when shuffle sharding is disabled, the queriers check all ingesters for blocks, whereas when it’s enabled, they only check the ones that are assigned to that tenant through the shuffle-shard.

Use one of the following methods to safely decrease the tenant shard size:

(Recommended, ingest storage only) Use the write shard size override.
(Classic architecture) Temporarily disable shuffle sharding.

Use the write shard size override

You can use the -ingest-storage.ingestion-partition-tenant-write-shard-size configuration to decrease the tenant shard size without disabling shuffle sharding globally:

Set the write shard size to the desired smaller value using the -ingest-storage.ingestion-partition-tenant-write-shard-size=<new-smaller-size> setting. You can also set this value per-tenant using the ingestion_partitions_tenant_write_shard_size setting in runtime configuration.
Wait for at least the amount of time specified in the -blocks-storage.tsdb.retention-period setting.
Update the main shard size to match the write shard size using the -ingest-storage.ingestion-partition-tenant-shard-size=<new-smaller-size> setting.
Remove the write shard size override by setting -ingest-storage.ingestion-partition-tenant-write-shard-size=0 or removing the runtime configuration override.

This method is only available with ingest storage enabled. If you’re using classic architecture, use the method described in the following section instead.

This method allows queries to continue reading from the previous larger set of partitions while new writes go to the smaller set, without requiring you to disable shuffle sharding for all tenants during the migration.

Temporarily disable shuffle sharding

If you’re using classic architecture, follow these steps:

Disable shuffle sharding on the ingester read path via -querier.shuffle-sharding-ingesters-enabled=false.
Decrease the configured tenant shard size.
Wait for at least the amount of time specified in the -blocks-storage.tsdb.retention-period setting.
Re-enable shuffle sharding on the ingester read path via -querier.shuffle-sharding-ingesters-enabled=true.

Decreasing the tenant shard size without following one of these procedures could lead to inaccurate or incomplete query results. The queriers and rulers can’t determine the previous shard size and could miss an ingester with data for a given tenant. When you change a tenant’s shard size, the tenant’s series are assigned to a new set of partitions with only new samples distributed to it. Samples written before the shard size change remain in the previously-assigned partition set and the ingesters consuming those partitions. Because the queriers can’t determine the previous set of ingesters through the sharding ring, they must check all ingesters for the series until the value set in blocks-storage.tsdb.retention-period has passed. At this point, you can query the series from object storage through the store-gateway.

Note
The procedure for decreasing the tenant shard size is the same as for enabling shuffle sharding. Enabling shuffle sharding for ingesters effectively decreases the tenant shard size.

Ingesters shuffle sharding in classic architecture

Note
This guidance applies to classic architecture. For more information about the supported architectures in Grafana Mimir, refer to Grafana Mimir architecture.

By default, the Grafana Mimir distributor divides the received series among all running ingesters.

When you enable ingester shuffle sharding, the distributor and ruler on the write path divide each tenant series among -distributor.ingestion-tenant-shard-size number of ingesters, while on the read path, the querier and ruler queries only the subset of ingesters that hold the series for a given tenant.

The shard size can be overridden on a per-tenant basis by setting ingestion_tenant_shard_size in the overrides section of the runtime configuration.

Ingesters write path

To enable shuffle sharding for ingesters on the write path, configure the following flags (or their respective YAML configuration options) on the distributor, ingester, and ruler:

-distributor.ingestion-tenant-shard-size=<size>
<size>: Set the size to the number of ingesters each tenant series should be sharded to. If <size> is 0 or is greater than the number of available ingesters in the Grafana Mimir cluster, the tenant series are sharded across all ingesters.

Ingesters read path

Assuming that you have enabled shuffle sharding for the write path, to enable shuffle sharding for ingesters on the read path, configure the following flags (or their respective YAML configuration options) on the querier and ruler:

-distributor.ingestion-tenant-shard-size=<size>

The following flag is set appropriately by default to enable shuffle sharding for ingesters on the read path. If you need to modify its defaults:

-querier.shuffle-sharding-ingesters-enabled=true
Shuffle sharding for ingesters on the read path can be explicitly enabled or disabled.
- If shuffle sharding is enabled, queriers and rulers fetch in-memory series from the minimum set of required ingesters, selecting only ingesters which might have received series since now - -blocks-storage.tsdb.retention-period. Otherwise, the request is sent to all ingesters.

If you enable ingesters shuffle sharding only for the write path, queriers and rulers on the read path always query all ingesters instead of querying the subset of ingesters that belong to the tenant’s shard. Keeping ingesters shuffle sharding enabled only on the write path does not lead to incorrect query results, but might increase query latency.

Rollout strategy

If you’re running a Grafana Mimir cluster with shuffle sharding disabled, and you want to enable it for the ingesters, use the following rollout strategy to avoid missing querying for any series currently in the ingesters:

Explicitly disable shuffle sharding on the ingester read path via -querier.shuffle-sharding-ingesters-enabled=false, since this is enabled by default.
Enable shuffle sharding on the ingester write path.
Wait for at least the amount of time specified via -blocks-storage.tsdb.retention-period.
Re-enable shuffle sharding on the ingester read path via -querier.shuffle-sharding-ingesters-enabled=true.

Limitation: Decreasing the tenant shard size

The shuffle sharding implementation in Grafana Mimir has a limitation that prevents you from abruptly decreasing the tenant shard size when shuffle sharding is enabled for ingesters on the read path. This is because when shuffle sharding is disabled, the queriers check all ingesters for blocks, whereas when it’s enabled, they only check the ones that are assigned to that tenant through the shuffle-shard.

To safely decrease the tenant shard size, follow these steps.

Disable shuffle sharding on the ingester read path via -querier.shuffle-sharding-ingesters-enabled=false.
Decrease the configured tenant shard size.
Wait for at least the amount of time specified via -blocks-storage.tsdb.retention-period.
Re-enable shuffle sharding on the ingester read path via -querier.shuffle-sharding-ingesters-enabled=true.

Decreasing the tenant shard size without following this procedure could lead to inaccurate or incomplete query results. The queriers and rulers can’t determine the previous shard size and could miss an ingester with data for a given tenant. When you change a tenant’s shard size, the tenant’s series are assigned to a new set of ingesters with only new samples distributed to it. Samples written before the shard size change remain in the previously-assigned ingester set. Because the queriers can’t determine the previous set of ingesters through the sharding ring, they must check all ingesters for the series until the value set in blocks-storage.tsdb.retention-period has passed. At this point, you can query the series from object storage through the store-gateway.

Note
The procedure for decreasing the tenant shard size is the same as for enabling shuffle sharding. Enabling shuffle sharding for ingesters effectively decreases the tenant shard size.

Query-frontend and query-scheduler shuffle sharding

By default, all Grafana Mimir queriers can execute queries for any tenant.

When you enable shuffle sharding by setting -query-frontend.max-queriers-per-tenant (or its respective YAML configuration option) to a value higher than 0 and lower than the number of available queriers, only the specified number of queriers are eligible to execute queries for a given tenant.

Note that this distribution happens in query-frontend, or query-scheduler, if used. When using query-scheduler, the -query-frontend.max-queriers-per-tenant option must be set for the query-scheduler component. When you don’t use query-frontend (with or without query-scheduler), this option is not available.

You can override the maximum number of queriers on a per-tenant basis by setting max_queriers_per_tenant in the overrides section of the runtime configuration.

The impact of a “query of death”

In the event a tenant sends a “query of death” which causes a querier to crash, the crashed querier becomes disconnected from the query-frontend or query-scheduler, and another running querier is immediately assigned to the tenant’s shard.

If the tenant repeatedly sends this query, the new querier assigned to the tenant’s shard crashes as well, and yet another querier is assigned to the shard. This cascading failure can potentially result in all running queriers to crash, one by one, which invalidates the assumption that shuffle sharding contains the blast radius of queries of death.

To mitigate this negative impact, there are experimental configuration options that enable you to configure a time delay between when a querier disconnects due to a crash and when the crashed querier is replaced by a healthy querier. When you configure a time delay, a tenant that repeatedly sends a “query of death” runs with reduced querier capacity after a querier has crashed. The tenant could end up having no available queriers, but this configuration reduces the likelihood that the crash impacts other tenants.

A delay of 1 minute might be a reasonable trade-off:

Query-frontend: -query-frontend.querier-forget-delay=1m
Query-scheduler: -query-scheduler.querier-forget-delay=1m

Store-gateway shuffle sharding

By default, a tenant’s blocks are divided among all Grafana Mimir store-gateways.

When you enable store-gateway shuffle sharding by setting -store-gateway.tenant-shard-size (or its respective YAML configuration option) to a value higher than 0 and lower than the number of available store-gateways, only the specified number of store-gateways are eligible to load and query blocks for a given tenant. You must set this flag on the store-gateway, querier, and ruler.

You can override the store-gateway shard size on a per-tenant basis by setting store_gateway_tenant_shard_size in the overrides section of the runtime configuration.

For more information about the store-gateway, refer to store-gateway.

Ruler shuffle sharding

By default, tenant rule groups are sharded by all Grafana Mimir rulers.

When you enable ruler shuffle sharding by setting -ruler.tenant-shard-size (or its respective YAML configuration option) to a value higher than 0 and lower than the number of available rulers, only the specified number of rulers are eligible to evaluate rule groups for a given tenant.

You can override the ruler shard size on a per-tenant basis by setting ruler_tenant_shard_size in the overrides section of the runtime configuration.

Compactor shuffle sharding

By default, tenant blocks can be compacted by any Grafana Mimir compactor.

When you enable compactor shuffle sharding by setting -compactor.compactor-tenant-shard-size (or its respective YAML configuration option) to a value higher than 0 and lower than the number of available compactors, only the specified number of compactors are eligible to compact blocks for a given tenant.

You can override the compactor shard size on a per-tenant basis setting by compactor_tenant_shard_size in the overrides section of the runtime configuration.

Alertmanager shuffle sharding

Alertmanager only performs distribution across replicas per tenant. The state and workload is not divided any further. The replication factor setting -alertmanager.sharding-ring.replication-factor determines how many replicas are used for a tenant.

As a result, shuffle sharding is effectively always enabled for Alertmanager.

Shuffle sharding impact to the KV store

Shuffle sharding does not add additional overhead to the KV store. Shards are computed client-side and are not stored in the ring. KV store sizing depends primarily on the number of replicas of any component that uses the ring, for example, ingesters, and the number of tokens per replica.

However, in some components, each tenant’s shard is cached in-memory on the client-side, which might slightly increase their memory footprint. Increased memory footprint can happen mostly in the distributor.

Migrate ingesters to spread-minimizing tokens

Wed, 03 Jun 2026 09:01:40 +0200

Migrate ingesters to spread-minimizing tokens

You can configure Mimir’s ingesters to use spread-minimizing token generation.

Note
You only need to configure spread-minimizing token generation if you’re running Grafana Mimir with classic architecture. With ingest storage architecture, this process is automatic and doesn’t require configuration. Refer to Grafana Mimir architecture.

Before you begin

The ingester time series replication should be configured with enabled zone-awareness.

Note
Spread-mimizing tokens are recommended if shuffle-sharding is disabled on the write path of your ingesters, or, if it is enabled, but most of the tenants of your system use all available ingesters.

Note
In order to prevent incorrect query results, shuffle-sharding on the read path of your ingesters must be disabled before migrating ingesters to the spread-minimizing tokens. Shuffle-sharding on ingester’s read path can be re-enabled at least -querier.query-store-after time after the last ingester zone was migrated to the spread-minimizing tokens.

If ingesters are configured with a non-empty value of -ingester.ring.tokens-file-path, this is the file where ingesters store the tokens at shutdown and restore them at startup. Keep track of this value, because you need it in the last step.

For simplicity, let’s assume that there are three configured availability zones named zone-a, zone-b, and zone-c. Migration to the spread-minimizing token generation strategy is a complex process performed zone by zone to prevent any data loss.

Step 1: Disable write requests to ingesters from `zone-a`

To disable write requests, configure the following flag on the distributor and the ruler:

-ingester.ring.excluded-zones=zone-a

Before proceeding to the next step, use the following query to ensure that there are no write requests to the ingesters from zone-a:

sum by(route) (
  rate(
    cortex_request_duration_seconds_count{
      namespace="<your-namespace>",
      container="ingester",
      pod=~"ingester-zone-a-.*",
      route="/cortex.Ingester/Push"}[5m]
  )
)

You should see something like this:

Step 2: Shut down ingesters from `zone-a`

Call the /ingester/shutdown endpoint on all ingesters from zone-a to flush all in-memory series and metrics to long-term storage and to clear all ingesters from the ring.

Before proceeding to the next step, ensure that all calls complete successfully.

Step 3: Enable spread-minimizing tokens for ingesters in `zone-a`

Configure the following flags on the ingesters from zone-a:

-ingester.ring.tokens-file-path=
-ingester.ring.token-generation-strategy=spread-minimizing
-ingester.ring.spread-minimizing-zones=zone-a,zone-b,zone-c

Note
The example uses zone-a,zone-b,zone-c to denote a comma-separated list of configured availability zones.

Before proceeding to the next step, ensure that all the ingester pods related to zone-a are up and running with the new configuration.

Optional step: In-order registration of ingesters

Mimir can force the ring to perform an in-order registration of ingesters. When this feature is enabled, an ingester can register its tokens within the ring only after all previous ingesters (with ID lower than its own ID) have already been registered. This feature minimizes a probability that a write request that should be handled by an ingester actually arrives to the ring before the ingester is registered within the ring. In this case, the request gets handled by another ingester. This situation could introduce some deviation from an optimal load distribution.

To configure this capability:

-ingester.ring.spread-minimizing-join-ring-in-order=true

Step 4: Re-enable write requests to ingesters from `zone-a`

To re-enable write requests, revert link Step 1: Disable write requests to ingesters from zone-a.

At this point, you can check the number of in-memory time series of ingesters from zone-a using the following query:

sum by(pod) (
  cortex_ingester_memory_series{
    namespace="<your-namespace>",
    pod=~"ingester-zone-a-.*"}
)

If everything went smoothly, you should see something like this:

Step 5: Migrate ingesters from `zone-b`

Repeat steps 1 to 4, replacing all the occurrences of zone-a with zone-b.

Step 6: Migrate ingesters from `zone-c`

Repeat steps 1 to 4, replacing all the occurrences of zone-a with zone-c.

Step 7: Delete the old token files

If, before the migration, you configured ingesters to store their tokens under -ingester.ring.tokens-file-path, you must delete these files after migrating all ingester zones to spread-minimizing tokens.

For example, if an ingester pod called ingester-zone-a from a namespace called mimir-prod used to store its tokens in a file called /data/tokens, you can run the following command to delete the /data/tokens file:

kubectl -n mimir-prod exec ingester-zone-a-0 -- rm /data/tokens

Configure Grafana Mimir hash rings

Wed, 03 Jun 2026 09:01:40 +0200

Configure Grafana Mimir hash rings

Hash rings are a distributed consistent hashing scheme that Grafana Mimir uses for sharding, replication, and service discovery.

Each of the following Grafana Mimir components builds an independent hash ring. The CLI flags used to configure the hash ring of each component have the following prefixes:

Ingesters: -ingester.ring.*
Ingest storage partitions: -ingester.partition-ring.*
Distributors: -distributor.ring.*
Compactors: -compactor.ring.*
Store-gateways: -store-gateway.sharding-ring.*
(Optional) Query-schedulers: -query-scheduler.ring.*
(Optional) Rulers: -ruler.ring.*
(Optional) Alertmanagers: -alertmanager.sharding-ring.*
(Optional) Overrides-exporters: -overrides-exporter.ring.*

The rest of the documentation refers to these prefixes as <prefix>. You can configure each parameter either via the CLI flag or its respective YAML config option.

Configuring the key-value store

Hash ring data structures need to be shared between Grafana Mimir instances. To propagate changes to a given hash ring, Grafana Mimir uses a key-value store. You can configure the key-value store independently for the hash rings of different components.

Grafana Mimir supports the following key-value (KV) store backends for hash rings:

You can configure the KV store backend setting the <prefix>.store CLI flag (for example, -ingester.ring.store).

Memberlist (default)

By default, Grafana Mimir uses memberlist as the KV store backend.

At startup, a Grafana Mimir instance connects to other Mimir replicas to join the cluster. A Grafana Mimir instance discovers the other replicas to join by resolving the addresses configured in -memberlist.join. The -memberlist.join CLI flag must resolve to other replicas in the cluster and can be specified multiple times.

The -memberlist.join can be set to:

An address in the <ip>:<port> format.
An address in the <hostname>:<port> format.
An address in the DNS service discovery format.

The default port is 7946.

Note
At a minimum, configure one or more addresses that resolve to a consistent subset of replicas (for example, all the ingesters).

Note
If you’re running Grafana Mimir in Kubernetes, define a headless Kubernetes Service which resolves to the IP addresses of all Grafana Mimir Pods.

Then set -memberlist.join to dnssrv+<service name>.<namespace>.svc.cluster.local:<port>.

Note
The memberlist backend is configured globally, unlike other supported backends, and can’t be customized on a per-component basis.

Grafana Mimir supports TLS for memberlist connections between its components. For more information about TLS configuration, refer to secure communications with TLS.

To see all supported configuration parameters, refer to memberlist.

Configuring the memberlist address and port

By default, Grafana Mimir memberlist protocol listens on address 0.0.0.0 and port 7946. If you run multiple Mimir processes on the same node or the port 7946 is not available, you can change the bind and advertise port by setting the following parameters:

-memberlist.bind-addr: IP address to listen on the local machine.
-memberlist.bind-port: Port to listen on the local machine.
-memberlist.advertise-addr: IP address to advertise to other Mimir replicas. The other replicas will connect to this IP to talk to the instance.
-memberlist.advertise-port: Port to advertise to other Mimir replicas. The other replicas will connect to this port to talk to the instance.

Cluster label verification

By default, a Grafana Mimir memberlist joins a cluster with any instance that is discovered when hosts are resolved, based on the -memberlist.join CLI flag setting or the memberlist’s YAML configuration option. If, for any reason, the discovered addresses include instances of other Grafana Mimir clusters, or instances of other distributed systems that use a memberlist, Grafana Mimir joins these unrelated clusters together.

To avoid this, Grafana Mimir provides an additional type of validation known as cluster label verification.

When cluster label verification is enabled, all memberlist internal traffic is prefixed with the configured cluster label. Any traffic that does not match that prefix is discarded, to ensure that only the replicas that have the same configured label can connect to each other.

Migrate to using cluster label verification

Migrate a Grafana Mimir cluster to use cluster label verification:

Disable cluster label verification on all cluster instances via the -memberlist.cluster-label-verification-disabled=true CLI flag (or its respective YAML configuration option).
Wait until the configuration change has been rolled out to all Grafana Mimir instances.
On each cluster, define a label that is unique to that cluster and the same on all instances via the -memberlist.cluster-label CLI flag (or its respective YAML configuration option). This label must be the same on all instances that are part of the same cluster. For example, if you run a Grafana Mimir cluster in a dedicated namespace, then set the cluster label to the name of the namespace.
Wait until the configuration change has been rolled out to all Grafana Mimir instances.
Enable cluster label verification on all clusters instances by removing the configuration option -memberlist.cluster-label-verification-disabled=true.
Wait until the configuration change has been rolled out to all Grafana Mimir instances.

Fine tuning memberlist changes propagation latency

The cortex_ring_oldest_member_timestamp metric can be used to measure the latency of hash ring changes propagation. This metric tracks the oldest heartbeat timestamp across all instances in the ring. You can execute the following query to measure the age of the oldest heartbeat timestamp in the ring:

max(time() - (cortex_ring_oldest_member_timestamp{state="ACTIVE"} > 0))

The measured age shouldn’t be higher than the configured <prefix>.heartbeat-period plus a reasonable delta (for example, 15 seconds). If you experience a higher changes propagation latency, you can adjust the following settings:

Decrease -memberlist.gossip-interval
Increase -memberlist.gossip-nodes
Increase -memberlist.retransmit-factor

Consul

To use Consul as a backend KV store, set the following parameters:

<prefix>.consul.hostname: Consul hostname and port separated by colon. For example, consul:8500.
<prefix>.consul.acl-token: ACL token used to authenticate to Consul. If Consul authentication is disabled, you can leave the token empty.

To see all supported configuration parameters, refer to consul.

Etcd

To use etcd as a backend KV store, set the following parameters:

<prefix>.etcd.endpoints: etcd hostname and port separated by a colon. For example, etcd:2379.
<prefix>.etcd.username: Username used to authenticate to etcd. If etcd authentication is disabled, you can leave the username empty.
<prefix>.etcd.password: Password used to authenticate to etcd. If etcd authentication is disabled, you can leave the password empty.

Grafana Mimir supports TLS between its components and etcd. For more information about TLS configuration, refer to secure communications with TLS.

To see all supported configuration parameters, refer to etcd.

Multi

The multi backend is an implementation that you should use only for migrating between two other backends.

The multi backend uses two different KV stores: the primary and the secondary. The primary backend receives both reads and writes. The secondary backend only receives writes. The primary and secondary backends can be swapped in real-time, which enables you to switch from one backend to another with no downtime.

You can use the following parameters to configure the multi KV store settings:

<prefix>.multi.primary: The type of primary backend store (e.g. consul).
<prefix>.multi.secondary: The type of the secondary backend store (e.g. memberlist).
<prefix>.multi.mirror-enabled: Whether mirroring of writes to the secondary backend store is enabled.
<prefix>.multi.mirror-timeout: The maximum time allowed to mirror a change to the secondary backend store.

Note
Grafana Mimir doesn’t log an error if it’s unable to mirror writes to the secondary backend store. However, the total number of errors is tracked through the metric cortex_multikv_mirror_write_errors_total.

You can also configure the multi KV primary backend and mirroring in the runtime configuration file. Changes to a multi KV Store in the runtime configuration apply to all components using a multi KV store.

The following example shows a runtime configuration file for the multi KV store:

multi_kv_config:
  # The runtime configuration only allows to override the primary backend and whether mirroring is enabled.
  primary: consul
  mirror_enabled: true

Note
The runtime configuration settings take precedence over CLI flags.

Ingester migration example

The following steps show how to migrate ingesters from Consul to etcd:

Configure -ingester.ring.store=multi, -ingester.ring.multi.primary=consul, -ingester.ring.multi.secondary=etcd, and -ingester.ring.multi.mirror-enabled=true. Configure both Consul settings -ingester.ring.consul.* and etcd settings -ingester.ring.etcd.*.
Apply changes to your Grafana Mimir cluster. After changes have rolled out, Grafana Mimir uses Consul as primary KV store, and all writes are mirrored to etcd too.
Configure primary: etcd in the multi_kv_config block of the runtime configuration file. Changes in the runtime configuration file are reloaded live, without the need to restart the process.
Wait until all Mimir instances have reloaded the updated configuration.
Configure mirror_enabled: false in the multi_kv_config block of the runtime configuration file.
Wait until all Mimir instances have reloaded the updated configuration.
Configure -ingester.ring.store=etcd and remove both the multi and Consul configuration because they are no longer required.
Apply changes to your Grafana Mimir cluster. After changes have rolled out, Grafana Mimir only uses etcd.

About Grafana Mimir reactive limiters

Wed, 03 Jun 2026 09:01:40 +0200

About Grafana Mimir reactive limiters

Mimir’s reactive concurrency limiters automatically adapt concurrency limits based on indications of overload. You can use reactive limiters to guard against ingester overload.

How do reactive limiters work?

Reactive limiters detect overload by observing response times, inflight requests, and throughput.

When recent response times increase significantly relative to the longer term trend, an reactive limiter temporarily decreases concurrency limits to avoid potential overload. Similarly, when increasing inflight requests correlate with flat or decreasing throughput and increasing response times, this indicates overload and causes an reactive limiter to decrease concurrency limits.

When an ingester is not overloaded, the concurrency limit increases to a multiple of current inflight requests, based on the max-limit-factor. This provides headroom for bursts of requests without being too high to lower quickly if overload is detected.

When ingester overload is detected, the limit gradually decreases to the concurrent request processing capacity of whichever resource is overloaded, then oscillates around that level, as long as the overload continues.

Request queueing

Since inflight limits are, at most, a multiple of the current inflight ingester requests, some queueing of requests occurs when an ingester’s reactive limiter is full. This allows short bursts of requests to be tolerated while still keeping the number of inflight requests under control. The amount of blocking that is allowed is based on the initial-rejection-factor and max-rejection-factor, and requests only queue when the limiter is full.

Request rejection

Ingesters support separate reactive limiters for push and read requests, since these requests may be overloaded by different resources at different times. When ingesters are heavily loaded, these limiters may be full, and when the load is sustained, queues might fill up as well. When this happens, requests may be rejected, with a preference for rejecting read requests before rejecting push requests.

Note
Push request reactive limiters aren’t available in ingest storage architecture. Refer to Grafana Mimir architecture.

Configure Grafana Mimir reactive limiters

To enable Grafana Mimir ingester push and read reactive limiters, set -ingester.push-reactive-limiter.enabled=true and -ingester.read-reactive-limiter.enabled=true. You can enable the push and read reactive limiters independently. Enabling one of them does not require enabling the other one.

Primary configuration

When enabled, reactive limiters come with some default configurations that work well for many workloads. First among these are configurations for the recent and baseline response time windows. These track how changes in recent, short-term response times compare to long-term baseline response times.

When requests are processed by ingesters, their response times are first aggregated in the recent window. A quantile from these is regularly taken once the window is full and added to the baseline window. By default, the minimum duration of the recent window is 1s, the minimum number of responses that must be observed is 50, and the aggregated response time quantile that’s taken is .9. The baseline window defaults to an age of 10 measurements, which are smoothed over time. These values can be changed via recent-window-min-duration, recent-window-max-duration, recent-window-min-samples, recent-quantile, and baseline-window-age.

The inflight request limit has a default range of 2 to 200, and an initial value of 20. You can change these values via min-limit, max-limit, and initial-limit. Additionally, the inflight limit only increases to a multiple of the current inflight requests, which defaults to 5.0. You can change this value via max-limit-factor.

Additional configurations

While response times are the primary mechanism for detecting overload, a secondary mechanism is tracking the correlation between inflight requests and throughput. The default size of this correlation window is 50 samples, which you can change via correlation-window.

The amount of queueing that is allowed when a limiter is full is based on the inflight request limit. By default, requests begin to be rejected instead of queued when the queue reaches 2 times the inflight limit, and all requests are rejected when the queue reaches 3 times the inflight limit. You can configure these settings via initial-rejection-factor and max-rejection-factor.

Rejection rates, which are based on recent limiter throughput statistics, are computed at 1s intervals by default for all limiters. You can adjust this setting via -ingester.rejection-prioritizer.calibration-interval.

Grafana Mimir reactive limiter metrics

Grafana Mimir ingester reactive limiters add the following metrics. These metrics aren’t part of any API guarantee, and you can change them at any time:

cortex_ingester_reactive_limiter_inflight_requests: Gauge showing the current number of requests that are inflight within the reactive limiter. It contains the request_type label, which is either push or read.
cortex_ingester_reactive_limiter_inflight_limit: Gauge showing the current inflight request limit. It contains the request_type label, which is either push or read.
cortex_ingester_reactive_limiter_queued_requests: Gauge showing the current number of requests that are queued waiting on the limiter. It contains the request_type label, which is either push or read.
cortex_ingester_rejection_rate: Gauge showing the current rate that new requests are rejected at.

About Grafana Mimir ingester circuit breakers

Wed, 03 Jun 2026 09:01:40 +0200

About Grafana Mimir ingester circuit breakers

You can use circuit breakers to prevent an application from repeatedly trying to run an operation that is likely to fail. A circuit breaker monitors the number of recent failures and then uses this information to decide whether to allow a new operation to proceed, or to simply return an exception immediately. In the case of a failing operation, a circuit breaker allows an application to proceed, without waiting for you to resolve the failure cause. Because the failing operation is immediately rejected, the application doesn’t retry it. This reduces the application’s CPU usage.

The circuit breaker pattern operates in three states: closed, open, and half-open.

In the closed state, a circuit breaker operates normally, forwarding all requests to the application it protects.
In the open state, the circuit breaker immediately stops forwarding requests to the failing application, effectively isolating it.
After a specified timeout period in the open state, the circuit breaker transitions to the half-open state, where it forwards a limited number of trial requests to the application and monitors their execution. Successful trial requests indicate application recovery, and the circuit breaker transitions back to the closed state. Failing trial requests indicate that the issues persist, and the circuit breaker transitions back to the open state.

How do Grafana Mimir Ingester circuit breakers work?

A request to a resource protected by a circuit breaker follows these steps:

The request tries to acquire a circuit breaker permit.
If the circuit breaker is open, no permit is acquired, and the request fails with a circuit breaker open error.
Otherwise, the request acquires a circuit breaker permit and runs.
If the request meets the circuit breaker’s failure condition, the circuit breaker records a failure. Otherwise, the circuit breaker records a success.

Depending on the configured frequencies of successes and failures, the circuit breaker transits from one state to another.

Note
Grafana Mimir ingester circuit breakers are an experimental feature. They independently protect Mimir’s write and read paths from slow requests. They don’t protect ingesters from other issues.

More precisely, Grafana Mimir ingesters distinguish between push requests circuit breakers and read requests circuit breakers, and you can configure them independently. It is possible to configure the maximum allowed duration of both push and read requests, as well as the highest frequency of the slow requests before the circuit breakers open and start protecting the ingesters.

Push requests circuit breakers

A push request meets the push requests circuit breaker failure condition if its duration is longer than the configured maximum push request duration.

Note
Push requests circuit breakers aren’t available in ingest storage architecture. Refer to Grafana Mimir architecture.

Read requests circuit breakers

Read requests circuit breakers follow these conditions:

In order to protect the write path as much as possible, ingesters do not allow read push requests if their push circuit breakers are open. This means that before an ingester tries to acquire a read circuit breaker permit, it first checks if its push circuit breaker is open. If it’s open, a circuit breaker open error is returned. Otherwise, the ingester tries to acquire a read requests circuit breaker permit.
A read request that acquires a read requests circuit breaker permit meets the read requests circuit breaker failure condition if its duration is longer than the configured read request maximum duration.

Configure Grafana Mimir ingester circuit breakers

To enable Grafana Mimir ingester push and read circuit breakers, set -ingester.push-circuit-breaker.enabled=true and -ingester.read-circuit-breaker.enabled=true. You can enable the circuit breakers independently. Enabling one of them does not require enabling the other one.

When enabled, push and read circuit breakers come with some default configurations. For example, the percentage of failing requests is computed over a moving window of one minute (1m). To change these configurations, set -ingester.push-circuit-breaker.thresholding-period or -ingester.read-circuit-breaker.thresholding-period to your desired values.

The default percentage of push and pull requests that can fail over the configured moving window before the corresponding circuit breaker opens is 10% (10). You can set a different percentage via the -ingester.push-circuit-breaker.failure-threshold-percentage and -ingester.read-circuit-breaker.failure-threshold-percentage flags.

After a circuit breaker reaches the open state, it waits 10 seconds (10s) before it transitions to the half-open state and allows trial requests. You can change these waiting times via the -ingester.push-circuit-breaker.cooldown-period and -ingester.read-circuit-breaker.cooldown-period flags.

Grafana Mimir ingester push and read circuit breakers come with a default configuration for the maximum duration of a request before it triggers a timeout. The default values are two seconds (2s) for push requests, and thirty seconds (30s) for read requests. You can configure these settings via the -ingester.push-circuit-breaker.request-timeout and -ingester.read-circuit-breaker.request-timeout settings. Circuit breakers use these timeouts internally and never report them as errors.

Grafana Mimir Ingester circuit breakers metrics

Grafana Mimir ingester circuit breakers add several metrics, which aren’t part of any API guarantee and can be changed at any time.

cortex_ingester_circuit_breaker_transitions_total: Counter showing the number of times a circuit breaker enters a state. It contains the labels state, with possible values of closed, open, and half-open, and the request_type setting , with possible values of push and read.
cortex_ingester_circuit_breaker_results_total: Counter showing the results of executing requests via a circuit breaker. It contains lables result, with possible values of success, error, and circuit_breaker_open, and the request_type setting, with possible values of push and read.
cortex_ingester_circuit_breaker_request_timeouts_total: Counter showing the number of times the circuit breaker records a request that reaches timeout. It contains the request_type label, with possible values of push and read.
cortex_ingester_circuit_breaker_current_state: Gauge set to 1 when the circuit breaker is in a state corresponding to the label name. It contains the labels state, with possible values of closed, open, and half-open, and the request_type setting with possible values of push and read.

Configure resource utilization based ingester read path limiting

Wed, 03 Jun 2026 09:01:40 +0200

Configure resource utilization based ingester read path limiting

Grafana Mimir allows you to configure limits for CPU and/or memory utilization, which cause the ingester to reject incoming read requests while either of the limits is reached. The idea is to prevent expensive queries from interrupting the write path, ensuring a minimum of headroom for the latter. The ingester’s model of memory utilization corresponds to the Go heap size, which it tracks, along with a sliding window average of the process’ CPU utilization.

The process’ CPU utilization and Go memory heap size are sampled every second, and a sliding window average is taken of the CPU utilization. If either is greater than or equal to the corresponding configured limit, ingester read requests get rejected with HTTP status code 503 (Service Unavailable), until utilization levels are below respective limits again.

Whenever the ingester rejects a read request due to utilization based limiting, it increments the cortex_ingester_utilization_limited_read_requests_total counter metric. CPU and memory utilization are also tracked, via the cortex_ingester_utilization_limiter_current_cpu_load and cortex_ingester_utilization_limiter_current_memory_usage_bytes gauge metrics respectively.

To configure resource utilization based ingester read path limiting, you may use the following flags:

-ingester.read-path-cpu-utilization-limit: CPU utilization limit, as CPU cores
-ingester.read-path-memory-utilization-limit: Memory limit, in bytes

Alternatively, you may configure the ingester via YAML, as in the following snippet:

ingester:
  # Configure ingester to reject read requests when average CPU utilization is >= 3.5 cores
  read_path_cpu_utilization_limit: 3.5
  # Configure ingester to reject read requests when memory utilization is >= 16 GiB
  read_path_memory_utilization_limit: 17179869184

Configure custom active series trackers

Wed, 03 Jun 2026 09:01:40 +0200

Configure custom active series trackers

You can use the custom tracker feature to count the number of active series on an ingester that match a particular label pattern.

The label pattern to match against is specified using the -ingester.active-series-custom-trackers CLI flag (or its respective YAML configuration option). Each custom tracker is defined as a key-value pair, where the key is the name of the tracker and the value is the label matcher. Both the key and the value are type <string>.

The following example configures a custom tracker to count the active series coming from dev and prod namespaces for each tenant.

active_series_custom_trackers:
  dev: '{namespace=~"dev-.*"}'
  prod: '{namespace=~"prod-.*"}'

If you configure a custom tracker for an ingester, the ingester exposes a cortex_ingester_active_series_custom_tracker gauge metric on its /metrics endpoint.

Each custom tracker counts the active series matching its label pattern on a per-tenant basis, which means that each custom tracker generates as many as # of tenants series with metric name cortex_ingester_active_series_custom_tracker. To reduce the cardinality of this metric, only custom trackers that have matched at least one series are exposed on the metric, and they are removed if they become 0.

Series with metric name cortex_ingester_active_series_custom_tracker have two labels applied: name and user. The value of the name label is the name of the custom tracker specified in the configuration. The value of the user label is the tenant-id for which the series count applies.

To illustrate this, assume that two custom trackers are configured as in the preceding YAML snippet, and that your Grafana Mimir cluster has two tenants: tenant_1 and tenant_with_only_prod_metrics. Assume that tenant_with_only_prod_metrics has three series with labels that match the pattern {namespace=~"prod-.*"} and none that match the pattern {namespace=~"dev-.*"}. Also assume that tenant_1 has five series that match the pattern {namespace=~"dev-.*"} and 10 series that match the pattern {namespace=~"prod-.*"}.

In this example, the following output appears when the /metrics endpoint for the ingester component is scraped:

cortex_ingester_active_series_custom_tracker{name="dev", user="tenant_1"}                         5
cortex_ingester_active_series_custom_tracker{name="prod", user="tenant_1"}                       10
cortex_ingester_active_series_custom_tracker{name="prod", user="tenant_with_only_prod_metrics"}   3

For specific tenants, you can override the default configuration as previously described. To do so, edit the runtime configuration.

You can override the active series custom trackers’ configuration for the tenant tenant_with_only_prod_metrics to track two services instead of the default matchers. See the following example:

overrides:
  tenant_with_only_prod_metrics:
    active_series_custom_trackers:
      service1: '{service="service1"}'
      service2: '{service="service2"}'

After adding this override, and assuming that there is one matching series for service1 and two matching series for service2, the output at /metrics changes:

cortex_ingester_active_series_custom_tracker{name="dev", user="tenant_1"}                                           5
cortex_ingester_active_series_custom_tracker{name="prod", user="tenant_1"}                                         10
cortex_ingester_active_series_custom_tracker{name="service1", user="tenant_with_only_prod_metrics"}                 1
cortex_ingester_active_series_custom_tracker{name="service2", user="tenant_with_only_prod_metrics"}                 2

To set up runtime overrides, refer to runtime configuration.

Note
Each ingester exposes the custom active series trackers.

To understand the count of active series matching a particular label pattern in your Grafana Mimir cluster at a global level, you must collect and sum this metric across all ingesters.

If you’re running Grafana Mimir with a replication_factor > 1, you must also adjust for the fact that Mimir replicates the same series across your ingesters replication factor times.

Configure queries to block

Wed, 03 Jun 2026 09:01:40 +0200

Configure queries to block

In certain situations, you might want to control what queries are being sent to your Mimir installation. These queries might be intentionally or unintentionally expensive to run, and they might affect the overall stability or cost of running your service.

Each rule must include a pattern field; rules without a pattern are a configuration error. Rules with regex: true must have a valid regular expression pattern; an invalid pattern is also a configuration error. To match all queries, use pattern: ".*" with regex: true. Optional filter conditions (time_range_longer_than, step_size_shorter_than, unaligned_range_queries) narrow which matching queries are blocked; all configured conditions must be satisfied to block.

You can block queries using per-tenant overrides:

overrides:
  "tenant-id":
    blocked_queries:
      # exact match
      - pattern: 'sum(rate(node_cpu_seconds_total{env="prod"}[1m]))'

      # regex match
      - pattern: '.*env="prod".*'
        regex: true

      # match all queries longer than 7 days
      - pattern: ".*"
        regex: true
        time_range_longer_than: 7d
        reason: "queries longer than 7 days are not allowed"

      # match expensive queries longer than 1 day
      - pattern: ".*expensive.*"
        regex: true
        time_range_longer_than: 24h
        reason: "expensive queries over 1 day are blocked"

      # match all queries with a step shorter than 1 minute
      - pattern: ".*"
        regex: true
        step_size_shorter_than: 1m
        reason: "step resolution too fine-grained"

      # match this query only when the time range is not a multiple of the step
      - pattern: 'sum(rate(node_cpu_seconds_total{env="prod"}[1m]))'
        unaligned_range_queries: true

The blocking is enforced on instant and range queries as well as remote read queries.

For instant and range queries, the pattern is evaluated against the query. For remote read requests, the pattern is evaluated against each set of matchers, as if the matchers formed a vector selector. If any set of matchers is blocked, the whole remote read request is rejected.

Setting time_range_longer_than on a rule blocks queries where the time range duration (calculated as end - start) exceeds the specified threshold. time_range_longer_than does not apply to instant queries.

Setting step_size_shorter_than on a rule blocks queries where the step is shorter than the configured duration. step_size_shorter_than does not apply to instant queries or queries without a step.

Setting unaligned_range_queries: true on a rule limits it to range queries where the time range is not a multiple of the step. Such queries are not eligible for range query result caching by default. This can be useful to discourage unaligned queries without impacting clients that already send aligned requests. unaligned_range_queries does not apply to instant queries, aligned range queries, or remote read requests.

For remote read requests, each set of matchers is evaluated as a vector selector. For example, a matcher on __name__ regex-matched to foo.* is interpreted as {__name__=~"foo.*"}. To restrict the blocking to such selectors, use a regex pattern with the curly braces escaped, e.g. pattern: '\{.*foo.*\}' with regex: true.

To set up runtime overrides, refer to runtime configuration.

Note
The order of patterns is preserved, so the first matching pattern will be used.

Format queries to block

You can ignore this section if you’re not using a regular expression. If you are, you need to ensure that it matches against formatted queries as follows.

Use Mimirtool’s mimirtool promql format <query> command to apply the Prometheus formatter to a query that is expected to be blocked by the regular expression pattern, and then check the formatted query will match the regular expression.

Mimir parses queries into PromQL expressions before blocking is applied. When a regular expression is not used, the pattern from the blocked queries is also similarly parsed before being compared against the formatted representation of the parsed query. When a regular expression is used, the pattern is used as-is. This process allows for consistent query blocking behavior regardless of formatting differences in the submitted queries.

Among other transformations the Prometheus formatter may reorder operators, remove empty selector braces, and eliminate newlines, extraneous whitespace, and comments.

Formatted query examples

Empty selector braces removed:

mimirtool promql format 'foo{}'

foo

Operators reordered:

mimirtool promql format 'sum(container_memory_rss) by (namespace)'

sum by (namespace) (container_memory_rss)

Newlines, extra whitespace, and comments eliminated:

mimirtool promql format '
rate(
  metric_counter[15m] # comment 1
) /
rate(
  other_counter[15m] # comment 2
)
'

rate(metric_counter[15m]) / rate(other_counter[15m])

View blocked queries

Blocked queries are logged with msg="query blocked" at info level, including the query text, duration, step, rule index, and reason. Use these fields to identify which rule matched and why.

Blocked queries are also counted in the cortex_query_frontend_rejected_queries_total metric per tenant (user label) with reason="blocked". Note that the same metric tracks rate-limited queries under reason="limited".

To see the rate of blocked queries by tenant:

sum by (user, reason) (rate(cortex_query_frontend_rejected_queries_total[$__interval]))

Troubleshoot invalid configuration

Invalid rules are rejected when the runtime configuration is loaded. The behaviour differs depending on whether the invalid configuration is present at startup or introduced at runtime.

At startup, Mimir logs an error at ERROR level and exits, resulting in a crash loop in Docker Compose or Kubernetes until the configuration is corrected. Search for msg="module failed" with module=runtime-config in your logs:

Missing pattern field:

level=error msg="module failed" module=runtime-config err="starting module runtime-config: invalid service state: Failed, expected: Running, failure: failed to load runtime config: load file: tenant \"anonymous\": blocked_queries[0]: pattern is required"

Invalid regex pattern with regex: true:

level=error msg="module failed" module=runtime-config err="starting module runtime-config: invalid service state: Failed, expected: Running, failure: failed to load runtime config: load file: tenant \"anonymous\": blocked_queries[0]: invalid regex pattern \"[a-9}\": error parsing regexp: invalid character class range: `a-9`"

At runtime, Mimir rejects the updated config and continues running with the previous valid configuration. Search for msg="failed to load config" in your logs:

level=error msg="failed to load config" err="load file: tenant \"anonymous\": blocked_queries[1]: invalid regex pattern \"[a-9}\": error parsing regexp: invalid character class range: `a-9`"

In both cases, the err field identifies the tenant, rule index, and the specific problem. Correct or remove the invalid rule.

Configure experimental PromQL functions

Wed, 03 Jun 2026 09:01:40 +0200

Configure experimental PromQL functions

Experimental PromQL functions are disabled by default in Mimir. You can selectively enable them per-tenant or globally for all users. You can enable all functions using the special function name all, or you can specifically enable individual functions.

You can enable functions globally for all users using the main Mimir configuration file.

limits:
  enabled_promql_experimental_functions: info,sort_by_label,sort_by_label_desc

You can enable functions for specific tenants using per-tenant overrides:

overrides:
  "tenant-id-1":
    # Enable only specific functions for this tenant
    enabled_promql_experimental_functions: sort_by_label,sort_by_label_desc
  "tenant-id-2":
    # Enable all experimental functions for this tenant
    enabled_promql_experimental_functions: all

Blocking or allowing the use of experimental PromQL functions is enforced on instant and range queries. When blocked an error is returned when running a query that includes the name of the experimental function.

 function "sort_by_label" is not enabled

Configure Grafana Mimir zone-aware replication

Wed, 03 Jun 2026 09:01:40 +0200

Configure Grafana Mimir zone-aware replication

Zone-aware replication is the replication of data across failure domains. Zone-aware replication helps to avoid data loss during a domain outage. Grafana Mimir defines failure domains as zones. Depending on the underlying infrastructure that Grafana Mimir is deployed on, zones could be, for example,

Availability zones
Data centers
Racks
Anti-affinity groups within a single availability zone

Without zone-aware replication enabled, Grafana Mimir replicates data randomly across all component replicas, regardless of whether the replicas are running within the same zone. Even with a Grafana Mimir cluster deployed across multiple zones, the replicas for any given data could reside in the same zone. If an outage affects a zone containing multiple replicas, data loss might occur.

With zone-aware replication enabled, Grafana Mimir ensures data replication to replicas across different zones.

Warning
Ensure that you configure deployment tooling so that it’s also zone-aware. The deployment tooling is responsible for executing rolling updates.

Rolling updates should only update replicas in a single zone at any given time. You can utilize the Kubernetes rollout Operator to assist with this.

Grafana Mimir supports zone-aware replication for the following:

Configuring Alertmanager alerts replication

Zone-aware replication in the Alertmanager ensures that Grafana Mimir replicates alerts across -alertmanager.sharding-ring.replication-factor Alertmanager replicas, with one replica located in each zone.

To enable zone-aware replication for alerts:

Configure the zone of each Alertmanager replica via the -alertmanager.sharding-ring.instance-availability-zone CLI flag or its respective YAML configuration parameter.
Roll out Alertmanagers so that each Alertmanager replica runs with a configured zone.
Set the -alertmanager.sharding-ring.zone-awareness-enabled=true CLI flag or its respective YAML configuration parameter for Alertmanagers.

Configuring ingester time series replication

Mimir replicates time series across ingesters in fundamentally different ways between the ingest storage and classic architectures.

Configuring ingester time series replication in ingest storage architecture

Note
This guidance applies to ingest storage architecture. For more information about the supported architectures in Grafana Mimir, refer to Grafana Mimir architecture.

In the ingest storage architecture, distributors shard incoming series across Kafka partitions, writing each series to a single partition. Series are not replicated across partitions, since Kafka itself provides replication within each partition.

Each ingester owns one Kafka partition and continuously consumes series data from it, making that data available for querying. A partition can be assigned to multiple ingesters for high availability – typically two ingesters per partition, corresponding to a replication factor of two.

In this architecture, you don’t need to configure an explicit replication factor. Zone-aware replication is achieved simply by deploying the ingesters for the same partition in different zones.

To enable zone-aware replication for time series:

Configure the zone of each ingester replica via the -ingester.ring.instance-availability-zone CLI flag or its respective YAML configuration parameter. The -ingester.ring.replication-factor CLI flag is not used in the ingest storage architecture.
Roll out ingesters so that each ingester replica runs with a configured zone.
Set the -ingester.ring.zone-awareness-enabled=true CLI flag or its respective YAML configuration parameter for distributors, ingesters, and queriers.

Configuring ingester time series replication in classic architecture

Note
This guidance applies to classic architecture. For more information about the supported architectures in Grafana Mimir, refer to Grafana Mimir architecture.

In the classic architecture, distributors shard and replicate incoming series across ingesters, writing each series to the number of ingesters defined by -ingester.ring.replication-factor. When zone-aware replication is enabled, each time series is replicated to ingesters located in different zones.

To enable zone-aware replication for time series:

Configure the zone of each ingester replica via the -ingester.ring.instance-availability-zone CLI flag and the replication factor via the -ingester.ring.replication-factor CLI flag. You can also set their respective YAML configuration parameters.
Roll out ingesters so that each ingester replica runs with a configured zone.
Set the -ingester.ring.zone-awareness-enabled=true CLI flag or its respective YAML configuration parameter for distributors, ingesters, and queriers.

Minimum number of zones in classic architecture

To ensure zone-aware replication, deploy Grafana Mimir across a number of zones equal-to or greater-than the configured replication factor. With a replication factor of 3, which is the default, deploy the Grafana Mimir cluster across at least three zones.

Deploying Grafana Mimir clusters to more zones than the configured replication factor does not have a negative impact. Deploying Grafana Mimir clusters to fewer zones than the configured replication factor can cause writes to the replica to be missed, or can cause writes to fail completely.

A Grafana Mimir deployment requires at least floor(replication factor / 2) + 1 healthy zones to operate.

Configuring store-gateway blocks replication

To enable zone-aware replication for the store-gateways, refer to Zone awareness.

Unbalanced zones

To ensure that the workload across zones is balanced, run the same number of replicas of each component in each zone. When replica counts are unbalanced, zones with fewer replicas have higher resource utilization than those with more replicas.

Costs

Most cloud providers charge for inter-availability zone networking. Deploying Grafana Mimir with zone-aware replication across multiple cloud provider availability zones likely results in additional networking costs.

Note
In classic architecture, the requests that the distributors receive are usually compressed, and the requests that the distributors send to the ingesters are uncompressed by default. This can result in increased cross-zone bandwidth costs because at least two ingesters will be in different availability zones. If this cost is a concern, you can compress those requests by setting the -ingester.client.grpc-compression CLI flag, or its respective YAML configuration parameter, to snappy or gzip in the distributors.

In the ingest storage architecture, the Kafka records produced by distributors are always sent to Kafka in compressed form.

Kubernetes operator for simplifying rollouts of zone-aware components

The Kubernetes Rollout Operator is a Kubernetes Operator that makes it easier for you to manage multi-availability-zone rollouts. Consider using the Kubernetes Rollout Operator when you run Grafana Mimir on Kubernetes with zone awareness enabled.

Note that even in a scenario where you have deployed Grafana Mimir to a single physical availability zone, the Rollout Operator is immensely useful.

This is because the Operator automates and accelerates the process of rollouts by:

Allowing an entire zone to be updated at one time
Prioritizing rollout of unhealthy zones over healthy zones to ensure stability
Allowing the administrator to determine how aggressive rollouts should be

The automated functionality provided by an Operator is particularly useful in larger deployments where rollouts would otherwise take a very long time and likely require manual intervention if issues were to occur.

Enabling zone-awareness via the Grafana Mimir Jsonnet

Instead of configuring Grafana Mimir directly, you can use the Grafana Mimir Jsonnet to enable ingester and store-gateway zone awareness. To enable ingester and store-gateway zone awareness, set the top level multi_zone_store_gateway_enabled or multi_zone_ingester_enabled Jsonnet fields to true. These flags set the required Grafana Mimir configuration parameters that support ingester and store-gateway zone awareness.

Mirror requests to a second Grafana Mimir cluster

Wed, 03 Jun 2026 09:01:40 +0200

Mirror requests to a second Grafana Mimir cluster

Use request mirroring when you want to set up a Grafana Mimir testing cluster that receives the same series that are ingested by a primary cluster and when you don’t have control over the Prometheus remote write configuration.

If you have control over the Prometheus remote write configuration, we recommend that you configure two remote write entries in Prometheus. For more information about Prometheus remote write configuration, refer to Prometheus remote write reference.

Mirroring with Envoy proxy

You can use Envoy proxy to mirror HTTP requests to a secondary upstream cluster. From a network path perspective, run Envoy in front of both clusters’ distributors.

This approach enables the Envoy proxy to make requests to the primary Grafana Mimir cluster and then in the background, Envoy mirrors the requests to the secondary cluster.

The performance and availability of the secondary cluster have no impact on the requests to the primary cluster. The response to the client is always sent from the primary cluster.

Requests from Envoy to the secondary cluster are “fire and forget”, which means that Envoy doesn’t wait until the requests are completed to the secondary cluster before sending back the response to the client.

The following diagram illustrates a simplified network structure.

Example Envoy configuration

The following Envoy configuration shows an example that includes two Grafana Mimir clusters. Envoy listens on port 9900 and proxies all requests to mimir-primary:8080, while also mirroring them to mimir-secondary:8080.

admin:
  # No access logs.
  access_log_path: /dev/null
  address:
    socket_address: { address: 0.0.0.0, port_value: 9901 }

static_resources:
  listeners:
    - name: mimir_listener
      address:
        socket_address: { address: 0.0.0.0, port_value: 9900 }
      filter_chains:
        - filters:
            - name: envoy.http_connection_manager
              config:
                stat_prefix: mimir_ingress
                route_config:
                  name: all_routes
                  virtual_hosts:
                    - name: all_hosts
                      domains: ["*"]
                      routes:
                        - match: { prefix: "/" }
                          route:
                            cluster: mimir_primary

                            # Specifies the upstream timeout. This spans between the point at which the entire downstream
                            # request has been processed and when the upstream response has been completely processed.
                            timeout: 15s

                            # Specifies the cluster that requests will be mirrored to. The performance
                            # and availability of the secondary cluster have no impact on the requests to the primary
                            # one. The response to the client will always be the one from the primary one. In this sense,
                            # the requests from Envoy to the secondary cluster are "fire and forget".
                            request_mirror_policies:
                              - cluster: mimir_secondary
                http_filters:
                  - name: envoy.router
  clusters:
    - name: mimir_primary
      type: STRICT_DNS
      connect_timeout: 1s
      # Replace mimir-primary with the address and port the distributor of your primary mimir cluster
      hosts: [{ socket_address: { address: mimir-primary, port_value: 8080 }}]
      dns_refresh_rate: 5s
    - name: mimir_secondary
      type: STRICT_DNS
      connect_timeout: 1s
      # Replace mimir-secondary with the address and port the distributor of your secondary mimir cluster
      hosts: [{ socket_address: { address: mimir-secondary, port_value: 8080 }}]
      dns_refresh_rate: 5s

About Grafana Mimir DNS service discovery

Wed, 03 Jun 2026 09:01:40 +0200

About Grafana Mimir DNS service discovery

Some clients in Grafana Mimir support service discovery via DNS to locate the addresses of backend servers to connect to. When using DNS service discovery, search domains are not supported. As a result, all domain names must be fully qualified. The following clients support service discovery via DNS:

Memcached server addresses
- -blocks-storage.bucket-store.chunks-cache.memcached.addresses
- -blocks-storage.bucket-store.index-cache.memcached.addresses
- -blocks-storage.bucket-store.metadata-cache.memcached.addresses
- -query-frontend.results-cache.memcached.addresses
Memberlist KV store
- -memberlist.join
Alertmanager URL configured in the ruler
- -ruler.alertmanager-url

Supported discovery modes

DNS service discovery supports different discovery modes. You select a discovery mode by adding one of the following supported prefixes to the address:

dns+
The domain name after the prefix is looked up as an A/AAAA query. For example: dns+memcached.local:11211.
dnssrv+
The domain name after the prefix is looked up as a SRV query, and then each SRV record is resolved as an A/AAAA record. For example: dnssrv+_memcached._tcp.memcached.namespace.svc.cluster.local.
dnssrvnoa+
The domain name after the prefix is looked up as a SRV query, with no A/AAAA lookup made after that. For example: dnssrvnoa+_memcached._tcp.memcached.namespace.svc.cluster.local.

The Alertmanager URL must also contain the scheme. For example: dns+http://alermanager.local:8080.

If no prefix is provided, the provided IP or hostname is used without pre-resolving it.

Configure Grafana Mimir tracing

Wed, 03 Jun 2026 09:01:40 +0200

Configure Grafana Mimir tracing

Distributed tracing is a valuable tool for troubleshooting the behavior of Grafana Mimir in production.

Grafana Mimir supports distributed tracing using OpenTelemetry and provides backward compatibility with Jaeger configuration.

Mimir automatically detects which tracing configuration method to use based on the environment variables you set:

OpenTelemetry format: Uses standard OTel environment variables for modern tracing backends.
Jaeger format: Uses Jaeger-specific environment variables for backward compatibility.

Dependencies

Set up a tracing backend to collect and store traces from Grafana Mimir:

For OpenTelemetry: Use any OTLP-compatible tracing backend such as Jaeger, Tempo, or cloud-based solutions.
For Jaeger: Set up a Jaeger deployment including either the Jaeger all-in-one binary or a distributed system of agents, collectors, and queriers.

If you run Grafana Mimir on Kubernetes, refer to Jaeger Kubernetes.

Configuration

Grafana Mimir automatically detects the tracing configuration method based on the environment variables you set. You can use either OpenTelemetry standard environment variables or Jaeger-specific environment variables.

OpenTelemetry Configuration (Recommended)

To configure Grafana Mimir with OpenTelemetry format tracing, set any of the following environment variables.

OpenTelemetry configuration follows the standard documentation from OpenTelemetry SDK Configuration.

OTEL_EXPORTER_OTLP_ENDPOINT: The OTLP endpoint URL. For example, http://otlp-endpoint:4318.
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT: The OTLP traces-specific endpoint URL. This value overrides the previous variable. For example, http://tempo:4318/v1/traces.
OTEL_TRACES_EXPORTER: The traces exporter to use. Default: otlp.
OTEL_TRACES_SAMPLER: The sampling strategy to use.

Traces exporter configuration

The OTEL_TRACES_EXPORTER environment variable specifies which trace exporter to use:

otlp (default): OpenTelemetry Protocol exporter.
none: No trace exporter. You can set this value if you want to configure the tracing library to propagate trace context without exporting traces.

Note
The jaeger exporter option is not available, as it was deprecated by the OpenTelemetry project in 2023. Instead, use the otlp exporter, since Jaeger supports OTLP ingestion natively.

Sampling configuration

Configure sampling behavior using the OTEL_TRACES_SAMPLER environment variable:

always_on: Sample all traces
always_off: Sample no traces
traceidratio: Sample traces based on trace ID ratio
parentbased_always_on (default): Sample based on parent span decision. Always sample root spans.
parentbased_always_off: Sample based on parent span decision. Never sample root spans.
parentbased_traceidratio: Sample based on parent span decision, use ratio for root spans
jaeger_remote: Use Jaeger remote sampling
parentbased_jaeger_remote: Use parent-based Jaeger remote sampling

For jaeger_remote and parentbased_jaeger_remote samplers, you must also set OTEL_TRACES_SAMPLER_ARG with the configuration in the format:

endpoint=http://localhost:14250,pollingIntervalMs=5000,initialSamplingRate=0.25

Propagation configuration

Configure trace context propagation using OTEL_PROPAGATORS:

tracecontext: W3C Trace Context (default)
baggage: W3C Baggage (default)
jaeger: Jaeger propagation (default)
none: No propagation

Default propagators include tracecontext, baggage, and jaeger.

Example OpenTelemetry configuration

# Basic OTLP configuration
export OTEL_EXPORTER_OTLP_ENDPOINT="http://jaeger:4318"
export OTEL_TRACES_SAMPLER="always_on"

# Advanced configuration with Jaeger remote sampling
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="http://jaeger:4318/v1/traces"
export OTEL_TRACES_SAMPLER="jaeger_remote"
export OTEL_TRACES_SAMPLER_ARG="endpoint=http://jaeger:14250,pollingIntervalMs=5000,initialSamplingRate=0.25"
export OTEL_PROPAGATORS="tracecontext,baggage,jaeger"

Jaeger configuration (deprecated)

Caution
Jaeger tracing configuration using JAEGER_* environment variables is deprecated since Mimir 2.17. Use OpenTelemetry configuration instead, as Jaeger supports OTLP ingestion natively.

To configure Grafana Mimir with Jaeger format tracing, set any of the following environment variables:

JAEGER_AGENT_HOST: Jaeger agent hostname
JAEGER_ENDPOINT: Jaeger collector endpoint URL
JAEGER_SAMPLER_MANAGER_HOST_PORT: Jaeger sampler manager endpoint

Required environment variables

You must specify at least one of the following environment variables to enable Jaeger configuration:

JAEGER_AGENT_HOST
JAEGER_ENDPOINT
JAEGER_SAMPLER_MANAGER_HOST_PORT

Sampling configuration

Configure sampling using Jaeger-specific environment variables:

JAEGER_SAMPLER_TYPE: Sampling strategy (const, probabilistic, remote)
JAEGER_SAMPLER_PARAM: Sampling parameter value
JAEGER_SAMPLER_MANAGER_HOST_PORT: Remote sampling endpoint
JAEGER_SAMPLING_ENDPOINT: Alternative remote sampling endpoint

Additional Jaeger environment variables

JAEGER_AGENT_PORT: Jaeger agent port (default: 6831)
JAEGER_TAGS: Additional tags to add to spans (format: key1=value1,key2=value2)

Example Jaeger configuration

# Basic agent configuration
export JAEGER_AGENT_HOST="jaeger-agent"
export JAEGER_SAMPLER_TYPE="const"
export JAEGER_SAMPLER_PARAM="1"

# Collector endpoint configuration
export JAEGER_ENDPOINT="http://jaeger-collector:14268/api/traces"
export JAEGER_SAMPLER_TYPE="probabilistic"
export JAEGER_SAMPLER_PARAM="0.1"

# Remote sampling configuration
export JAEGER_AGENT_HOST="jaeger-agent"
export JAEGER_SAMPLER_MANAGER_HOST_PORT="jaeger-agent:5778"

For a complete list of Jaeger environment variables, refer to the Jaeger Client Go documentation.

Enabling sampling

Enable sampling in the appropriate components:

The ingester and ruler self-initiate traces. You should explicitly enable sampling for these components.
You can enable sampling for the distributor and query-frontend in Grafana Mimir or in an upstream service, like a proxy or gateway running in front of Grafana Mimir

Note
Mimir automatically detects the configuration method based on the environment variables present. OpenTelemetry configuration takes precedence if both OTel and Jaeger environment variables are set.

About Grafana Mimir IP address logging of a reverse proxy

Wed, 03 Jun 2026 09:01:40 +0200

About Grafana Mimir IP address logging of a reverse proxy

If a reverse proxy is used in front of Mimir, it might be difficult to troubleshoot errors. You can use the following settings to log the IP address passed along by the reverse proxy in headers such as X-Forwarded-For.

-server.log-source-ips-enabled

Set this to true to add IP address logging when a Forwarded, X-Real-IP, or X-Forwarded-For header is used. A field called sourceIPs is added to error logs when data is pushed into Grafana Mimir.
-server.log-source-ips-header

The header field stores the source IP addresses and is used only if -server.log-source-ips-enabled is true, and if -server.log-source-ips-regex is set. If you do not set these flags, the default Forwarded, X-Real-IP, or X-Forwarded-For headers are searched.
-server.log-source-ips-regex

A regular expression that is used to match the source IPs. The regular expression must contain at least one capturing group, the first of which is returned. This flag is used only if -server.log-source-ips-enabled is true and if -server.log-source-ips-header is set.
-server.log-source-ips-full

Set this to true to log all source IPs listed in headers instead of only the originating one. Only used if server.log-source-ips-enabled is set to true