Grafana Mimir release notes on Grafana Labs

Grafana Mimir version 2.0 release notes

Wed, 03 Jun 2026 09:01:40 +0200

Grafana Mimir version 2.0 release notes

Grafana Labs is excited to announce the first release of Grafana Mimir, the most scalable, most performant open source time series database in the world. In customer tests, we’ve shown that a single cluster can support more than one billion active time series.

Besides massive scale, Grafana Mimir offers a host of other benefits, including easy deployment, native multi-tenancy, high availability, durable long-term storage, and exceptional query performance on even the highest-cardinality queries.

For more information about Grafana Mimir features and capabilities, watch the Introducing Grafana Mimir video.

We’re launching Grafana Mimir with a 2.0 version number to signal our respect for Cortex, the project from which Grafana Mimir was forked. The choice of 2.0 also represents our conviction that Grafana Mimir is real-world-tested, production-ready software. It has served as the backbone of our Grafana Cloud Metrics and Grafana Enterprise Metrics products since their inception.

For more on the story of why we created Grafana Mimir, please see the following blog posts:

Grafana Mimir builds on Cortex 1.10.0, adding features we developed to run Grafana Enterprise Metrics and Grafana Cloud Metrics at massive scale. As a result, these listed features and enhancements, migration considerations, and bug fixes highlight changes between Cortex 1.10.0 and Grafana Mimir 2.0.

The complete list of changes is recorded in the Changelog.

Features and enhancements

These features and enhancements distinguish Grafana Mimir from Cortex 1.10.0:

Simplified deployment experience: These changes improve the experience of setting up and maintaining Grafana Mimir:
- We revamped Cortex’s single process mode, renaming it monolithic mode. Monolithic mode now runs the required compactor component, and has better configuration defaults, so that it works with fewer changes out-of-the-box.
- We switched to memberlist as the default store for Grafana Mimir’s hash rings. With this, users no longer have to run Consul or etcd as external dependencies. We made performance optimizations to memberlist to reduce its CPU utilization, which ensures that memberlist runs smoothly on Grafana Mimir clusters with lots of active series.
- We’ve included our own internal best practice dashboards, mixins, and alerts for monitoring Grafana Mimir. While installing monitoring best practices such as these has typically required use of Jsonnet, we’ve eliminated this requirement. We include dashboards as JSON and alerting and recording rules as YAML which can be directly imported into your Grafana and Prometheus deployments. The alerts are accompanied by runbooks distilled from our own internal operations.
Configuration parameter reduction and classification: We removed 36% of the configuration parameters in Grafana Mimir. All remaining configuration parameters have been classified as basic, advanced, or experimental. This is meant to make Grafana Mimir’s configuration more approachable for new users. In a default installation, you can focus exclusively on basic configuration. As you become more familiar with Grafana Mimir and want to push your clusters further, you can choose to tune advanced parameters or use experimental parameters. Refer to parameter categories to learn more.
More-scalable metrics compaction: Grafana Mimir’s new compactor uses a split-and-merge compaction algorithm. This new compactor parallelizes the compaction of overlapping blocks across multiple machines, and splits blocks in such a way as to overcome Prometheus’ 64 GB index size limit. This solves a problem that we’ve seen with tenants with greater than 30M time series in Cortex - namely, that the count of uncompacted blocks grew indefinitely because the compactor couldn’t work fast enough. With its new compactor, a single Grafana Mimir cluster can support over one billion active time series. Refer to compactor to learn more.
Query sharding for improved query speeds: Grafana Mimir introduces query sharding to accelerate the execution of high-cardinality or CPU-intensive queries. Query sharding distributes the execution of a single query across multiple machines, to significantly reduce query execution time. We have seen speedups of 10 to 30x in our Grafana Cloud Metrics clusters. Refer to query sharding to learn more.
Federated rule groups: Grafana Mimir makes it possible to write alerting and recording rules that use metrics data from multiple tenants. For example, a user can now create a recording rule that adds metricA from tenantA to metricB from tenantB and writes the result to tenantC. This feature is experimental. For more information on how to use it, refer to federated rule groups.
Understand your metrics cardinality: Grafana Mimir adds two API endpoints to help users identify high-cardinality metrics. The label_names endpoint takes a metric name and returns all label names applied to that metric, as well as the count of values for each label name. When run without a metric name, it returns the highest-cardinality label names. The label_values endpoint returns the highest-cardinality metrics, and can be used to get a count of how many series have a given label-value pair applied. The new custom tracker feature takes this one step further by allowing you to track the count of active series over time that match a specific label matcher.

These features are no longer considered experimental; they are stable features:

Cross tenant query federation is a feature that allows you to issue queries that aggregate series across multiple tenants, providing a way to get a global view of your data. Refer to tenant-federation.enabled in the configuration parameters block to learn more.
Zone-aware replication protects against data loss due to an outage in a specific zone by ensuring data is replicated across multiple zones. In addition to promoting this feature to stable, we’ve also released a Kubernetes operator that makes it easier for those running Grafana Mimir on Kubernetes to manage multi-zone rollouts. Refer to Configuring zone-aware replication to learn more.
Azure Blob storage and OpenStack Swift support: Two additional object storage options are available. Grafana Mimir now supports AWS S3, GCS, Azure Blob Storage, OpenStack Swift, as well as any S3-compatible object storage.
Horizontally scalable Alertmanager: The Alertmanager can distribute its workload among multiple Alertmanager replicas, increasing the maximum number of tenants a cluster can comfortably handle. The Alertmanager uses the tenant ID to determine which replica(s) own which tenant. The replication factor is configurable, so alerts are replicated across multiple replicas, reducing the likelihood of a dropped alert.
Query-scheduler component: The query-scheduler component solves the scalability limitations of the query-frontend component. We strongly encourage users operating larger Grafana Mimir clusters to run this optional component. Refer to the query-scheduler to learn more about this component.

Migration considerations

This information is relevant only to users migrating from Cortex to Grafana Mimir. Those who are not currently using Cortex today can follow our tutorial to Get Mimir up and running in 10 minutes.

Grafana Mimir is 100% compatible with Cortex, and the migration from Cortex to Mimir can be done without any data loss or downtime.

Grafana Mimir 2.0 is a major version upgrade, and therefore it introduces several breaking changes for those migrating from Cortex. To make this upgrade as easy as possible, we’ve added functionality to mimirtool to automate the required configuration changes. We’ve paired this tooling with detailed migration guides that walk through all required changes.

For a demonstration that illustrates how easy this migration can be, please see the following video:

Bug fixes

2.0.0 bug fixes

Version 2.0.0 fixes these bugs:

PR 424: Fixed a bug where cached results for step-aligned queries were used to respond to unaligned queries, causing the user to receive incorrect results. The new configuration parameter query-frontend.cache-unaligned-requests safely enables the caching of query results, even when the query step is not aligned.
PR 224: Fixed a race condition in the Alertmanager, which sometimes resulted in a tenant’s configuration being replaced with the blank fallback configuration.
PR 551: Fixed a bug in memberlist that caused large messages to be corrupted. This bug fix, in addition to the memberlist performance optimizations, enabled us to switch to memberlist as the default store for Grafana Mimir’s hash rings.

Grafana Mimir version 2.1 release notes

Wed, 03 Jun 2026 09:01:40 +0200

Grafana Mimir version 2.1 release notes

Grafana Labs is excited to announce version 2.1 of Grafana Mimir, the most scalable, most performant open source time series database in the world.

Below we highlight the top features, enhancements and bugfixes in this release, as well as relevant callouts for those upgrading from Grafana Mimir 2.0. The complete list of changes is recorded in the Changelog.

Features and enhancements

Mimir on ARM: We now publish Docker images for both amd64 and arm64, making it easier for those on arm-based machines to develop and run Mimir. Multiplaform images are available from the Mimir docker registry. Note that our existing integration test suite only uses the amd64 images, which means we cannot make any functional or performance guarantees about the arm64 images.
Remote ruler mode for improved rule evaluation performance: We’ve added a remote mode for the Grafana Mimir ruler, in which the ruler delegates rule evaluation to the query-frontend rather than evaluating rules directly within the ruler process itself. This allows recording and alerting rules to benefit from the query parallelization techniques implemented in the query-frontend (like query sharding). Remote mode is considered experimental and is off by default. To enable, see remote ruler.
Per-tenant custom trackers for monitoring cardinality: In Grafana Mimir 2.0, we introduced a custom tracker feature that allows you to track the count of active series over time that match a specific label matcher. In Grafana Mimir 2.1, we’ve made it possible to configure custom trackers via the runtime configuration file. This means you can now define different trackers for each tenant in your cluster and modify those trackers without an ingester restart.
Reduce cardinality of Grafana Mimir’s /metrics endpoint: While Grafana Mimir does a good job of exposing a relatively small number of series about its own state, this number can tick up when running Grafana Mimir clusters with high tenant counts or high active series counts. To reduce this number (and the accompanying cost of scraping and storing these time series), we made several optimizations which decreased series count on the /metrics endpoint by more than 10%.

Upgrade considerations

We’ve updated the default values for 2 parameters in Grafana Mimir to give users better out-of-the-box performance:

We’ve changed the default for -blocks-storage.tsdb.isolation-enabled from true to false. We’ve marked this flag as deprecated and will remove it completely in 2 releases. TSDB isolation is a feature inherited from Prometheus that didn’t provide any benefit given Grafana Mimir’s distributed architecture and in our 1 billion series load test we found it actually hurt performance. Disabling it reduced our ingester 99th percentile latency by 90%.
The store-gateway attributes cache is now enabled by default (achieved by updating the default for -blocks-storage.bucket-store.chunks-cache.attributes-in-memory-max-items from 0 to 50000). This in-memory cache makes it faster to look up object attributes for chunk data. We’ve been running this optional cache internally for a while and upon a recent configuration audit, realized it made sense to do the same for all users. The increase in store-gateway memory utilization from enabling this cache is negligible and easily justified given the performance gains.
As part of the enhancement to make active series custom trackers configurable on a per-tenant basis, we’ve moved where they get configured. Users of this feature should migrate from defining active_series_custom_trackers_config in the ingester section of Grafana Mimir’s YAML configuration to defining it in the limits section. Both options are supported for now to give users time to make the change, but configuring active_series_custom_trackers_config via the ingester section has been marked deprecated and will be removed in Mimir 2.3. Note: Configuring active_series_custom_trackers_config in both the limits and ingester sections will cause Grafana Mimir to fail on startup.

Bug fixes

2.1.0 bug fixes

PR 1704: Fixed a bug that previously caused Grafana Mimir to crash on startup when trying to run in monolithic mode with the results cache enabled due to duplicate metric names.
PR 1835: Fixed a bug that caused Grafana Mimir to crash when an invalid Alertmanager configuration was set even though the Alertmanager component was disabled. After this fix, the Alertmanager configuration is only validated if the Alertmanager component is loaded.
PR 1836: The ability to run Alertmanager with local storage broke in Grafana Mimir 2.0 when we removed the ability to run the Alertmanager without sharding. With this bugfix, we’ve made it possible to again run Alertmanager with local storage. However, for production use, we still recommend using external store since this is needed to persist Alertmanager state (e.g. silences) between replicas.
PR 1715: Restored Grafana Mimir’s ability to use CNAME DNS records to reach memcached servers. The bug was inherited from an upstream change to Thanos; we contributed a fix to Thanos and subsequently updated our Thanos version.

Grafana Mimir version 2.2 release notes

Wed, 03 Jun 2026 09:01:40 +0200

Grafana Mimir version 2.2 release notes

Grafana Labs is excited to announce version 2.2 of Grafana Mimir, the most scalable, most performant open source time series database in the world.

The highlights that follow include the top features, enhancements, and bugfixes in this release. If you are upgrading from Grafana Mimir 2.1, there is upgrade-related information as well. For the complete list of changes, see the Changelog.

Features and enhancements

Support for ingesting out-of-order samples: Grafana Mimir includes new, experimental support for ingesting out-of-order samples. This support is configurable, and it allows you to set how far out-of-order Mimir accepts samples on a per-tenant basis. This feature still needs additional testing; we do not recommend using it in a production environment. For more information, see Configuring out-of-order samples ingestion
Improved error messages: The error messages that Mimir reports are more human readable, and the messages include error codes that are easily searchable. For error descriptions, see the Grafana Mimir runbooks’ Errors catalog.
Configurable prefix for object storage: Mimir can now store block data, rules, and alerts in one bucket, with each under its own user-defined prefix, rather than requiring one bucket for each. You can configure the storage prefix by using -<storage>.storage-prefix option for corresponding storage: ruler-storage, alertmanager-storage or blocks-storage.
Store-gateway performance optimization The store-gateway can now pre-populate the file system cache when memory-mapping index-header files. This avoids the store-gateway from appearing to be stuck while loading index-headers. This feature is experimental and disabled by default; enable it using the flag -blocks-storage.bucket-store.index-header.map-populate-enabled.
Faster ingester startup: Ingesters now replay their WALs (write ahead logs) about 50% faster, and they also re-join the ring sooner under some conditions.
Helm Chart improvements: The Mimir Helm chart is the best way to install Mimir on Kubernetes. As part of the Mimir 2.2 release, we’re also releasing version 3.0 of the Helm chart. Notable enhancements follow. For the full list of changes, see the Helm chart changelog.
- The Helm chart now supports OpenShift.
- The Helm chart can now easily deploy Grafana Agent in order to scrape metrics and logs from all Mimir pods, and ship them to a remote store, which makes it easier to monitor the health of your Mimir installation. For more information, see Collecting metrics and logs from Grafana Mimir.
- The Helm chart now enables multi-tenancy by default. This makes it easy for you to add tenants as you grow your cluster. You can take advantage of Mimir’s per-tenant quality-of-service features, which improves stability and resilience at high scale. To learn more about how multi-tenancy in Mimir works, see Grafana Mimir authorization and authentication. This change is backwards-compatible. To read about how we implemented this, see #2117.
- We have significantly improved the configuration experience for the Helm chart, and here are a few of the most salient changes:
  - We’ve added an extraEnvFrom capability to all Mimir services to enable you to inject secrets via environment variables.
  - We’ve made it possible to globally set environment variables and inject secrets across all pods in the chart using global.extraEnv and global.extraEnvFrom. Note that the memcached and minio pods are not included.
  - We’ve switched the default storage of the Mimir configuration from a Secret to a ConfigMap, which makes it easier to quickly see the differences between your Mimir configurations between upgrades. We especially like the Helm diff plugin for this purpose.
  - We’ve added a structuredConfig option, which allows you to overwrite specific key-value pairs in the mimir.config template, which saves you from having to maintain the entire mimir.config in your own values.yaml file.
  - We’ve added the ability to create global pod annotations. This unlocks the ability to trigger a restart of all services in response to a single event, such as the update of the secret containing Mimir’s storage credentials.
- We’ve set the chart to disable -ingester.ring.unregister-on-shutdown and -distributor.extend-writes, for a smoother upgrade experience. Rolling restarts of ingesters are now less likely to cause spikes in resource usage.
- We’ve improved the documentation for the Helm chart by adding a Get started with Mimir using the Helm chart guide.
- We’ve added a smoke test for your Mimir cluster to help catch errors immediately after you install or upgrade Mimir via the Helm chart.

Upgrade considerations

All deprecated API endpoints that are under /api/v1/rules* and /prometheus/rules* have now been removed from the ruler component in favor of identical endpoints that use the prefix /prometheus/config/v1/rules*.

In Grafana Mimir 2.2, we have updated default values and some parameters to give you a better out-of-the-box experience:

Message size limits for gRPC messages that are exchanged between internal Mimir components have increased to 100 MiB from 4 MiB. This helps to avoid internal server errors when pushing or querying large data.
The -blocks-storage.bucket-store.ignore-blocks-within parameter changed from 0 to 10h. The default value of -querier.query-store-after changed from 0 to 12h. For most-recent data, both changes improve query performance by querying only the ingesters, rather than object storage.
The option -querier.shuffle-sharding-ingesters-lookback-period has been deprecated. If you previously changed this option from its default of 0s, set -querier.shuffle-sharding-ingesters-enabled to true and specify the lookback period by setting the -querier.query-ingesters-within option.
The -memberlist.abort-if-join-fails parameter now defaults to false. When Mimir is using memberlist as the backend store for its hash ring, and it fails to join the memberlist cluster, Mimir no longer aborts startup by default.

If you have used a previous version of the Mimir Helm chart, you must address some of the chart’s breaking changes before upgrading to helm chart version 3.0. For a detailed information about how to do this, see the Upgrade the Grafana Mimir Helm chart from version 2.1 to 3.0 guide.

Bug fixes

PR 1883: Fixed a bug that caused the query-frontend and querier to crash when they received a user query with a special regular expression label matcher.
PR 1933: Fixed a bug in the ingester ring page, which showed incorrect status of entries in the ring.
PR 2090: Ruler in remote rule evaluation mode now applies the timeout correctly. Previously the ruler could get stuck forever, which halted rule evaluation.
PR 2036: Fixed panic at startup when Mimir is running in monolithic mode and query sharding is enabled.

Grafana Mimir version 2.3 release notes

Wed, 03 Jun 2026 09:01:40 +0200

Grafana Mimir version 2.3 release notes

Grafana Labs is excited to announce version 2.3 of Grafana Mimir, the most scalable, most performant open source time series database in the world.

The highlights that follow include the top features, enhancements, and bugfixes in this release. For the complete list of changes, see the changelog.

Note: If you are upgrading from Grafana Mimir 2.2, review the list of important changes that follow.

Features and enhancements

Ingest metrics in OpenTelemetry format: This release of Grafana Mimir introduces experimental support for ingesting metrics from the OpenTelemetry Collector’s otlphttp exporter. This adds a second ingestion option for users of the OTel Collector; Mimir was already compatible with the prometheusremotewrite exporter. For more information, please see Configure OTel Collector.
Tenant federation for metadata queries: Users with tenant federation enabled could already issue instant queries, range queries, and exemplar queries to multiple tenants at once and receive a single aggregated result. With Grafana Mimir 2.3, we’ve added tenant federation support to the /api/v1/metadata endpoint as well.
Simpler object storage configuration: Users can now configure block, alertmanager, and ruler storage all at once with the common YAML config option key (or -common.storage.* CLI flags). By centralizing your object storage configuration in one place, this enhancement makes configuration faster and less error prone. Users may still individually configure storage for each of these components if they desire. For more information, see the Common Configurations.
.deb and .rpm packages for Mimir: Starting with version 2.3, we’re publishing .deb and .rpm files for Grafana Mimir, which will make installing and running it on Debian or RedHat-based linux systems much easier. Thank you to community contributor wilfriedroset for your work to implement this!
Import historic data: Users can now backfill time series data from their existing Prometheus or Cortex installation into Mimir using mimirtool, making it possible to migrate to Grafana Mimir without losing your existing metrics data. This support is still considered experimental and does not yet work for data stored in Thanos. To learn more about this feature, please see mimirtool backfill and Configure TSDB block upload
Increased instant query performance: Grafana Mimir now supports splitting instant queries by time. This allows it to better parallelize execution of instant queries and therefore return results faster. At present, splitting is only supported for a subset of instant queries, which means not all instant queries will see a speedup. This feature is currently experimental and is disabled by default. It can be enabled with the split_instant_queries_by_interval YAML config option in the limits section (or the CLI flag -query-frontend.split-instant-queries-by-interval).

Helm chart improvements

The Mimir Helm chart is the best way to install Mimir on Kubernetes. As part of the Mimir 2.3 release, we’re also releasing version 3.1 of the Mimir Helm chart.

Notable enhancements follow. For the full list of changes, see the Helm chart changelog.

We’ve upgraded the MinIO subchart dependency from a deprecated chart to the supported one. This creates a breaking change in how the administrator password is set. However, as the built-in MinIO is not a recommended object store for production use cases, this change did not warrant a new major version of the Mimir Helm chart.
Query sharding is now enabled by default which should give you better performance on high cardinality metrics queries.
- To compensate for the increased number of queries generated by query sharding, the query scheduler component is now enabled by default.
The backfill API endpoints for importing historic time series data are now exposed on the Nginx gateway.
Nginx now sets the value of the X-Scope-OrgID header equal to the value of Mimir’s no_auth_tenant parameter by default. The previous release had set the value of X-Scope-OrgID to anonymous by default which complicated the process of migrating to Mimir.
Memberlist now uses DNS service-discovery by default, which decreases startup time for large Mimir clusters.

Important changes

In Grafana Mimir 2.3 we have removed the following previously deprecated configuration options:

The extend_writes parameter in the distributor YAML configuration and -distributor.extend-writes CLI flag have been removed.
The active_series_custom_trackers parameter has been removed from the YAML configuration. It had already been moved to the runtime configuration. See #1188 for details.
The blocks-storage.tsdb.isolation-enabled parameter in the YAML configuration and -blocks-storage.tsdb.isolation-enabled CLI flag have been removed.

With Grafana Mimir 2.3 we have also updated the default value for the CLI flag -distributor.ha-tracker.max-clusters to 100 to provide Denial-of-Service protection. Previously -distributor.ha-tracker.max-clusters was unlimited by default which could allow a tenant with HA Dedupe enabled to overload the HA tracker with __cluster__ label values that could cause the HA Dedupe database to fail.

Also, as noted above, the administrator password for Helm chart deployments using the built-in MinIO is now set differently.

Bug fixes

PR 2447: Fix incorrect mapping of http status codes 429 to 500 when the request queue is full in the query-frontend. This corrects behavior in the query-frontend where a retryable 429 "Too Many Outstanding Requests" error from a querier was incorrectly returned as an unretryable 500 system error.
PR 2505: The Memberlist key-value (KV) store now tries to “fast-join” the cluster to avoid serving an empty KV store. This fix addresses the confusing “empty ring” error response and the error log message “ring doesn’t exist in KV store yet” emitted by services when there are other members present in the ring when a service starts. Those using other key-value store options (e.g., consul, etcd) are not impacted by this bug.
PR 2289: The “List Prometheus rules” API endpoint of the Mimir Ruler component is no longer blocked while rules are being synced. This means users can now list rules while syncing larger rule sets.

Grafana Mimir version 2.4 release notes

Wed, 03 Jun 2026 09:01:40 +0200

Grafana Mimir version 2.4 release notes

Grafana Labs is excited to announce version 2.4 of Grafana Mimir.

The highlights that follow include the top features, enhancements, and bugfixes in this release. For the complete list of changes, see the changelog.

Note: If you are upgrading from Grafana Mimir 2.3, review the list of important changes that follow.

Features and enhancements

Query-scheduler ring-based service discovery: The query-scheduler is an optional, stateless component that retains a queue of queries to execute, and distributes the workload to available queriers. The use the query-scheduler, query-frontends and queriers are required to discover the addresses of the query-scheduler instances.

In addition to DNS-based service discovery, Mimir 2.4 introduces the ring-based service discovery for the query-scheduler. When enabled, the query-schedulers join their own hash ring (similar to other Mimir components), and the query-frontends and queriers discover query-scheduler instances via the ring.

Ring-based service discovery makes it easier to set up the query-scheduler in environments where you can’t easily define a DNS entry that resolves to the running query-scheduler instances. For more information, refer to query-scheduler configuration.
New API endpoint exposes per-tenant limits: Mimir 2.4 introduces a new API endpoint, which is available on all Mimir components that load the runtime configuration. The endpoint exposes the limits of the authenticated tenant. You can use this new API endpoint when developing custom integrations with Mimir that require looking up the actual limits that are applied on a given tenant. For more information, refer to Get tenant limits.
New TLS configuration options: Mimir 2.4 introduces new options to configure the accepted TLS cipher suites, and the minimum versions for the HTTP and gRPC clients that are used between Mimir components, or by Mimir to communicate to external services such as Consul or etcd.

You can use these new configuration options to override the default TLS settings and meet your security policy requirements. For more information, refer to Securing Grafana Mimir communications with TLS.
Maximum range query length limit: Mimir 2.4 introduces the new configuration option -query-frontend.max-total-query-length to limit the maximum range query length, which is computed as the query’s end minus start timestamp. This limit is enforced in the query-frontend and defaults to -store.max-query-length if unset.

The new configuration option allows you to set different limits between the received query maximum length (-query-frontend.max-total-query-length) and the maximum length of partial queries after splitting and sharding (-store.max-query-length).

The following experimental features have been promoted to stable:

Helm chart improvements

The mimir-distributed Helm chart is the best way to install Mimir on Kubernetes. As part of the Mimir 2.4 release, we’re also releasing version 3.2 of the mimir-distributed Helm chart.

Notable enhancements follow. For the full list of changes, see the Helm chart changelog.

Added support for topologySpreadContraints.
Replaced the default anti-affinity rules with topologySpreadContraints for all components which puts less restrictions on where Kubernetes can run pods.

Important: if you are not using the sizing plans (small.yaml, large.yaml, capped-small.yaml, capped-large.yaml) in production, you must reintroduce pod affinity rules for the ingester and store-gateway. This also fixes a missing label selector for the ingester. Merge the following with your custom values file:

ingester:
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        - labelSelector:
            matchExpressions:
              - key: target
                operator: In
                values:
                  - ingester
          topologyKey: "kubernetes.io/hostname"
        - labelSelector:
            matchExpressions:
              - key: app.kubernetes.io/component
                operator: In
                values:
                  - ingester
          topologyKey: "kubernetes.io/hostname"
store_gateway:
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        - labelSelector:
            matchExpressions:
              - key: target
                operator: In
                values:
                  - store-gateway
          topologyKey: "kubernetes.io/hostname"
        - labelSelector:
            matchExpressions:
              - key: app.kubernetes.io/component
                operator: In
                values:
                  - store-gateway
          topologyKey: "kubernetes.io/hostname"

Updated the anti affinity rules in the sizing plans (small.yaml, large.yaml, capped-small.yaml, capped-large.yaml). The sizing plans now enforce that no two pods of the ingester, store-gateway, or alertmanager StatefulSets are scheduled on the same Node. Pods from different StatefulSets can share a Node.
Support for Openshift Route resource for nginx has been added.

Important changes

In Grafana Mimir 2.4, the default values of the following configuration options have changed:

-distributor.remote-timeout has changed from 20s to 2s.
-distributor.forwarding.request-timeout has changed from 10s to 2s.
-blocks-storage.tsdb.head-compaction-concurrency has changed from 5 to 1.
The hash-ring heartbeat period for distributors, ingesters, rulers, and compactors has increased from 5s to 15s.

In Grafana Mimir 2.4, the following deprecated configuration options have been removed:

The YAML configuration option limits.active_series_custom_trackers_config.
The CLI flag -ingester.ring.join-after and its respective YAML configuration option ingester.ring.join_after.
The CLI flag -querier.shuffle-sharding-ingesters-lookback-period and its respective YAML configuration option querier.shuffle_sharding_ingesters_lookback_period.

With Grafana Mimir 2.4, the anonymous usage statistics tracking is enabled by default. Mimir maintainers use this anonymous information to learn more about how the open source community runs Mimir and what the Mimir team should focus on when working on the next features and documentation improvements. If possible, we ask you to keep the usage reporting feature enabled. In case you want to opt-out from anonymous usage statistics reporting, refer to Disable the anonymous usage statistics reporting.

Bug fixes

PR 2979: Fix remote write HTTP response status code returned by Mimir when failing to write only to one ingester (the quorum is still honored when running Mimir with the default replication factor of 3) and some series are not ingested because of validation errors or some limits being reached.
PR 3005: Fix the querier to re-balance its workers connections when a query-frontend or query-scheduler instance is terminated.
PR 2963: Fix the remote read endpoint to correctly support the Accept-Encoding: snappy HTTP request header.

Grafana Mimir version 2.5 release notes

Wed, 03 Jun 2026 09:01:40 +0200

Grafana Mimir version 2.5 release notes

Grafana Labs is excited to announce version 2.5 of Grafana Mimir.

The highlights that follow include the top features, enhancements, and bugfixes in this release. For the complete list of changes, see the changelog.

Features and enhancements

Alertmanager Discord support Alertmanager can now be configured to send alerts in Discord channels.
Configurable TLS minimum version and cipher suites We added the flags -server.tls-min-version and -server.tls-cipher-suites that can be used to define the minimum TLS version and the supported cipher suites in all HTTP and gRPC servers in Mimir.
Lower memory usage in store-gateway, ingester and alertmanager We made various changes related to how index lookups are performed and how the active series custom trackers are implemented, which results in better performance and lower overall memory usage in the store-gateway and ingester. We also optimized the alertmanager, which results in a 50% reduction in memory usage in use cases with larger numbers of tenants.
Improved Mimir dashboards We added two new dashboards named Mimir / Overview resources and Mimir / Overview networking. Furthermore, we have made various improvements to the following existing dashboards:
- Mimir / Overview: Add “remote read”, “metadata”, and “exemplar” queries.
- Mimir / Writes: Add optional row about the distributor’s new forwarding feature.
- Mimir / Tenants: Add insights into the read path.

Helm chart improvements

Zone aware replication Helm now supports deploying the ingesters and store-gateways as different availability zones. The replication is also zone-aware, therefore multiple instances of one zone can fail without any service interruption and roll outs can be performed faster because many instances of each zone can be restarted together, as opposed to them all restarting in sequence.

This is a breaking change, for details on how to upgrade please review the Helm changelog.
Running without root privileges All Mimir, GEM and Agent processes now don’t require root privileges to run anymore.
Unified reverse proxy (gateway) configuration for Mimir and GEM This change allows for an easier upgrade path from Mimir to GEM, without any downtime. The unified configuration also makes it possible to autoscale the GEM gateway pods and it supports OpenShift Route. The change also deprecates the nginx section in the configuration. The section will be removed in release 7.0.0.
Updated MinIO The MinIO sub-chart was updated from 4.x to 5.0.0, note that this update inherits a breaking change because the MinIO gateway mode was removed.
Updated sizing plans We updated our sizing plans to make them reflect better how we recommend running Mimir and GEM in production. Note that this includes a breaking change for users of the “small” plan, more details can be found in the Helm changelog.
Various quality of life improvements
- Rollout strategies without downtime
- Read path and compactor configuration refresh, providing better default settings
- OTLP ingestion support in the Nginx configuration
- A default configuration for alertmanager, so the user interface and the sending of alerts from the ruler works out of the box

Bug fixes

Flusher: Added Overrides as a dependency to prevent panics when starting with -target=flusher. PR 3151
Query-frontend: properly close gRPC streams to the query-scheduler to stop memory and goroutines leak. PR 3302
Ruler: persist evaluation delay configured in the rulegroup. PR 3392
Fix panics in OTLP ingest path when parse errors occur. PR 3538

Grafana Mimir version 2.6 release notes

Wed, 03 Jun 2026 09:01:40 +0200

Grafana Mimir version 2.6 release notes

Grafana Labs is excited to announce version 2.6 of Grafana Mimir.

The highlights that follow include the top features, enhancements, and bugfixes in this release. For the complete list of changes, see the changelog.

Features and enhancements

Lower memory usage in store-gateway by streaming series results The store-gateway can now stream results back to the querier instead of buffering them. This is expected to greatly reduce peak memory consumption while keeping latency the same. This is still an experimental feature but Grafana Labs is already running it in production and there’s no known issue. This feature can be enabled setting the -blocks-storage.bucket-store.batch-series-size configuration option (if you want to try it out, we recommend you setting to 5000).
Improved stability in store-gateway by removing mmap usage The store-gateway can now use an alternate code path to read index-headers that does not use memory mapped files. This is expected to improve stability of the store-gateway. This is still an experimental feature but Grafana Labs is already running it in production and there’s no known issue. This feature can be enabled setting -blocks-storage.bucket-store.index-header.stream-reader-enabled=true.

Alertmanager improvements

Webex support Alertmanager can now use Webex to send alerts.
tenantID template function A new template function tenantID, returning the ID of the tenant owning the alert, has been added.
grafanaExploreURL template function A new template function grafanaExploreURL, returning the URL to the Grafana explore page with range query, has been added.

Helm chart improvements

The Grafana Mimir and Grafana Enterprise Metrics Helm chart is now released independently. See the corresponding documentation for more information.

Important changes

In Grafana Mimir 2.6 we have removed the following previously deprecated or experimental configuration options:

The CLI flag -blocks-storage.bucket-store.max-concurrent-reject-over-limit and its respective YAML configuration option blocks_storage.bucket_store.max_concurrent_reject_over_limit.
The CLI flag -query-frontend.align-querier-with-step and its respective YAML configuration option frontend.align_querier_with_step.

The following configuration options are deprecated and will be removed in Grafana Mimir 2.8:

The CLI flag -store.max-query-length and its respective YAML configuration option limits.max_query_length have been replaced with -querier.max-partial-query-length and limits.max_partial_query_length.

The following experimental options and features are now stable:

The CLI flag -query-frontend.max-total-query-length and its respective YAML configuration option limits.max_total_query_length.
The CLI flags -distributor.request-rate-limit and -distributor.request-burst-limit and their respective YAML configuration options limits.request_rate_limit and limits.request_rate_burst.
The CLI flag -ingester.max-global-exemplars-per-user and its respective YAML configuration option limits.max_global_exemplars_per_user.
The CLI flag -ingester.tsdb-config-update-period its respective YAML configuration option ingester.tsdb_config_update_period.
The API endpoint /api/v1/query_exemplars.

Bug fixes

Alertmanager: Fix template spurious deletion with relative data dir. PR 3604
Security: Update prometheus/exporter-toolkit for CVE-2022-46146. PR 3675
Security: Update golang.org/x/net for CVE-2022-41717. PR 3755
Debian package: Fix post-install, environment file path and user creation. PR 3720
Memberlist: Fix panic during Mimir startup when Mimir receives gossip message before it’s ready. PR 3746
Update github.com/thanos-io/objstore to address issue with Multipart PUT on s3-compatible Object Storage. PR 3802 PR 3821
Querier: Canceled requests are no longer reported as “consistency check” failures. PR 3837 PR 3927
Distributor: Don’t panic when metric_relabel_configs in overrides contains null element. PR 3868
Ingester, Compactor: Fix panic that can occur when compaction fails. PR 3955

2.6.1

Security: updated the Go version to 1.20.3 to fix CVE-2023-24538. PR 4798

2.6.2

Security: updated Go to version 1.20.4 to fix CVE-2023-24539, CVE-2023-24540, CVE-2023-29400. PR 4903

Grafana Mimir version 2.7 release notes

Wed, 03 Jun 2026 09:01:40 +0200

Grafana Mimir version 2.7 release notes

Grafana Labs is excited to announce version 2.7 of Grafana Mimir.

The highlights that follow include the top features, enhancements, and bugfixes in this release. For the complete list of changes, see the changelog.

Features and enhancements

Store-gateway streaming enabled by default The new default value of 5000 for -blocks-storage.bucket-store.batch-series-size enables store-gateway streaming in the default configuration. This means that series are loaded from object storage in batches rather than buffering them all in memory before returning to the querier. Enabling streaming can reduce memory utilization peaks in the store-gateway.
Store-gateway index header reader no longer uses mmap by default Along with streaming enabled in the store-gateway, this change contributes to more efficient memory usage. See the Important changes section for more details.
Support for keep_firing_for option to ruler configuration This new option determines the amount of time an alert should keep firing while the ruler expression doesn’t return results.
More efficient chunks fetching and caching Enable with the new experimental feature flag -blocks-storage.bucket-store.chunks-cache.fine-grained-chunks-caching-enabled=true. This should reduce CPU, memory utilization, and receive bandwidth of a store-gateway.
Experimental query sharding improvements: A new configuration parameter, -query-frontend.query-sharding-target-series-per-shard, allows query sharding to take into account cardinality of similar requests executed previously when computing the maximum number of shards to use. If you want to try it out, we recommend starting with a value of 2500.
Experimental support for native histogram ingestion: Native histograms can now be ingested. The new per-tenant limit -ingester.native-histograms-ingestion-enabled controls whether native histograms are stored or ignored. The support for querying native histograms is not complete yet and it’s expected to be available in the next release.

Alertmanager improvements

New metrics The following upstream metrics are now exposed:
- cortex_alertmanager_dispatcher_aggregation_groups
- cortex_alertmanager_dispatcher_alert_processing_duration_seconds

Helm chart improvements

The Grafana Mimir and Grafana Enterprise Metrics Helm chart is now released independently. See the Grafana Mimir Helm chart documentation.

Important changes

In Grafana Mimir 2.7, the default vaules of the following configuration options have changed:

-blocks-storage.bucket-store.batch-series-size is now enabled by default with a value of 5000.
-ruler.evaluation-delay-duration has changed from 0 to 1m.

In Grafana Mimir 2.7, the following configuration options are now deprecated:

-blocks-storage.bucket-store.chunks-cache.subrange-size since there’s no benefit to changing the default of 16000
-blocks-storage.bucket-store.consistency-delay has been deprecated and will be removed in Mimir 2.9.
-compactor.consistency-delay has been deprecated and will be removed in Mimir 2.9.
-ingester.ring.readiness-check-ring-health has been deprecated and will be removed in Mimir 2.9.

In Grafana Mimir 2.7, the following options, metrics, and labels have been removed:

Experimental support for ephemeral storage introduced in Mimir 2.6.0 has been removed.
- Following options are no longer available:
  - -blocks-storage.ephemeral-tsdb.*
  - -distributor.ephemeral-series-enabled
  - -distributor.ephemeral-series-matchers
  - -ingester.max-ephemeral-series-per-user
  - -ingester.instance-limits.max-ephemeral-series
- The following metrics have been removed:
  - cortex_ingester_ephemeral_series
  - cortex_ingester_ephemeral_series_created_total
  - cortex_ingester_ephemeral_series_removed_total
  - cortex_ingester_ingested_ephemeral_samples_total
  - cortex_ingester_ingested_ephemeral_samples_failures_total
  - cortex_ingester_memory_ephemeral_users
  - cortex_ingester_queries_ephemeral_total
  - cortex_ingester_queried_ephemeral_samples
  - cortex_ingester_queried_ephemeral_series
- Additionally, querying using the {__mimir_storage__="ephemeral"} selector no longer works. All label values with the ephemeral- prefix within the reason label of the cortex_discarded_samples_total metric are no longer available.
The store-gateway default index header reader no longer uses mmap and the mmap-based index header reader has been removed. The following flags have been changed:
- -blocks-storage.bucket-store.index-header.map-populate-enabled has been removed
- -blocks-storage.bucket-store.index-header.stream-reader-enabled has been removed
- -blocks-storage.bucket-store.index-header.stream-reader-max-idle-file-handles has been renamed to -blocks-storage.bucket-store.index-header.max-idle-file-handles, and the corresponding configuration file option has been renamed from stream_reader_max_idle_file_handles to max_idle_file_handles

Bug fixes

Store-gateway: return Canceled rather than Aborted or Internal error when the calling querier cancels a label names or values request, and return Internal if processing the request fails for another reason. PR 4061
Querier: track canceled requests with status code 499 in the metrics instead of 503 or 422. PR 4099
Ingester: compact out-of-order data during /ingester/flush or when TSDB is idle. PR 4180
Ingester: conversion of global limits max-series-per-user, max-series-per-metric, max-metadata-per-user and max-metadata-per-metric into corresponding local limits now takes into account the number of ingesters in each zone. PR 4238
Ingester: track cortex_ingester_memory_series metric consistently with cortex_ingester_memory_series_created_total and cortex_ingester_memory_series_removed_total. PR 4312
Querier: fixed a bug which was incorrectly matching series with regular expression label matchers with begin/end anchors in the middle of the regular expression. PR 4340
Security: updated the Go version to 1.20.3 to fix CVE-2023-24538. PR 4795
Security: updated the Go version to 1.20.4 to fix CVE-2023-24539, CVE-2023-24540, CVE-2023-29400. PR 4905

Grafana Mimir version 2.8 release notes

Wed, 03 Jun 2026 09:01:40 +0200

Grafana Mimir version 2.8 release notes

Grafana Labs is excited to announce version 2.8 of Grafana Mimir.

The highlights that follow include the top features, enhancements, and bugfixes in this release. For the complete list of changes, see the changelog.

Features and enhancements

Experimental support for using Redis as cache Mimir now can use Redis for caching results, chunks, index and metadata.
Experimental support for fetching secret from Vault for TLS configuration.
Experimental support for querying native histograms. This support is not finalized as the related Prometheus API is also experimental, thus the exact behavior might change in future releases.
Query-frontend and ruler now use protobuf internal query result payload format by default This reduces the CPU and memory utilisation of the querier, query-frontend and ruler, as well as reducing network bandwidth consumed between these components.
Query-frontend cached results now contain timestamp This allows Mimir to check if cached results are still valid based on current TTL configured for tenant. Results cached by previous Mimir version are used until they expire from cache, which can take up to 7 days. If you need to use per-tenant TTL sooner, please flush results cache manually.
Optimized regular expression label matchers This reduces CPU utilisation in ingesters and store-gateways when running queries containing regular expression label matchers.
Store-gateway now use streaming for LabelNames RPC This improves memory utilization in store-gateway when calling LabelNames RPC.

Helm chart improvements

The Grafana Mimir and Grafana Enterprise Metrics Helm chart is now released independently. See the Grafana Mimir Helm chart documentation.

Important changes

In Grafana Mimir 2.8 we have removed the following previously deprecated or experimental metrics:

cortex_bucket_store_series_get_all_duration_seconds
cortex_bucket_store_series_merge_duration_seconds
cortex_ingester_tsdb_wal_replay_duration_seconds

The following configuration options are deprecated and will be removed in Grafana Mimir 2.10:

The CLI flag -blocks-storage.tsdb.max-tsdb-opening-concurrency-on-startup and its respective YAML configuration option tsdb.max_tsdb_opening_concurrency_on_startup.

The following configuration options that were deprecated in 2.6 are removed:

The CLI flag -store.max-query-length and its respective YAML configuration option limits.max_query_length.

The following configuration options that were deprecated in 2.5 are removed:

The CLI flag -azure.msi-resource.

The following experimental options and features are now stable:

The protobuf internal query result payload format, which is now enabled by default

We changed default value of block storage retention period. The default value for -blocks-storage.tsdb.retention-period was 24h and now is 13h

Bug fixes

Querier: Streaming remote read will now continue to return multiple chunks per frame after the first frame. PR 4423
Query-frontend: don’t retry queries which error inside PromQL. PR 4643
Store-gateway & query-frontend: report more consistent statistics for fetched index bytes. PR 4671

Grafana Mimir version 2.9 release notes

Wed, 03 Jun 2026 09:01:40 +0200

Grafana Mimir version 2.9 release notes

Grafana Labs is excited to announce version 2.9 of Grafana Mimir.

The highlights that follow include the top features, enhancements, and bugfixes in this release. For the complete list of changes, see the changelog.

Features and enhancements

Reduced store-gateway memory utilization on fetching series from long-term storage For queries that include broad label matchers (e.g. datacenter="dc1"), Mimir 2.9 will fetch a reduced volume of index data, which leads to a significant reduction in memory allocations in the store-gateway.
Reduced CPU utilisation for some shuffle sharding scenarios Mimir queriers will now use significantly less CPU in cases where shuffle sharding is enabled for tenants with a shard size that’s large but lower than the total number of ingesters.
Reduced object storage API calls in compactors and rulers Mimir 2.9 comes with optimizations that will reduce the amount of times compactors and rulers need to access rules stored in object storage.
- This release adds experimental support for a ruler storage cache. This cache should reduce the number of “list objects” API calls issued to the object storage when there are 2+ ruler replicas running in a Mimir cluster. The cache can be configured by setting the -ruler-storage.cache.* CLI flags or their respective YAML config options.
- We also introduced a new feature to trigger a synchronization of tenant’s rule groups as soon as changes to the rule configuration are made via API. This synchronization is in addition of the periodic syncing done every -ruler.poll-interval, which has then been relaxed from every 1m to every 10m. The new behaviour is enabled globally by default but can be disabled with -ruler.sync-rules-on-changes-enabled=false or tuned at a per-tenant level.
Experimental support for streaming chunks from ingester to querier This is expected to greatly reduce querier memory consumption when evaluating queries that select a large number of series, because chunks streamed from the querier can now be read into memory as needed.

Helm chart improvements

The Grafana Mimir and Grafana Enterprise Metrics Helm chart is now released independently. See the Grafana Mimir Helm chart documentation.

Important changes

In Grafana Mimir 2.9 we have removed the following previously deprecated or experimental metrics:

cortex_bucket_store_chunk_pool_requested_bytes_total
cortex_bucket_store_chunk_pool_returned_bytes_total

The following configuration options are deprecated and will be removed in Grafana Mimir 2.11:

The CLI flag -querier.query-ingesters-within. This configuration is moved to per-tenant overrides.
The CLI flag -blocks-storage.bucket-store.bucket-index.enabled.
The CLI flags -blocks-storage.bucket-store.chunk-pool-min-bucket-size-bytes, -blocks-storage.bucket-store.chunk-pool-max-bucket-size-bytes and -blocks-storage.bucket-store.max-chunk-pool-bytes.
The CLI flags -querier.iterators and -query.batch-iterators.

The following configuration options that were deprecated in 2.7 are removed:

The CLI flag -blocks-storage.bucket-store.chunks-cache.subrange-size. A fixed value of 16000 is now always used.
The CLI flag -blocks-storage.bucket-store.consistency-delay.
The CLI flag -compactor.consistency-delay.
The CLI flag -ingester.ring.readiness-check-ring-health.

The following experimental options and features are now stable:

The CLI flag -query-frontend.query-sharding-max-regexp-size-bytes.
The CLI flag -query-scheduler.max-used-instances.
The CLI flags -(alertmanager|blocks|ruler)-storage.storage-prefix.
The CLI flag -compactor.first-level-compaction-wait-period.
The CLI flags -usage-stats.enabled and -usage-stats.installation-mode.
The CLI flag -query-frontend.query-sharding-target-series-per-shard.

The following configuration option defaults were changed:

The default value for the CLI flag -query-frontend.query-sharding-max-regexp-size-bytes was changed from 0 to 4096. As a result, queries with regex matchers exceeding this limit will not be sharded by default.
The default value for the CLI flag -compactor.partial-block-deletion-delay was changed from 0s to 1d. As a result, partial blocks resulting from a failed block upload or deletion will be cleaned up automatically.
The default value for the CLI flag -ruler.poll-interval was changed from 1m to 10m.

Bug fixes

Store-gateway: Detect collisions in the postings cache. PR 4770
Store-gateway: Fix panic caused by cached LabelValues responses with more than 655360 values. PR 5021

2.9.4

Security: updated the Alpine base image version to 3.18.5 to address CVE-2023-5363. PR 6895

Grafana Mimir version 2.10 release notes

Wed, 03 Jun 2026 09:01:40 +0200

Grafana Mimir version 2.10 release notes

Grafana Labs is excited to announce version 2.10 of Grafana Mimir.

The highlights that follow include the top features, enhancements, and bugfixes in this release. For the complete list of changes, see the changelog.

Features and enhancements

Added support for rule filtering by passing file, ruler_group and rule_name parameters to the ruler endpoint /api/v1/rules.
Added support to only count series that are considered active through the Cardinality API endpoint /api/v1/cardinality/label_values by passing the count_method parameter. You can set it to active to count only series that are considered active according to the -ingester.active-series-metrics-idle-timeout flag setting rather than counting all in-memory series.
Reduced the overall memory consumption by changing the internal data structure for labels. Expect ingesters to use around 15% less memory with this change, depending on the pattern of labels used, number of tenants, etc.
Reduced the memory usage of the Active Series Tracker in the ingester.
Added a buffered logging implementation that can be enabled through the -log.buffered CLI flag. This should reduce contention and resource usage under heavy usage patterns.
Improved the performance of the OTLP ingestion and more detailed information was added to the traces in order to make troubleshooting problems easier.
Improved the performance of series matching in the store-gateway by always including the __name__ posting group causing a reduction in the number of object storage API calls.
Improved the performance of label values with matchers calls when number of matched series is small. If you’re using Grafana to query Grafana Mimir, you’ll need to be sure your Prometheus data source configuration has the Prometheus type set to Mimir and the Version set correctly in order to benefit from this improvement.
Support to cache cardinality, label names and label values query responses in query frontend. The cache will be used when -query-frontend.cache-results is enabled, and -query-frontend.results-cache-ttl-for-cardinality-query or -query-frontend.results-cache-ttl-for-labels-query is set to a value greater than 0.
Reduced wasted effort spent computing results that won’t be used by having queriers cancel the requests sent to the ingesters in a zone upon receiving first error from that zone.
Reduced object storage use by enhancing the compactor to remove the bucket index, markers, and debug files when it detects zero remaining blocks in the bucket index. This cleanup process can be enabled by setting the -compactor.no-blocks-file-cleanup-enabled option to true.
Added new debug HTTP endpoints /ingester/tenants and /ingester/tsdb/{tenant} to the ingester that provide debug information about tenants and their TSDBs.
Added new metrics for tracking native histograms in active series: cortex_ingester_active_native_histogram_series, cortex_ingester_active_native_histogram_series_custom_tracker, cortex_ingester_active_native_histogram_buckets, cortex_ingester_active_native_histogram_buckets_custom_tracker. The first 2 are the subsets of the existing and unmodified cortex_ingester_active_series and cortex_ingester_active_series_custom_tracker respectively, only tracking native histogram series, and the last 2 are the equivalent for tracking the number of buckets in native histogram series.

Additionally, the following previously experimental features are now considered stable:

Support for a ruler storage cache. This cache should reduce the number of “list objects” API calls issued to the object storage when there are 2+ ruler replicas running in a Mimir cluster. The cache can be configured by setting the -ruler-storage.cache.* CLI flags or their respective YAML config options.
Query sharding cardinality estimation. This feature allows query sharding to take into account cardinality of similar requests executed previously when computing the maximum number of shards to use. You can enable it through the advanced CLI configuration flag -query-frontend.query-sharding-target-series-per-shard; we recommend starting with a value of 2500.
Query expression size limit. You can limit the size in bytes of the queries allowed to be processed through the CLI configuration flag -query-frontend.max-query-expression-size-bytes.
Peer discovery / tenant sharding for overrides exporters. You can enable it through the CLI configuration flag -overrides-exporter.ring.enabled.
Overrides exporter enabled metrics selection. You can select which metrics the overrides exporter should export through the CLI configuration flag -overrides-exporter.enabled-metrics.
Per-tenant results cache TTL. The time-to-live duration for cached query results can be configured using the results_cache_ttl and results_cache_ttl_for_out_of_order_time_window parameters.

Experimental features

Grafana Mimir 2.10 includes new features that are considered as experimental and disabled by default. Please use them with caution and report any issues you encounter:

Support for ingesting exponential histograms in OpenTelemetry format. The exponential histograms that are over the native histogram scale limit of 8 are downscaled to allow their ingestion.
Store-gateway index-header loading improvements, which include the ability to persist the sparse index-header to disk instead of reconstructing it on every restart (-blocks-storage.bucket-store.index-header-sparse-persistence-enabled) as well as the ability to persist the list of block IDs that were lazy-loaded while running to eagerly load them upon startup to prevent starting up with no loaded blocks (-blocks-storage.bucket-store.index-header.eager-loading-startup-enabled) and an option to limit the number of concurrent index-header loads when lazy-loading (-blocks-storage.bucket-store.index-header-lazy-loading-concurrency).
Option to allow queriers to reduce pressure on ingesters by initially querying only the minimum set of ingesters required to reach quorum. (-querier.minimize-ingester-requests).
Early TSDB Head compaction in the ingesters to reduce in-memory series when a certain threshold is reached. Useful to deal with high series churning rate. (-blocks-storage.tsdb.early-head-compaction-min-in-memory-series).
Spread-minimizing token generation algorithm for the ingesters. This new method drastically reduces the difference in series pushed to different ingesters. Please note that a migration process is required to switch from previous random generation algorithm, which will be detailed once the feature is declared stable.
Support for chunks streaming from store-gateways to queriers that should reduce the memory usage in the queriers. Can be enabled through the -querier.prefer-streaming-chunks-from-store-gateways option.
Support for circuit-breaking the distributor write requests to the ingesters. This can be enabled through the -ingester.client.circuit-breaker.* configuration options and should serve to let ingesters recover when under high pressure.
Support to limit read requests based on CPU/memory utilization. This should alleviate pressure on the ingesters after receiving heavy queries and reduce the likelihood of disrupting the write path. (-ingester.read-path-cpu-utilization-limit, -ingester.read-path-memory-utilization-limit, -ingester.log-utilization-based-limiter-cpu-samples).

Helm chart improvements

The Grafana Mimir and Grafana Enterprise Metrics Helm chart is now released independently. See the Grafana Mimir Helm chart documentation.

Important changes

In Grafana Mimir 2.10 we have changed the following behaviors:

Query requests are initiated only to ingesters in the ACTIVE state in the ring. This is not expected to introduce any degradation in terms of query results correctness or high-availability.
Per-instance limit errors are not logged anymore, to reduce resource usage when ingesters are under pressure. We encourage you to use metrics and alerting to monitor them instead. The following metrics have been added to count the number of requests rejected for hitting per-instance limits:
- cortex_distributor_instance_rejected_requests_total
- cortex_ingester_instance_rejected_requests_total
The CLI flag -validation.create-grace-period is now enforced in the ingester. If you’ve configured -validation.create-grace-period, make sure the configuration is applied to ingesters too.
The CLI flag -validation.create-grace-period is now enforced for exemplars. The cortex_discarded_exemplars_total{reason="exemplar_too_far_in_future",user="..."} series is incremented when exemplars are dropped because their timestamp is greater than “now + grace_period”.
The CLI flag -validation.create-grace-period is now enforced in the query-frontend even when the configured value is 0. When the value is 0, the query end time range is truncated to the current real-world time.

The following metrics were removed:

cortex_ingester_shipper_dir_syncs_total
cortex_ingester_shipper_dir_sync_failures_total

The following configuration options are deprecated and will be removed in Grafana Mimir 2.12:

The CLI flag -blocks-storage.bucket-store.index-header-lazy-loading-enabled is deprecated, use the new configuration -blocks-storage.bucket-store.index-header.lazy-loading-enabled.
The CLI flag -blocks-storage.bucket-store.index-header-lazy-loading-idle-timeout is deprecated, use the new configuration -blocks-storage.bucket-store.index-header.lazy-loading-idle-timeout.
The CLI flag -blocks-storage.bucket-store.index-header-lazy-loading-concurrency is deprecated, use the new configuration -blocks-storage.bucket-store.index-header.lazy-loading-concurrency.

The following configuration options that were deprecated in Grafana Mimir 2.8 are removed:

The CLI flag blocks-storage.tsdb.max-tsdb-opening-concurrency-on-startup.

The following experimental configuration options were renamed or removed:

The CLI flag -querier.prefer-streaming-chunks was renamed to -querier.prefer-streaming-chunks-from-ingesters.
The CLI flag -blocks-storage.bucket-store.chunks-cache.fine-grained-chunks-caching-enabled was removed.
The CLI flag -blocks-storage.bucket-store.fine-grained-chunks-caching-ranges-per-series was removed.

The following experimental options are now stable:

The CLI flag -shutdown-delay.
The CLI flag -ingester.ring.excluded-zones.

The following configuration option defaults were changed:

The default value for the CLI flag -querier.streaming-chunks-per-ingester-buffer-size was changed from 512 to 256.
The default value for gRPC clients connect timeout was set to 5s (default inherited from gRPC client was 20s) with a default max backoff delay of 5s (default inherited from gRPC client was 120s).

Bug fixes

Ruler: fixed graceful shutdown for rule evaluations.
Ingester: fixed ingesters getting stuck when previous state is LEAVING and the number of tokens has changed upon restarting.
Querier: fixed timestamp() function fail with execution: attempted to read series at index 0 from stream, but the stream has already been exhausted if the experimental feature to stream chunks from ingesters to queriers is enabled.
Memberlist: brought back memberlist_client_kv_store_count metric that used to exist in Cortex, but got lost during grafana/dskit updates before Mimir 2.0.
Store-gateway: fixed an issue where stopping a store-gateway could cause all store-gateways to unload all blocks.
Ingester: prevented setting “last update time” of TSDB into the future when opening TSDB. This could prevent detecting of idle TSDB for a long time, if sample in distant future was ingested.
General: changed ballast to allocate smaller blocks to avoid problem when entire ballast was kept in memory working set.

2.10.5

Security: updated the Alpine base image version to 3.18.5 to address CVE-2023-5363. PR 6897
Bugfix: Ingester: fixed possible series matcher corruption leading to wrong series being included in query results. PR 6886

2.10.4

[BUGFIX] Update otelhttp library to v0.44.0 as a mitigation for CVE-2023-45142. PR 6634

2.10.3

[BUGFIX] Update grpc-go library to 1.57.2-dev that includes a fix for a bug introduced in 1.57.1. PR 6419

2.10.2

Note
This release contains a known performance regression fixed in 2.10.3

[BUGFIX] Update grpc-go library to 1.57.1 and golang.org/x/net to 0.17, which include fix for CVE-2023-44487. PR 6349

2.10.1

Security: Update Go version to 1.21.3. PR 6244 PR 6325
Bugfix: Query-frontend: Don’t retry read requests rejected by the ingester due to utilization based read path limiting. PR 6032
Bugfix: Ingester: fix panic in WAL replay of certain native histograms. PR 6086

Grafana Mimir version 2.11 release notes

Wed, 03 Jun 2026 09:01:40 +0200

Grafana Mimir version 2.11 release notes

Grafana Labs is excited to announce version 2.11 of Grafana Mimir.

The highlights that follow include the top features, enhancements, and bugfixes in this release. For the complete list of changes, see the changelog.

Features and enhancements

Sampled logging of errors in the ingester. A high-traffic Mimir cluster can occasionally become bogged down logging high volumes of repeated errors. You can now reduce the amount of errors outputted to logs by setting a sample rate via the -ingester.error-sample-rate CLI flag.
Add total request size instance limit for ingesters. This limit protects the ingesters against requests that together may cause an OOM. Enable this feature by setting the -ingester.instance-limits.max-inflight-push-requests-bytes CLI flag in combination with the -ingester.limit-inflight-requests-using-grpc-method-limiter CLI flag.
Reduce the resolution of incoming native histograms samples if the incoming sample has too many buckets compared to -validation.max-native-histogram-buckets. This is enabled by default but can be turned off by setting the -validation.reduce-native-histogram-over-max-buckets CLI flag to false.
Improved query-scheduler performance under load. This is particularly apparent for clusters with large numbers of queriers.
Ingester to querier chunks streaming reduces the memory utilization of queriers and reduces the likelihood of OOMs.
Ingester query request minimization reduces the number of query requests to ingesters, improving performance and resource utilization for both ingesters and queriers.
The markblocks tool is released in compiled form together with the rest of Mimir. It is used to mark blocks for deletion or as “non-compactable”.

Experimental features

Grafana Mimir 2.11 includes new features that are considered experimental and disabled by default. Please use them with caution and report any issue you encounter:

Block specified queries on a per-tenant basis. This is configured via the blocked_queries limit. See the docs for more information.
Store metadata when ingesting metrics via OTLP. This makes metric description and type available when ingesting metrics via OTLP. You can enable this feature by setting the CLI flag -distributor.enable-otlp-metadata-storage to true.
Reject gRPC push requests that the ingester/distributor is unable to accept before reading them into memory. You can enable this feature by using the -ingester.limit-inflight-requests-using-grpc-method-limiter and/or the -distributor.limit-inflight-requests-using-grpc-method-limiter CLI flags for the ingester and/or the distributor, respectively.
Customize the memcached client write and read buffer size. The buffer allocated for each memcached connection can be configured via the following CLI flags:
- For the blocks storage:
  - -blocks-storage.bucket-store.chunks-cache.memcached.read-buffer-size-bytes
  - -blocks-storage.bucket-store.chunks-cache.memcached.write-buffer-size-bytes
  - -blocks-storage.bucket-store.index-cache.memcached.read-buffer-size-bytes
  - -blocks-storage.bucket-store.index-cache.memcached.write-buffer-size-bytes
  - -blocks-storage.bucket-store.metadata-cache.memcached.read-buffer-size-bytes
  - -blocks-storage.bucket-store.metadata-cache.memcached.write-buffer-size-bytes
- For the query frontend:
  - -query-frontend.results-cache.memcached.read-buffer-size-bytes
  - -query-frontend.results-cache.memcached.write-buffer-size-bytes
- For the ruler storage:
  - -ruler-storage.cache.memcached.read-buffer-size-bytes
  - -ruler-storage.cache.memcached.write-buffer-size-bytes
Configure the number of long-living workers used to process gRPC requests. This can decrease CPU usage by reducing the number of stack allocations. Configure this feature by using the -server.grpc.num-workers CLI flag.
Enforce a limit in bytes on the PostingsForMatchers cache used by ingesters. This limit can be configured via the -blocks-storage.tsdb.head-postings-for-matchers-cache-max-bytes and -blocks-storage.tsdb.block-postings-for-matchers-cache-max-bytes CLI flags.
Pre-allocate the pool of workers in the distributor that are used to send push requests to ingesters. This can decrease CPU usage by reducing the number of stack allocations. You can enable this feature by using the -distributor.reusable-ingester-push-worker flag.
Include a Retry-After header in recoverable error responses from the distributor. This can protect your Mimir cluster from clients including Prometheus that default to retrying very quickly. Enable this feature by setting the -distributor.retry-after-header.enabled CLI flag.

Helm chart improvements

The Grafana Mimir and Grafana Enterprise Metrics Helm chart is now released independently. See the Grafana Mimir Helm chart documentation.

Important changes

In Grafana Mimir 2.11 the following behavior has changed:

The utilization-based read path limiter now operates on Go heap size instead of RSS from the Linux proc file system.

The following configuration options had been previously deprecated and are removed in Grafana Mimir 2.11:

The CLI flag -querier.iterators.
The CLI flag -query.batch-iterators.
The CLI flag -blocks-storage.bucket-store.bucket-index.enabled.
The CLI flag -blocks-storage.bucket-store.chunk-pool-min-bucket-size-bytes.
The CLI flag -blocks-storage.bucket-store.chunk-pool-max-bucket-size-bytes.
The CLI flag -blocks-storage.bucket-store.max-chunk-pool-bytes.

The following configuration options are deprecated and will be removed in Grafana Mimir 2.13:

The CLI flag -log.buffered; this is now the default behavior.

The following metrics are removed:

cortex_query_frontend_workers_enqueued_requests_total; use cortex_query_frontend_enqueue_duration_seconds_count instead.

The following configuration option defaults were changed:

The CLI flag -blocks-storage.bucket-store.index-header.sparse-persistence-enabled now defaults to true.
The default value for the CLI flag -blocks-storage.bucket-store.index-header.lazy-loading-concurrency was changed from 0 to 4.
The default value for the CLI flag -blocks-storage.tsdb.series-hash-cache-max-size-bytes was changed from 1GB to 350MB.
The default value for the CLI flag -blocks-storage.tsdb.early-head-compaction-min-estimated-series-reduction-percentage was changed from 10 to 15.

Bug fixes

Ingester: Respect context cancelation during query execution. PR 6085
Distributor: Return 529 when ingestion rate limit is hit and the distributor.service_overload_status_code_on_rate_limit_enabled flag is active. PR 6549
Query-scheduler: Prevent accumulation of stale querier connections. PR 6100
Packaging: Fix preremove script preventing upgrades on RHEL based OS. PR 6067
Ingester: Fixed possible series matcher corruption leading to wrong series being included in query results. PR 6884
Mimirtool: Fixed missing .metricsUsed field in the output of analyze rule-file. PR 6953

Grafana Mimir version 2.12 release notes

Wed, 03 Jun 2026 09:01:40 +0200

Grafana Mimir version 2.12 release notes

Grafana Labs is excited to announce version 2.12 of Grafana Mimir.

The highlights that follow include the top features, enhancements, and bug fixes in this release. For the complete list of changes, refer to the CHANGELOG.

Features and enhancements

Added support to only count series that are considered active through the Cardinality API endpoint /api/v1/cardinality/label_names by passing the count_method parameter. If set to active it counts only series that are considered active according to the -ingester.active-series-metrics-idle-timeout flag setting rather than counting all in-memory series.
The “Store-gateway: bucket tenant blocks” admin page contains a new column “No Compact”. If block no compaction marker is set, it specifies the reason and the date the marker is added.
The estimated number of compaction jobs based on the current bucket-index is now computed by the compactor. The result is tracked by the new cortex_bucket_index_compaction_jobs metric. If this computation fails, the cortex_bucket_index_compaction_jobs_errors_total metric is updated instead. The estimated number of compaction jobs is also shown in Top tenants, Tenants, and Compactor dashboards.
Added mimir-distroless container image built upon a distroless image (gcr.io/distroless/static-debian12). This improvement minimizes attack surfaces and potential CVEs by trimming down the dependencies within the image. After comprehensive testing, the Mimir maintainers plan to shift from the current image to the distroless version.

Additionally, the following previously experimental features are now considered stable:

The number of pre-allocated workers used to forward push requests to the ingesters, configurable via the -distributor.reusable-ingester-push-workers CLI flag on distributors. It now defaults to 2000. Note that this is a performance optimization, and not a limiting feature. If not enough workers available, new goroutines will be spawned.
The number of gRPC server workers used to serve the requests, configurable via the -server.grpc.num-workers CLI flag. It now defaults to 100. Note that this is the number of pre-allocated long-lived workers, and not a limiting feature. If not enough workers are available, new goroutines will be spawned.
The maximum number of concurrent index header loads across all tenants, configurable via the -blocks-storage.bucket-store.index-header.lazy-loading-concurrency CLI flag on store-gateways. It defaults to 4.
The maximum time to wait for the query-frontend to become ready before rejecting requests, configurable via the -query-frontend.not-running-timeout CLI flag on query-frontends. It now defaults to 2s.
The CLI flag that allows queriers to reduce pressure on ingesters by initially querying only the minimum set of ingesters required to reach quorum, -querier.minimize-ingester-requests. It is now enabled by default.
Spread-minimizing token-related CLI flags: -ingester.ring.token-generation-strategy, -ingester.ring.spread-minimizing-zones and -ingester.ring.spread-minimizing-join-ring-in-order. You can read more about this feature in our blog post.

Important changes

In Grafana Mimir 2.12 the following behavior has changed:

Store-gateway now persists a sparse version of the index-header to disk on construction and loads sparse index-headers from disk instead of the whole index-header. This improves the speed at which index headers are lazy-loaded from disk by up to 90%. The added disk usage is in the order of 1-2%.
Alertmanager deprecated the v1 API. All v1 API endpoints now respond with a JSON deprecation notice and a status code of 410. All endpoints have a v2 equivalent. The list of endpoints is:
- <alertmanager-web.external-url>/api/v1/alerts
- <alertmanager-web.external-url>/api/v1/receivers
- <alertmanager-web.external-url>/api/v1/silence/{id}
- <alertmanager-web.external-url>/api/v1/silences
- <alertmanager-web.external-url>/api/v1/status
Exemplar’s label traceID has been changed to trace_id to be consistent with the OpenTelemetry standard.
Errors returned by ingesters now contain only gRPC status codes. Previously they contained both gRPC and HTTP status codes.

Warning
To guarantee backwards compatibility when migrating from a version prior to 2.11, it’s necessary to first migrate to version 2.11, and then to version 2.12. Otherwise, it might happen that during the migration, some ingester errors with HTTP status code 4xx won’t be recognized, and the corresponding request will be repeated.
Responses with gRPC status codes are now reported as status_code labels in the cortex_request_duration_seconds and cortex_ingester_client_request_duration_seconds metrics.
Responses with HTTP 4xx status codes are now treated as errors and used in status_code label of request duration metric.

The default value of the following CLI flags have been changed:

-blocks-storage.tsdb.head-postings-for-matchers-cache-max-bytes from 10MB to 100MB.
-blocks-storage.tsdb.block-postings-for-matchers-cache-max-bytes from 10MB to 100MB.
-blocks-storage.bucket-store.tenant-sync-concurrency from 10 to 1.
-query-frontend.max-cache-freshness from 1m to 10m.
-distributor.write-requests-buffer-pooling-enabled from false to true.
-locks-storage.bucket-store.block-sync-concurrency from 20 to 4.
-memberlist.stream-timeout from 10s to 2s.
-server.report-grpc-codes-in-instrumentation-label-enabled from false to true.

The following deprecated configuration options are removed in Grafana Mimir 2.12:

The YAML setting frontend.cache_unaligned_requests.
Experimental CLI flag -querier.prefer-streaming-chunks-from-ingesters.

The following configuration options are deprecated and will be removed in Grafana Mimir 2.14:

The CLI flag -ingester.limit-inflight-requests-using-grpc-method-limiter. It now defaults to true.
The CLI flag -ingester.return-only-grpc-errors. It now defaults to true.

Warning
To guarantee backwards compatibility when migrating from a version prior to 2.11, it’s necessary to first migrate to version 2.11, and then to version 2.12. Otherwise, it might happen that during the migration, some ingester errors with HTTP status code 4xx won’t be recognized, and the corresponding request will be repeated.
The CLI flag -ingester.client.report-grpc-codes-in-instrumentation-label-enabled. It now defaults to true.
The CLI flag -distributor.limit-inflight-requests-using-grpc-method-limiter. It now defaults to true.
The CLI flag -distributor.enable-otlp-metadata-storage. It now defaults to true.
The CLI flag -querier.max-query-into-future.

The following metrics are removed or deprecated:

cortex_bucket_store_blocks_loaded_by_duration has been removed.
cortex_distributor_sample_delay_seconds has been deprecated and will be removed in Mimir 2.14.

Experimental features

Grafana Mimir 2.12 includes new features that are considered experimental and disabled by default. Use them with caution and report any issues you encounter:

The maximum number of tenant IDs that may be for a federated query can be configured via the -tenant-federation.max-tenants CLI flag on query-frontends. By default, it’s 0, meaning that the limit is disabled.
Sharding of active series queries can be enabled via the -query-frontend.shard-active-series-queries CLI flag on query-frontends.
Timely head compaction can be enabled via the -blocks-storage.tsdb.timely-head-compaction-enabled on ingesters. If enabled, the head compaction happens when the min block range can no longer be appended, without requiring 1.5x the chunk range worth of data in the head.
Streaming of responses from querier to query-frontend can be enabled via the -querier.response-streaming-enabled CLI flag on queriers. This is currently supported only for responses from the /api/v1/cardinality/active_series endpoint.
The maximum response size for active series queries, in bytes, can be set via the -querier.active-series-results-max-size-bytes CLI flag on queriers.
Metric relabeling on a per-tenant basis can be forcefully disabled via the -distributor.metric-relabeling-enabled CLI flag on rulers. Metrics relabeling is enabled by default.
Query Queue Load Balancing by Query Component. Tenant query queues in the query-scheduler can now be split into subqueues by which query component is expected to be utilized to complete the query: ingesters, store-gateways, both, or uncategorized. Dequeuing queries for a given tenant will rotate through the query component subqueues via simple round-robin. In the event that the one of the query components (ingesters or store-gateways) experience a slowdown, queries only utilizing the other query component can continue to be serviced. This feature is recommended to be enabled. The following CLI flags must be set to true in order to be in effect:
- -query-frontend.additional-query-queue-dimensions-enabled on the query-frontend.
- -query-scheduler.additional-query-queue-dimensions-enabled on the query-scheduler.
Owned series tracking in ingesters can be enabled via the -ingester.track-ingester-owned-series CLI flag. When enabled, ingesters will track the number of in-memory series that still map to the ingester based on the ring state. These counts are more reactive to ring and shard changes than in-memory series, and can be used when enforcing tenant series limits by enabling the -ingester.use-ingester-owned-series-for-limits CLI flag. This feature requires zone-aware replication to be enabled, and the replication factor to be equal to the number of zones.
Support for UTF-8 in Alertmanager can be enabled via the -alertmanager.utf8-strict-mode-enabled CLI flag. When enabled, Alertmanager can receive alerts with labels containing UTF-8 characters, and match these alerts in routes, inhibition rules, and silences. This feature has backwards-incompatible changes. Follow the instructions here to enable it.

Bug fixes

Distributor: fixed an issue where -distributor.metric-relabeling-enabled could cause distributors to panic.
Distributor: fix an issue where -distributor.metric-relabeling-enabled could cause distributors to write unsorted labels and corrupt blocks.
Ingester: errors encountered while iterating through chunks or samples in response to a query request aren’t ignored anymore.
Compactor: out-of-order blocks aren’t allowed to prevent timely compaction anymore.
Querier: requests to store-gateway when a query gets canceled aren’t retried anymore.
Querier: status code 499 is now returned instead of 500 when a request to remote read endpoint gets canceled.
Querier: fixed an issue where -querier.max-fetched-series-per-query wasn’t applied to /series endpoint in case series loaded from ingesters.
Querier: fixed an issue with the remote-read requests HTTP status code translations. Previously, remote-read had conflicting behaviours: when returning samples all internal errors were translated to HTTP 400, while when returning chunks all internal errors were translated to HTTP 500. With this fix, all validation errors will be translated into HTTP 400 errors, while all other errors will be translated into HTTP 500 errors.
Query-frontend: the cortex_query_frontend_queries_total metric incorrectly reported op="query" for any request which wasn’t a range query. Now the op label value can be one of the following:
- query: instant query
- query_range: range query
- cardinality: cardinality query
- label_names_and_values: label names / values query
- active_series: active series query
- other: any other request
Ruler: fixed an issue where “failed to remotely evaluate query expression, will retry” messages were logged without context such as the trace ID and didn’t appear in trace events.
Ruler: requests to remote querier when server’s response exceeds its configured max payload size aren’t retried anymore.
Ruler: fixed a regression that caused client errors to be tracked in cortex_ruler_write_requests_failed_total metric.
Ruler: fixed an issue with recording rule result being corruption due to an usage of a bad native histogram pointer.

Helm chart improvements

The Grafana Mimir and Grafana Enterprise Metrics Helm charts are released independently. Refer to the Grafana Mimir Helm chart documentation.

Grafana Mimir version 2.13 release notes

Wed, 03 Jun 2026 09:01:40 +0200

Grafana Mimir version 2.13 release notes

Grafana Labs is excited to announce version 2.13 of Grafana Mimir.

The highlights that follow include the top features, enhancements, and bug fixes in this release. For the complete list of changes, refer to the CHANGELOG.

Features and enhancements

Improved CPU performance in processing queries with regular expressions that contain many alternations (for example foo|bar|baz|...).
Improved OTLP ingestion performance by using native translation, which reduces memory usage in the distributor by up to 30% and CPU usage by up to 8%.
Configurable S3 bucket lookup type improves interoperability with S3-compatible providers, such as Tencent and Alibaba. Configure lookup type via -<prefix>.s3.bucket-lookup-type or -common.storage.s3-bucket-lookup-type.
Configuration of TLS for S3 buckets is now possible via -<prefix>.s3.http.* and -common.storage.s3.http.* configuration options.
Mimirtool can now verify the validity of a Mimir runtime configuration file with the mimirtool runtime-config verify command.
Active series are now updated along with owned series. This means the number of active series for a tenant is more accurate after scaling out ingesters. As a result, ingesters now more precisely enforce tenants’ series limits. This feature is only enabled when -ingester.track-ingester-owned-series or -ingester.use-ingester-owned-series-for-limits are enabled.
Store-gateway can be configured to explicitly disable or enable tenants. This is useful in cases where compaction slows down for a tenant to temporarily exclude another tenant while compaction catches up without affecting other tenants.
Remote read (/prometheus/api/v1/read) is becoming a first-class endpoint in Mimir with support in query stats logs and query blocking in the query-frontend. More coming in future releases.

Additionally, the following previously experimental features are now considered stable:

Rules tenant federation via the source_tenants field in rule groups.
Enabling recording, and alerting rules evaluation on a per-tenant basis.
Limiting the number of tenants a federated query can query.

Important changes

In Grafana Mimir 2.13 the following behavior has changed:

The default Docker image grafana/mimir is now based on the distroless image gcr.io/distroless/static-debian12. See Debugging distroless container images for more details on how to work with distroless images.
Error logs in the ingester are now sampled at 10% by default. Sampled log lines contain (sampled 1/10). The cortex_discarded_samples_total metric still tracks all discarded samples.
Anonymous usage statistics now include actual CPU usage instead of available CPU cores.
Continuous-test is no longer a standalone binary and is now part of Mimir as its own target. The published Docker image has been updated to use the new packaging.

The following deprecated configuration options are removed in Grafana Mimir 2.13:

The configuration option -log.buffered, which was deprecated in 2.11 and is now enabled by default.

Experimental features

Grafana Mimir 2.13 includes new features that are considered experimental and disabled by default. Use them with caution and report any issues you encounter:

Experimental support for server-side circuit breakers in ingesters on read and write requests. This can be enabled using -ingester.push-circuit-breaker.enabled and -ingester.read-circuit-breaker.enabled options and further configured via the -ingester.push-circuit-breaker.* and -ingester.read-circuit-breaker.* options.

The following configuration options are deprecated and will be removed in a future Grafana Mimir release:

evaluation_delay field in the rule group configuration has been deprecated. Please use query_offset instead.

Bug fixes

OTLP ingestion: translate all HTTP 5xx errors into one of 502, 503, or 504, so that the otel-collector retries the failed requests.
OTLP ingestion: return properly formatted protobuf error messages when ingestion fails.
OTLP ingestion: generate target_info metric only when there are metrics in the request and there is at least one configured identifying label.
OTLP ingestion: don’t discard timeseries paired with invalid exemplars. Instead, try to ingest the timeseries and discard only the invalid exemplars.
Subqueries: @ end() and @ start() now work correctly with queries split by time.
Native histograms: order exemplars before ingestion to improve success rate when ingesting multiple exemplars.
Native histograms: return HTTP 400 on invalid native histogram samples instead of HTTP 500. The metric cortex_discarded_samples_total{reason="invalid-native-histogram"} is now incremented on invalid histogram samples.

Helm chart improvements

The Grafana Mimir and Grafana Enterprise Metrics Helm charts are released independently. Refer to the Grafana Mimir Helm chart documentation.

v2.13.1

[BUGFIX] Upgrade Go to 1.22.9 to address CVE-2024-34156. PR 10097
[BUGFIX] Update module google.golang.org/grpc to v1.64.1 to address GHSA-xr7q-jx4m-x55m. PR 8717

Grafana Mimir version 2.14 release notes

Wed, 03 Jun 2026 09:01:40 +0200

Grafana Mimir version 2.14 release notes

Grafana Labs is excited to announce version 2.14 of Grafana Mimir.

The highlights that follow include the top features, enhancements, and bug fixes in this release. For the complete list of changes, refer to the CHANGELOG.

Features and enhancements

The streaming of chunks from store-gateways to queriers is now enabled by default. This reduces the memory usage in queriers. This was an experimental feature since Mimir 2.10, and is now considered stable.

Compactor adds a new cortex_compactor_disk_out_of_space_errors_total counter metric that tracks how many times a compaction fails due to the compactor being out of disk.

The distributor now replies with the Retry-After header on retryable errors by default. This protects Mimir from clients, including Prometheus, that default to retrying very quickly, making recovering from an outage easier. The feature was originally added as experimental in Mimir 2.11.

Incoming OTLP requests were previously size-limited with the distributor’s -distributor.max-recv-msg-size configuration. The distributor has a new -distributor.max-otlp-request-size configuration for limiting OTLP requests. The default value is 100 MiB.

Ingesters can be marked as read-only as part of their downscaling procedure. The new prepare-instance-ring-downscale endpoint updates the read-only status of an ingester in the ring.

Important changes

In Grafana Mimir 2.14, the following behavior has changed:

When running a remote read request, the querier honors the time range specified in the read hints.

The default inactivity timeout of active series in ingesters, controlled by the -ingester.active-series-metrics-idle-timeout configuration, is increased from 10m to 20m.

The following featues of store-gateway are changed: -blocks-storage.bucket-store.max-concurrent-queue-timeout is set to five seconds; -blocks-storage.bucket-store.index-header.lazy-loading-concurrency-queue-timeout is set to five seconds; -blocks-storage.bucket-store.max-concurrent is set to 200;

The experimental support for Redis caching is now deprecated and set to be removed in the next major release. Users are encouraged to switch to use Memcached.

The following deprecated configuration options were removed in this release:

The -ingester.return-only-grpc-errors option in the ingester
The -ingester.client.circuit-breaker.* options in the ingester
The -ingester.limit-inflight-requests-using-grpc-method-limiter option in the ingester
The -ingester.client.report-grpc-codes-in-instrumentation-label-enabled option in the distributor and ruler
The -distributor.limit-inflight-requests-using-grpc-method-limiter option in the distributor
The -distributor.enable-otlp-metadata-storage option in the distributor
The -ruler.drain-notification-queue-on-shutdown option in the ruler
The -querier.max-query-into-future option in the querier
The -querier.prefer-streaming-chunks-from-store-gateways option in the querier and the store-gateway
The -query-scheduler.use-multi-algorithm-query-queue option in the querier-scheduler
The YAML configuration frontend.align_queries_with_step in the query-frontend

Experimental features

Grafana Mimir 2.14 includes some features that are experimental and disabled by default. Use these features with caution and report any issues that you encounter:

The ingester added an experimental -ingester.ignore-ooo-exemplars configuration. When set, out-of-order exemplars are no longer reported to the remote write client.

The querier supports the experimental limitk() and limit_ratio() PromQL functions. This feature is disabled by default, but you can enable it with the -querier.promql-experimental-functions-enabled=true setting in the query-frontend and the querier.

Bug fixes

Alertmanager: fix configuration validation gap around unreferenced templates.
Alertmanager: fix goroutine leak when stored configuration fails to apply and there is no existing tenant alertmanager.
Alertmanager: fix receiver firewall to detect 0.0.0.0 and IPv6 interface-local multicast address as local addresses.
Alertmanager: fix per-tenant silence limits not reloaded during runtime.
Alertmanager: fix bugs in silences that could cause an existing silence to expire/be deleted when updating the silence fails. This could happen when the updated silence was invalid or exceeded limits.
Alertmanager: fix help message for utf-8-strict-mode.
Compactor: fix a race condition between different compactor replicas that may cause a deleted block to be referenced as non-deleted in the bucket index.
Configuration: multi-line environment variables are flattened during injection to be compatible with YAML syntax.
HA Tracker: store correct timestamp for the last-received request from the elected replica.
Ingester: fix the sporadic not found error causing an internal server error if label names are queried with matchers during head compaction.
Ingester, store-gateway: fix case insensitive regular expressions not correctly matching some Unicode characters.
Ingester: fixed timestamp reported in the “the sample has been rejected because its timestamp is too old” error when the write request contains only histograms.
Query-frontend: fix -querier.max-query-lookback and -compactor.blocks-retention-period enforcement in query-frontend when one of the two is not set.
Query-frontend: “query stats” log includes the actual status_code when the request fails due to an error occurring in the query-frontend itself.
Query-frontend: ensure that internal errors result in an HTTP 500 response code instead of a 422 response code.
Query-frontend: return annotations generated during evaluation of sharded queries.
Query-scheduler: fix a panic in request queueing.
Querier: fix the issue where “context canceled” is logged for trace spans for requests to store-gateways that return no series when chunks streaming is enabled.
Querier: fix issue where queries can return incorrect results if a single store-gateway returns overlapping chunks for a series.
Querier: do not return grpc: the client connection is closing errors as HTTP 499.
Querier: fix issue where some native histogram-related warnings were not emitted when rate() was used over native histograms.
Querier: fix invalid query results when multiple chunks are merged.
Querier: support optional start and end times on /prometheus/api/v1/labels, /prometheus/api/v1/label/<label>/values, and /prometheus/api/v1/series when max_query_into_future: 0.
Querier: fix issue where both recently compacted blocks and their source blocks can be skipped during querying if store-gateways are restarting.
Ruler: add support for draining any outstanding alert notifications before shutting down. Enable this setting with the -ruler.drain-notification-queue-on-shutdown=true CLI flag.
Store-gateway: fixed a case where, on a quick subsequent restart, the previous lazy-loaded index header snapshot was overwritten by a partially loaded one.
Store-gateway: store sparse index headers atomically to disk.
Ruler: map invalid org-id errors to the 400 status code.

Helm chart improvements

The Grafana Mimir and Grafana Enterprise Metrics Helm charts are released independently. Refer to the Grafana Mimir Helm chart documentation.

Grafana Mimir version 2.15 release notes

Wed, 03 Jun 2026 09:01:40 +0200

Grafana Mimir version 2.15 release notes

Grafana Labs is excited to announce version 2.15 of Grafana Mimir.

The highlights that follow include the top features, enhancements, and bug fixes in this release. For the complete list of changes, refer to the CHANGELOG.

Features and enhancements

S2 compression for gRPC is now supported using the following flags:

-alertmanager.alertmanager-client.grpc-compression=s2
-ingester.client.grpc-compression=s2
-querier.frontend-client.grpc-compression=s2
-querier.scheduler-client.grpc-compression=s2
-query-frontend.grpc-client-config.grpc-compression=s2
-query-scheduler.grpc-client-config.grpc-compression=s2
-ruler.client.grpc-compression=s2
-ruler.query-frontend.grpc-client-config.grpc-compression=s2

Distributors now support lz4 OTLP compression, and you can deploy them in multiple availability zones.

The ruler’s <prometheus-http-prefix>/api/v1/rules endpoint now supports the exclude_alerts, group_limit, and group_next_token parameters.

mimirtool’s analyze ruler and analyze prometheus commands now support bearer tokens.

You can now tune HTTP client settings for GCS and Azure backends via an http block or corresponding CLI flags.

The compactor now refreshes deletion marks concurrently when updating the bucket index.

You can now set the number of Memcached replicas for each type of cache using configuration settings when using jsonnet.

Important changes

In Grafana Mimir 2.15, the following behavior has changed:

The following alertmanager metrics are not exported for a user label when the metric value is zero:

cortex_alertmanager_alerts_received_total
cortex_alertmanager_alerts_invalid_total
cortex_alertmanager_partial_state_merges_total
cortex_alertmanager_partial_state_merges_failed_total
cortex_alertmanager_state_replication_total
cortex_alertmanager_state_replication_failed_total
cortex_alertmanager_alerts
cortex_alertmanager_silences

PromQL compatibility has been upgraded from Prometheus 2.0 to 3.0. For more details, refer to the Prometheus documentation. The following changes are of note:

The . pattern in regular expressions in PromQL now matches newline characters.
Lookback and range selectors are left-open and right-closed. They were previously left-closed and right-closed.
Native histograms now use exponential interpolation.

Backwards compatibility in dashboards and alerts for thanos_memcached_-prefixed metrics has been removed. These metrics were removed in 2.12 in favor of thanos_cache_-prefixed metrics.

Experimental support for Redis as a cache backend has been removed from jsonnet.

The following deprecated configuration options were removed in this release:

-distributor.direct-otlp-translation-enabled, which has been enabled by default since 2.13 and is now considered stable.
-query-scheduler.prioritize-query-components, which is now always enabled.
-api.get-request-for-ingester-shutdown-enabled, a deprecated experimental flag which has been marked for removal in 2.15.

Experimental features

Grafana Mimir 2.15 includes some features that are experimental and disabled by default. Use these features with caution and report any issues that you encounter:

You can now enable Mimir’s experimental PromQL engine with -querier.query-engine=mimir. This new engine provides improved performance and reduced querier resource consumption. However, it only supports a subset of all PromQL features. It falls back to the regular Prometheus engine for queries containing unsupported features.

You can now use the query-frontend to cache non-transient errors using the experimental flags -query-frontend.cache-errors and -query-frontend.results-cache-ttl-for-errors.

The query-frontend and querier both support an experimental PromQL function, double_exponential_smoothing, which you can enable by setting -querier.promql-experimental-functions-enabled=true and -query-frontend.promql-experimental-functions-enabled=true.

The ingester now supports out-of-order native histogram ingestion via the flag -ingester.ooo-native-histograms-ingestion-enabled.

The ingester can now build 24h blocks for out-of-order data which is more than 24 hours old, using the setting -blocks-storage.tsdb.bigger-out-of-order-blocks-for-old-samples.

The ruler now supports caching the contents of rule groups via the setting -ruler-storage.cache.rule-group-enabled.

The distributor now supports promotion of OTel resource attributes to labels via the setting -distributor.otel-promote-resource-attributes.

Bug fixes

Alerts: Fix autoscaling metrics joins in MimirAutoscalerNotActive when series churn.
Alerts: Exclude failed cache “add” operations from alerting since failures are expected in normal operation.
Alerts: Exclude read-only replicas from IngesterInstanceHasNoTenants alert.
Alerts: Use resident set memory for the EtcdAllocatingTooMuchMemory alert so that ephemeral file cache memory doesn’t cause the alert to misfire.
Dashboards: Fix autoscaling metrics joins when series churn.
Distributor: Fix pooling buffer reuse logic when -distributor.max-request-pool-buffer-size is set.
Ingester: Fix issue where active series requests error when encountering a stale posting.
Ingester: Fix race condition in per-tenant TSDB creation.
Ingester: Fix race condition in exemplar adding.
Ingester: Fix race condition in native histogram appending.
Ingester: Fix bug in concurrent fetching where a failure to list topics on startup would cause an invalid topic ID (0x00000000000000000000000000000000).
Ingester: Fix data loss bug in the experimental ingest storage when a Kafka Fetch is split into multiple requests and some of them return an error.
Ingester: Fix bug where chunks could have one unnecessary zero byte at the end.
OTLP: Support integer exemplar value type.
OTLP receiver: Preserve colons and combine multiple consecutive underscores into one when generating metric names in suffix adding mode (-distributor.otel-metric-suffixes-enabled).
Prometheus: Fix issue where negation of native histograms (e.g. -some_native_histogram_series) did nothing.
Prometheus: Always return unknown hint for first sample in non-gauge native histograms chunk to avoid incorrect counter reset hints when merging chunks from different sources.
Prometheus: Ensure native histograms counter reset hints are corrected when merging results from different sources.
PromQL: Fix issue where functions such as rate() over native histograms could return incorrect values if a float stale marker was present in the selected range.
PromQL: round now removes the metric name again.
PromQL: Fix issue where metric might not be a counter, name does not end in _total/_sum/_count/_bucket annotation would be emitted even if rate or increase did not have enough samples to compute a result.
Querier: Fix the behavior of binary operators between native histograms and floats.
Querier: Fix stddev+stdvar aggregations to always ignore native histograms, and to treat infinity consistently.
Query-frontend: Fix issue where sharded queries could return annotations with incorrect or confusing position information.
Query-frontend: Fix issue where downstream consumers may not generate correct cache keys for experimental error caching.
Query-frontend: Support X-Read-Consistency-Offsets on labels queries too.
Ruler: Fix issue when using the experimental -ruler.max-independent-rule-evaluation-concurrency feature, where the ruler could panic as it updates a running ruleset or shutdowns.

Helm chart improvements

The Grafana Mimir and Grafana Enterprise Metrics Helm charts are released independently. Refer to the Grafana Mimir Helm chart documentation.

Grafana Mimir version 2.16 release notes

Wed, 03 Jun 2026 09:01:40 +0200

Grafana Mimir version 2.16 release notes

Grafana Labs is excited to announce version 2.16 of Grafana Mimir.

The highlights that follow include the top features, enhancements, and bug fixes in this release. For the complete list of changes, refer to the CHANGELOG.

Features and enhancements

In rulers, when rule concurrency is enabled for a rule group, its rules are now reordered and run in batches based on their dependencies. This increases the number of rules that can run concurrently. Note that the global and tenant-specific limits around the number of rule groups and rules per group still apply.

Using mimirtool to analyze Grafana dashboards now supports bar chart, pie chart, state timeline, status history, histogram, candlestick, canvas, flame graph, geomap, node graph, trend, and XY chart panels.

Important changes

In Grafana Mimir 2.16, the following behavior has changed:

Grafana Mimir only provides container images based on distroless images. Alpine Linux-based container images were deprecated in the 2.12 release and are no longer built.

How experimental PromQL functions are enabled has changed.

The experimental CLI flags -querier.promql-experimental-functions-enabled and -query-frontend.block-promql-experimental-functions and their respective YAML configurations have been removed from query-frontends and queriers.
Experimental PromQL functions are disabled by default. You can enable them using only the enabled_promql_experimental_functions per-tenant setting.

Support for native histograms and out-of-order native histograms is enabled by default in ingesters.

Distributors discard float and histogram samples with duplicated timestamps from each timeseries in a request before the request is forwarded to ingesters. Discarded samples are tracked by cortex_discarded_samples_total metrics with the reason sample_duplicate_timestamp.

Experimental features

Grafana Mimir 2.16 includes some features that are experimental and disabled by default. Use these features with caution and report any issues that you encounter:

Distributors now include experimental support for the Influx line protocol.

Query-frontends now include experimental support to “spin off” subqueries as actual range queries, so that they benefit from query acceleration techniques such as sharding, splitting, and caching.

Bug fixes

Distributor: Use a boolean to track changes while merging the ReplicaDesc components, rather than comparing the objects directly. #10185
Querier: Fix timeout responding to the query-frontend when the response size is within a few hundred bytes of -querier.frontend-client.grpc-max-send-msg-size. #10154
Query-frontend and querier: Show warning and info annotations in some cases where they were missing (if a lazy querier was used). #10277
Query-frontend: Fix an issue where transient errors are inadvertently cached. #10537 #10631
Ruler: Fix indeterminate rules always, instead of never, running concurrently when -ruler.max-independent-rule-evaluation-concurrency is set. https://github.com/prometheus/prometheus/pull/15560 #10258
PromQL: Fix various UTF-8 bugs related to quoting. https://github.com/prometheus/prometheus/pull/15531 #10258
Ruler: Fix an issue when using the experimental -ruler.max-independent-rule-evaluation-concurrency feature, where if a rule group was eligible for concurrency, it would flap between running concurrently or not based on the time it took after running concurrently. #9726 #10189
Mimirtool: remote-read commands now return data. #10286
PromQL: Fix deriv, predict_linear and double_exponential_smoothing with histograms https://github.com/prometheus/prometheus/pull/15686 #10383
MQE: Fix deriv with histograms. #10383
PromQL: Fix <aggr_over_time> functions with histograms. https://github.com/prometheus/prometheus/pull/15711 #10400
MQE: Fix <aggr_over_time> functions with histograms. #10400
Distributor: Return HTTP status 415, Unsupported Media Type, instead of 200, Success, for Remote Write 2.0 until we support it. #10423 #10916
Query-frontend: Add -query-frontend.prom2-range-compat flag and corresponding YAML to rewrite queries with ranges that worked in Prometheus 2 but are invalid in Prometheus 3. #10445 #10461 #10502
Distributor: Fix edge case at the HA-tracker with memberlist as KVStore, where when a replica in the KVStore is marked as deleted but not yet removed, it fails to update the KVStore. #10443
Distributor: Fix panics in DurationWithJitter util functions when computed variance is zero. #10507
Ingester: Fixed a race condition in the PostingsForMatchers cache that may have infrequently returned expired cached postings. #10500
Distributor: Report partially converted OTLP requests with status 400, Bad Request. #10588
Ruler: Fix issue where rule evaluations could be missed while shutting down a ruler instance if that instance owns many rule groups. prometheus/prometheus#15804 #10762
Ingester: Add additional check on reactive limiter queue sizes. #10722
TSDB: Fix unknown series errors and possible lost data during WAL replay when series are removed from the head due to inactivity and reappear before the next WAL checkpoint. https://github.com/prometheus/prometheus/pull/16060 #10824
Querier: Fix issue where label_join could incorrectly return multiple series with the same labels rather than failing with vector cannot contain metrics with the same labelset. https://github.com/prometheus/prometheus/pull/15975 #10826
Querier: Fix issue where counter resets on native histograms could be incorrectly under or over-counted when using subqueries. https://github.com/prometheus/prometheus/pull/15987 #10871
Ingester: Fix goroutines and memory leak when experimental ingest storage is enabled and a server-side error occurs during metrics ingestion. #10915
Mimirtool: Fix issue where MIMIR_HTTP_PREFIX environment variable was ignored and the value from MIMIR_MIMIR_HTTP_PREFIX was used instead. #10207
Update to Go v1.23.9 to address CVE-2025-22871. #11543
Update golang.org/x/net to v0.38.0 to address CVE-2025-22872. #11281
Query-frontend: Fix a panic in monolithic mode caused by a clash in labels of the cortex_client_invalid_cluster_validation_label_requests_total metric definition. #11455

2.16.2

Security: Update to Go v1.23.12 to address CVE-2025-22871, CVE-2025-4673, CVE-2025-0913. #12582
Security: Update Docker base images for tools from alpine:3.21.3 to alpine:3.21.5 to address CVE-2025-9230, CVE-2025-9231, CVE-2025-2025-9232. #12990

Helm chart improvements

The Grafana Mimir and Grafana Enterprise Metrics Helm chart is released independently. Refer to the Grafana Mimir Helm chart documentation.

Grafana Mimir version 2.17 release notes

Wed, 03 Jun 2026 09:01:40 +0200

Grafana Mimir version 2.17 release notes

Grafana Labs is excited to announce version 2.17 of Grafana Mimir.

The highlights that follow include the top features, enhancements, and bug fixes in this release. For the complete list of changes, refer to the CHANGELOG.

Features and enhancements

MQE is enabled by default in queriers. MQE provides benefits over the Prometheus engine, inluding reduced memory and CPU consumption and improved performance. To use the Prometheus engine instead of MQE, set -querier.query-engine=prometheus.

Grafana Mimir now supports using the Mimir Query Engine (MQE) in query-frontends in addition to queriers. You can enable MQE in query-frontends by setting the experimental CLI flag -query-frontend.query-engine=mimir or through the corresponding YAML option.

You can export the cortex_ingester_attributed_active_native_histogram_series and cortex_ingester_attributed_active_native_histogram_buckets native histogram cost attribution metrics to a custom Prometheus registry with user-specified labels.

Grafana Mimir supports converting OTel explicit bucket histograms to Prometheus native histograms with custom buckets using the distributor.otel-convert-histograms-to-nhcb flag.

The following experimental features have been removed:

The max_cost_attribution_labels_per_user cost attribution limit
Read-write deployment mode in the mixin

Important changes

In Grafana Mimir 2.17, the following behavior has changed:

The following default configuration values now apply to the memberlist KV store:

Key	Value
`memberlist.packet-dial-timeout`	`500ms`
`memberlist.packet-write-timeout`	`500ms`
`memberlist.max-concurrent-writes`	`5`
`memberlist.acquire-writer-timeout`	`1s`

These values perform better but might cause long-running packets to be dropped in high-latency networks.

The -ruler-storage.cache.rule-group-enabled experimental CLI flag has been removed. Caching rule group contents is now always enabled when a cache is configured for the ruler.

The -ingester.ooo-native-histograms-ingestion-enabled CLI flag and corresponding ooo_native_histograms_ingestion_enabled runtime configuration option have been removed. Out-of-order native histograms are now enabled whenever both native histogram and out-of-order ingestion is enabled.

The -ingester.stream-chunks-when-using-blocks CLI flag and corresponding ingester_stream_chunks_when_using_blocks runtime configuration option have been deprecated and will be removed in a future release.

The cortex_distributor_label_values_with_newlines_total metric has been removed.

In the distributor, memberlist is marked as a stable option for backend storage for the high availability tracker. etcd has been deprecated for this purpose.

Experimental features

Grafana Mimir 2.17 includes some features that are experimental. Use these features with caution and report any issues that you encounter:

Prometheus Remote-Write 2.0 protocol.
Duration expressions in PromQL. These are simple arithmetics on numbers in offset and range specification. For example, rate(http_requests_total[5m * 2]).
Promoting OTel scope metadata, including name, version, schema URL, and attributes, to metric labels, prefixed with otel_scope_. Enable this feature through the -distributor.otel-promote-scope-metadata flag.
Allowing primitive delta metrics ingestion through the OTLP endpoint with the -distributor.otel-native-delta-ingestion option.
Support for sort_by_label and sort_by_label_desc PromQL functions. Enable this feature through the -query-frontend.enabled-promql-experimental-functions flag.
Support for cluster validation in HTTP calls. When enabled, the HTTP server verifies if a request coming from an HTTP client comes from an expected cluster. Disabled by default. You can configure this validation with the following options:
- -server.cluster-validation.label
- -server.cluster-validation.http.enabled
- -server.cluster-validation.http.soft-validation
- -server.cluster-validation.http.exclude-paths

Bug fixes

For a detailed list of bug fixes, refer to the CHANGELOG.

Helm chart improvements

The Grafana Mimir and Grafana Enterprise Metrics Helm chart is released independently. Refer to the Grafana Mimir Helm chart documentation.

Grafana Mimir version 3.0 release notes

Wed, 03 Jun 2026 09:01:40 +0200

Grafana Mimir version 3.0 release notes

Grafana Labs is excited to announce version 3.0 of Grafana Mimir.

The highlights that follow include the top features, enhancements, and bug fixes in this release. For the complete list of changes, refer to the CHANGELOG.

Features and enhancements

Grafana Mimir version 3.0 includes the following key features and enhancements.

Ingest storage architecture

Mimir 3.0 introduces ingest storage architecture, a next-generation architecture that separates the read and write paths using a Kafka-based ingest storage layer. In previous versions, the ingester handled both ingestion and querying, which could cause contention under heavy query loads. By introducing Kafka as an asynchronous buffer between ingestion and query, reads and writes now scale independently. This design improves ingestion stability, query performance, and operational flexibility for large deployments.

Note that in Grafana Mimir version 3.0, classic architecture is still supported.

Mimir Query Engine (MQE)

The Mimir Query Engine (MQE) is now the default query engine for all queriers and query-frontends. MQE is fully PromQL-compatible and processes data in a streaming fashion, reducing peak memory usage by up to 92% compared to the Prometheus engine. You can expect faster, more stable query performance and lower resource consumption under load.

Additional improvements

Grafana Mimir 3.0 also includes:

Optimizations to MQE for histogram queries and query rewrites
Improvements to query planning in frontends and queriers
General stability and performance updates across core components

Important changes

Grafana Mimir 3.0 introduces several updates that change default behavior and configuration. Review these changes before upgrading:

Ingest storage architecture: Grafana Mimir now uses a Kafka-based ingest storage layer that separates reads and writes. This change improves performance by altering data flow between components.
MQE as the default engine: MQE replaces the Prometheus engine as the default query engine. To continue using the Prometheus engine, set -querier.query-engine=prometheus.
Removal of experimental features: Some deprecated and experimental flags, including the read-write deployment mode, have been removed.
Configuration updates: Some deprecated CLI flags and configuration options have been removed or renamed.

Experimental features

Grafana Mimir 3.0 includes some features that are experimental. Use these features with caution and report any issues that you encounter:

Prometheus Remote-Write 2.0 protocol
PromQL duration expressions that allow simple arithmetic in range or offset values
Cluster validation for HTTP requests (disabled by default, enable through -server.cluster-validation.* flags)
Native OTLP delta metric ingestion (disabled by default, enable through -distributor.otel-native-delta-ingestion)

Bug fixes

For a detailed list of bug fixes, refer to the CHANGELOG.

Helm chart improvements

The Grafana Mimir Helm chart is released independently. Refer to the Grafana Mimir Helm chart documentation.

Grafana Mimir version 3.1 release notes

Wed, 03 Jun 2026 09:01:40 +0200

Grafana Mimir version 3.1 release notes

Grafana Labs is excited to announce version 3.1 of Grafana Mimir.

The highlights that follow include the top features, enhancements, and bug fixes in this release. For the complete list of changes, refer to the CHANGELOG.

Features and enhancements

Grafana Mimir version 3.1 includes the following key features and enhancements.

More Kafka options for Ingest storage

Ingest storage now supports additional ways to authenticate with Kafka through the new -ingest-storage.kafka.sasl-mechanism flag, including SCRAM, OAUTHBEARER, and AWS MSK IAM authentication. In addition, new -ingest-storage.kafka.tls* flags allow connecting to Kafka clusters over TLS, including mTLS. Refer to Kafka configuration for details.

You can also configure multiple Kafka seed brokers via comma-separated values in -ingest-storage.kafka.address and enable rack-aware consumption with -ingest-storage.kafka.client-rack.

Separate ingestion limits by tenant metadata

Distributors can now track limits separately based on tenant metadata. This allows operators to track limits separately for subsets of write requests belonging to the same tenant, for example to prioritize some sources of metrics over others.

Clients may pass tenant metadata in the X-Scope-OrgID header using the format tenantID:key1=value1:key2=value2, and operators may define per-metadata overrides in the runtime configuration.

Mimir Query Engine (MQE) improvements

MQE continues to receive significant optimizations in this release:

Experimental extended range selector modifiers smoothed and anchored, enabled with -query-frontend.enabled-promql-extended-range-selectors.
Optimization passes for common subexpression elimination, subset selector elimination, projection pushdown, and multi-aggregation without buffering.
Improved per-query memory consumption limit enforcement in histogram and aggregation operations.
Experimental support for splitting and caching intermediate results for functions over range vectors in instant queries.
Experimental support for the info() PromQL function.

Zone-aware memberlist routing

A new experimental zone-aware routing feature for memberlist reduces cross-AZ data transfer by routing gossip messages within the local availability zone when possible. Configure it with -memberlist.zone-aware-routing.* flags.

Additional improvements

Grafana Mimir 3.1 also includes:

Store-gateways now verify CRC32 checksums for 1 out of every 128 chunks read from object storage and the chunks cache to detect corruption.
GCS uploads are now optionally retryable, with configurable max retries per storage backend.
Disk interaction has been removed when loading ruler rules. Rule evaluation failures now include a reason label (operator or user) in prometheus_rule_evaluation_failures_total for better error classification.
Query blocking via blocked_queries is now stable and no longer experimental, with support for blocking queries exceeding a time range duration (time_range_longer_than) or with steps smaller than a threshold (step_size_shorter_than).
The per-tenant postings-for-matchers cache is now stable.
Out-of-order ingestion support is now stable, configured via -ingester.out-of-order-time-window.
The -alertmanager.utf8-strict-mode-enabled flag is now stable.
The query-scheduler now drains the queue before exiting during shutdown.
Distributors support zone-aware rate limiting via -distributor.ring.instance-availability-zone, dividing the global ingestion rate by zones instead of total distributors.
Default ingest storage configuration now enables concurrency settings for improved throughput.
Optional per-tenant max limits for label name and label value requests via max_label_names_limit and max_label_values_limit.
Runtime configuration can now be loaded from HTTP URLs in addition to local files via -runtime-config.file. This may reduce configuration propagation times.

Important changes

Grafana Mimir 3.1 introduces several updates that change default behavior and configuration. Review these changes before upgrading:

Experimental support for disabling ring heartbeats and heartbeat timeouts has been removed.
The -target=flusher mode has been removed; use the /ingester/flush HTTP endpoint instead.
Uploaded TSDB blocks must now use v2 of the index file format. Store-gateways no longer generate index-headers from v1 index format blocks.
Per-step stats are no longer supported when MQE is enabled. The -query-frontend.cache-samples-processed-stats flag is deprecated and has no effect.
The -querier.response-streaming-enabled flag has been removed; active series responses are now always streamed.
cortex_ingest_storage_writer_buffered_produce_bytes has been renamed to cortex_ingest_storage_writer_buffered_produce_bytes_distribution.
Metric cortex_ingester_owned_target_info_series has been removed.
The cost_attribution_labels configuration option has been removed; use cost_attribution_labels_structured instead.
The minimum_step_size filter in blocked_queries has been renamed to step_size_shorter_than. Blocked queries configuration is now validated at load time.
-querier.prefer-availability-zone has been renamed to -querier.prefer-availability-zones and now accepts a comma-separated list.
The per-query memory consumption limit now considers buffered messages and spans all time-split sub-queries when MQE is enabled.
The following flags have been removed:
- -distributor.metric-relabeling-enabled
- -compactor.no-blocks-file-cleanup-enabled
- -compactor.in-memory-tenant-meta-cache-size
- -blocks-storage.bucket-store.index-header.eager-loading-startup-enabled
- *.memcached.dns-ignore-startup-failures

Experimental features

Grafana Mimir 3.1 includes some features that are experimental. Use these features with caution and report any issues that you encounter:

New usage-tracker component to enforce series limits before data is ingested.
Zone-aware memberlist routing to reduce cross-AZ data transfer.
Query planning in query-frontends with distributed execution across queriers.
MQE extended range selector modifiers (smoothed, anchored).
MQE info() function.
MQE optimization passes: projection pushdown, multi-aggregation, subset selector elimination, common subexpression elimination for range vector expressions.
Per-zone store-gateway shard size (-store-gateway.tenant-shard-size-per-zone).
Running ingesters with no tokens in the ring when ingest storage is enabled (-ingester.ring.num-tokens=0).
Per-sample HA deduplication (-distributor.ha-tracker.per-sample-dedupe).
Per-tenant early head compaction for ingesters based on owned series count.
Store-gateway excluded zones (-store-gateway.sharding-ring.excluded-zones).
Controlling OTLP metric name suffix addition and translation strategy via request headers, gated by -api.otlp-translation-headers-enabled.
Memberlist propagation delay tracker (-memberlist.propagation-delay-tracker.enabled).
Reporting the number of samples read per query in MQE.

Bug fixes

For a detailed list of bug fixes, refer to the CHANGELOG.

Helm chart improvements

The Grafana Mimir Helm chart is released independently. Refer to the Grafana Mimir Helm chart documentation.