Documentation Grafana Cloud Adaptive Telemetry Adaptive Traces Review and apply recommendations Use diversity sampling

Use diversity sampling

The diversity policy is a trace sampling policy for Adaptive Traces that ensures rare, unique, and low-traffic traces are always captured.

A unique trace combination (also called a fingerprint) is defined by a specific set of span attribute values, such as service, route, and status code. Rather than sampling traces purely based on volume, the diversity policy ensures that at least one trace for each fingerprint is retained every 15 minutes.

This makes it easier to explore, debug, and understand services that don’t receive enough traffic to be reliably captured by probabilistic sampling alone.

The diversity policy works alongside other sampling policies and is most commonly used to supplement a global probabilistic sampling policy.

Why use diversity sampling?

Capture rare and low-traffic behavior

Endpoints with low request volume, background jobs, or infrequently executed code paths are often missed by traditional sampling or require manually crafting highly specific policies. The diversity policy ensures these traces are still available for inspection.

Reduce sampling costs safely

Because unique traces are preserved, you can lower your overall sampling rate with confidence, reducing ingest costs without losing visibility into important edge cases.

Improve trace exploration

When browsing services, operations, or routes, you can always find representative example traces, even for endpoints that receive very little traffic.

Preserve error variety

By including attributes such as HTTP status codes, the policy helps ensure different error classes (for example, 4xx and 5xx responses) are captured instead of repeatedly sampling the same error type.

How it works

1. Adaptive Traces examines incoming server and consumer spans and extracts a predefined set of span attributes to describe each trace.

2. These attribute values are combined into a fingerprint, where each fingerprint represents a unique combination of attribute values found in the trace.

The fingerprint is built from the following attributes:

• service.name
• k8s.cluster.name
• k8s.namespace.name
• db.system.name
• db.collection.name
• http.route
• http.response.status_code
• rpc.method
• rpc.grpc.status_code

1. For each fingerprint, the diversity policy ensures that at least one trace is sampled within a fixed time window of 15 minutes. Additional traces with the same fingerprint during that window are not sampled by the diversity policy (other policies may still apply).

2. When the time window resets, the same fingerprints can be sampled again, ensuring continuous visibility into rare, low-traffic, and less-represented behavior.

Your workflow

The diversity policy fits into your Adaptive Traces workflow by ensuring you always have representative traces available, even for low-traffic services, routes, or regions.

Enable diversity sampling

Adaptive Traces automatically suggests a diversity policy when it detects that important traces may be underrepresented.

1. Navigate to Adaptive Traces.

2. Open the Overview page.

3. Review the Sample unique traces (diversity) recommendation.

4. Apply the recommendation to enable diversity sampling.

   After being applied, the policy immediately begins capturing representative traces for less common traffic patterns.

Adjust your sampling strategy

With the diversity policy in place, you can safely reduce other sampling policies:

• Lower your probabilistic sampling rate to reduce ingest volume.
• Remove policies that were previously needed to guarantee minimum trace coverage for specific services, regions, or routes.

Because the diversity policy guarantees at least one trace per unique fingerprint, you maintain visibility while controlling costs.

Explore rare and low-traffic traces

When you browse traces in Explore or Traces:

• Navigate to services or routes with low request volume.
• Open example traces that would otherwise be missing due to low traffic.
• Compare behavior across services, routes, and response types using consistent, representative samples.

This makes it easier to understand edge cases and confirm correct behavior outside of high-traffic paths.

Investigate issues

When troubleshooting:

• Use the captured diversity traces to validate whether an issue is isolated or systemic.
• Inspect different response codes, database interactions, or RPC methods that may only occur intermittently.
• Combine diversity policy traces with anomaly detection or other sampling policies for deeper analysis.

Deactivate diversity sampling

If you no longer need diversity sampling:

1. Navigate to Adaptive Traces.
2. Open the Policies page.
3. Remove the Diversity Policy.

No other configuration changes are required.

What to expect when using diversity policies