Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(0.14.1) https_total_latencies_bucket value discrepancy between Grafana and GCP #293

Open
Waizzzzz opened this issue Jan 2, 2024 · 1 comment

Comments

@Waizzzzz
Copy link

Waizzzzz commented Jan 2, 2024

Hello, recently we noticed that values exported by the exporter and the ones in GCP don't match, the metric labels (including bucket lengths) match but the values themselves don't, even when identical queries are used within GCP.
Visually the graphs sometimes match (even with wrong values exported) when there are spikes but the data itself still isn't correct.

Query used in Grafana:
histogram_quantile(0.99,sum by (le)(increase(stackdriver_https_lb_rule_loadbalancing_googleapis_com_https_total_latencies_bucket{target_proxy_name=~"**REDACTED**"}[1m])))

Query used in GCP:
histogram_quantile(0.99,sum by **(le)(increase(loadbalancing_googleapis_com:https_total_latencies_bucket{target_proxy_name=~"REDACTED"}[1m])))

@Waizzzzz Waizzzzz changed the title (0.13.0) Bucket value discrepancy between Grafana and GCP (0.13.0) https_total_latencies_bucket value discrepancy between Grafana and GCP Jan 2, 2024
@Waizzzzz Waizzzzz changed the title (0.13.0) https_total_latencies_bucket value discrepancy between Grafana and GCP (0.14.1) https_total_latencies_bucket value discrepancy between Grafana and GCP Jan 3, 2024
@kgeckhart
Copy link
Contributor

👋 I do believe this should be resolved now as of 0.15.1 which included a bugfix for histograms https://github.com/prometheus-community/stackdriver_exporter/releases/tag/v0.15.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants