v2.6.0 - Welcome kamon.enabled

Introducing the kamon.enabled setting

You can now toggle Kamon on and off using the kamon.enabled setting! The days of modifying code for test environments or making your own poor human's enabled flag are gone. This setting is considered experimental at the moment because there might be corner cases we are not covering, so please let us know if you find any issues with it! When Kamon is disabled, it wont:

• Start any of the configured reporters
• Apply automatic instrumentation
• Start modules like the status page

You should still call Kamon.init() when starting your application as usual and the kamon.enabled setting will decide whether the call initializes Kamon or basically turns into a noop. Please note that you can't reconfigure Kamon with a different value for kamon.enabled after initializing Kamon. If you need to enable/disable Kamon, you'll have to restart the app with the new config value.

Upgrade Notes:

• If you are using the Kanela agent directly, please upgrade to version 1.0.17
• Look for kamon.enabled settings on your codebase and clean them up! Over the years we have seen folks implementing this behavior themselves and now is time for cleaning that up.

Improvements:

• jdbc: Support for Snowflake JDBC. Contributed by @jatcwang via #1230
• jdbc: Support for PrestoDB JDBC. Contributed by @ivantopo via #1247
• prometheus: Escape backslashes on label values. Contributed by @ivantopo via #1244
• akka: Ensure the cluster sharding instrumentation works with Akka 2.7. Contributed by @ivantopo via #1245
• kafka: Allow W3C-TraceContext propagation through Kafka. Contributed by @Symbianx via #1237
• play: Ensure the Play Framework instrumentation is loaded after Logback to avoid initializing Logback classes too early. Contributed by @ivantopo via #1236
• logback: Allow adding the source thread ID to MDC. Contributed by @nvollmar via #1250
• logback: Rewrite the Logback instrumentation to ensure consistent context propagation to MDC. Contributed by @nvollmar via #1252
• newrelic: Add support for NewRelic's new license key, in addition to the previously supported NR Insights key. Contributed by @bwiercinski and @matwojcik via #1249

v2.5.12 - Maintenance Release

Fixes

• logback: Append context tags/keys to the MDC, even when the MDC is empty. This issue was introduced on 2.5.11. Fixed by @ivantopo via #1227. Many thanks to @scheleaap for providing a reproducible!

Improvements

• all: Bump OkHttp version to 4.10.0. Contributed by @azolotko via #1226
• akka-grpc: Propagage context to service implementations when handling unary requests on the Java API. Contributed by @ivantopo via #1229

v2.5.11 - Maintenance Release

Fixes

• apm: The Kamon APM reporter was not closing OkHttp's response body, leading to a warning about leaked connections. Fixed by @ivantopo via #1224

Improvements:

• redis, caffeine, and annotation: These modules are now published for Scala 3 as well. Contributed by @hughsimpson via #1123 and #1221.

v2.5.10 - Maintenance Release

Fixes:

• logback: #1212 Avoid stackoverflow errors when using Logback 1.3.x / 1.4.x. Fixed by @ivantopo via #1220

Improvements:

• bundle: The bundle now ships with Kanela 1.0.16. Now you can use Kanela on folders that have white spaces on their path.
• apm: Moved to the v2 ingestion API on the Kamon APM reporter. All timestamps on spans (start, finish, and marks) are now reported using the full nanosecond precision available on Kamon Telemetry. Contributed by @ivantopo via #1211

v2.5.9 - Cats Effect 3 Support & Minor Fixes

New Features

• cats-io-3: There is a new kamon-cats-io-3 module with automatic instrumentation for Cats Effect 3 fibers 🎉 This was a huge effort with tons of support from @m50d, @dispalt, @hughsimpson, @cmcmteixeira, @dpsoft, and @ivantopo via #1196. Thank you so much to everyone involved ❤️

Improvements

• jaeger: Bumped jaeger-thrift dependency to address security issues. Contributed by @mvollmar via #1209
• common: Increase default range for the http.server.connection.lifetime metric to a week. Contributed by @ivantopo via #1204
• jvm: Add memory utilization metrics and support ZGC/Shenandoah. Contributed by @ivantopo via #1203
• scala-future and kafka: Make the tests pass and publish artifacts for Scala 3. Contributed by @hughsimpson via #1208

v2.5.8 - Tapir 1.x Support and Fixes

Improvements

• tapir: The Tapir instrumentation is now compatible with Tapir 1.x. Contributed by @Grandys and @hughsimpson via #1188. Thanks for your help!

Fixes

• core: Ensure that ignored operations are taken into account when doing delayed sampling. Fixed by @ivantopo via #1198

v2.5.7 - More Scala 3 Support & Ignoring Operations

Ignoring Metrics and Spans for Specific Operations

We get asked pretty often: how can we disable sampling for health checks and status endpoints? This Kamon Telemetry version comes with a new setting for that:

kamon.trace.ignored-operations = ["/status", "/health", "/ready"]

This has a few advantages over the groups configuration on the adaptive sampler:

• It works with all samplers, including always and random.
• It comes with a track-metrics-on-ignored-operations setting, which allows disabling metrics for these operations too (the adaptive sampler would only disable sampling but still track metrics for all operations).

Improvements

• instrumentation-common: This module is now published for Scala 3 as well. Special thanks to @vaslabs for going the extra mile on that PR 😍. Contributed by @vaslabs via #1191.
• core: Introduce a new setting to ignore operations and disable metrics for them. Contributed by @ivantopo via #1197

v2.5.6 - Experimental Akka Cluster Metrics

Akka Cluster Metrics

This release comes with experimental support for a bunch of Akka Cluster-related metrics! We can break down the metrics into per-state member counts that help you get a general view of the cluster status, and per-member status counts, which let you see exactly which nodes are on each state.

Per-state Member Counts

These metrics track how many cluster members are on each of the possible states in the cluster. The metric names for these states are:

• akka.cluster.members.joining.count
• akka.cluster.members.weakly-up.count
• akka.cluster.members.up.count
• akka.cluster.members.leaving.count
• akka.cluster.members.exiting.count
• akka.cluster.members.down.count
• akka.cluster.members.removed.count
• akka.cluster.members.total.count

All members of the cluster expose these metrics with the number of members they see in each state. Since these metrics are exposed as gauges, you could always make a chart of the max(akka.cluster.members.[state].count) to see how many members are on a given state. For example, for members in the Up state you can see how a cluster starts growing in members:

Per-member Status Counts

The metrics we mentioned above will help you know, for example, that there are 3 members in the Joining state, but what if you want to know whichare those members? That's what the per-member metrics are for. These metrics have a member tag that identifies the member being measured, and the value of the metric tells you the state it is currently in. Here is a breakdown of the possible values:

akka.cluster.member.status

• Joining = 1
• WeaklyUp = 2
• Up = 3
• Leaving = 4
• Exiting = 5
• Down = 6
• Removed = 7

akka.cluster.member.reachability

• Reachable = 0
• Unreachable = 1

The same as with the status metrics, you can plot the max(akka.cluster.members.status) and see what's the latest on each member of the cluster! In the example below, all members are on state 3 (Up):

Enabling Cluster Metrics

This feature isn't fully tested yet so we are not enabling it by default. To enable Akka Cluster Metrics, add this setting to your application.conf file:

kamon.instrumentation.akka.cluster.track-cluster-metrics=yes

New Features

• akka: Publish Akka Cluster metrics for members' status and reachability. Contributed by @ivantopo via #1189

v2.5.5 - ClassLoading Improvements for Play Framework

Improvements

• play: We are now using the Play Environment's class loader to create all Kamon-related objects. This ensures that custom classes (like samples or SlowStatementProcessors) can be loaded from the classpath when running on development mode. Contributed by @ivantopo via #1185

v2.5.4 - Maintenance Update

Fixes

• bundle: The new kamon-aws-sdk module had a Compile dependency on the Dynalite test container that was brining a bunch of extra classes into the Bundle and messing with user-provided dependencies 🤦. Reported by @fggarcia on #1171 and fixed by @ivantopo via #1178
• aws-sdk: Duplicate execution attribute definitions when using more than one AWS SDK Client. Reported by @hhalex on #1173 and fixed by @ivantopo on #1175