The analytics-consumer is a Java Consumer<Message<?>> that computes analytics from the input data messages and publishes them as metrics to various monitoring systems.
It leverages the micrometer library for providing a uniform programming experience across the most popular monitoring systems and uses Spring Expression Language (SpEL) for defining how the metric names, values and tags are computed from the input data.
The analytics-consumer can produce two metrics types:
-
Counter - reports a single metric, a count, that increments by a fixed, positive amount. Counters can be used for computing the rates of how the data changes in time.
-
Gauge - reports the current value. Typical examples for gauges would be the size of a collection or map or number of threads in a running state.
A Meter (e.g. Counter or Gauge) is uniquely identified by its name and dimensions (the term dimensions and tags is used interchangeably). Dimensions allow a particular named metric to be sliced to drill down and reason about the data.
|
Note
|
As a metrics is uniquely identified by its name and dimensions, you can assign multiple tags (e.g. key/value pairs) to every metric, but you cannot randomly change those tags afterward!
Monitoring systems such as Prometheus will complain if a metric with the same name has different sets of tags.
|
Add the analytics-consumer dependency to your POM:
<dependency>
<groupId>org.springframework.cloud.fn</groupId>
<artifactId>analytics-consumer</artifactId>
<version>1.0.0-SNAPSHOT</version>
</dependency>The AnalyticsConsumerConfiguration auto-configures the following consumer bean:
Consumer<Message<?>> analyticsConsumerFor every input Message the analyticsConsumer computes the defined metrics and eventually, with the help of the micrometer library, publishes them to the backend monitoring systems. The Message is a generic container for data. Each Message instance includes a payload and headers containing user-extensible properties as key-value pairs. Any object can be provided as the payload.
The MessageBuilder helps to create a Message instance from any payload content and assign any key/value as a header:
Message<String> myMessage = MessageBuilder
.withPayload("My message text")
.setHeader("kind", "CUSTOM")
.setHeader("foo", "bar")
.build();The SpEL expressions use the headers and payload keywords to access message’s headers and payload values. For example a counter metrics can have a value amount computed from the size of the input message payload add a my_tag tag, extracted from the kind header value:
analytics.amount-expression=payload.lenght()
analytics.tag.expression.my_tag=headers['kind']Review the AnalyticsConsumerProperties javadocs for further details how to use the SpEL properties.
By default, Micrometer is packed with a SimpleMeterRegistry that holds the latest value of each meter in memory and doesn’t export the data anywhere.
To enable support for another monitoring system you have to add the spring-boot-starter-actuator dependency and the micrometer dependency of the monitoring system of choice:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-[MONITORING SYSTEM NAME]</artifactId>
</dependency>Follow the configuration instructions for the selected monitoring system.
All monitoring configuration properties start with a prefix: management.metrics.export.
All analytics-consumer configuration properties use the analytics prefix.
For more information on the various options available, please see AnalyticsConsumerProperties.
All monitoring configuration properties start with a prefix management.metrics.export.
Following examples show how to configure counter and gauge metrics over a series of stock-exchange messages like this:
{
"data": {
"symbol": "AAPL",
"exchange": "XNAS",
"open": 318.66,
"close": 316.85,
"volume": 25672211.0
}
}The following configuration will create a counter metrics called stockrates with two tags: symbol and exchange computed from the json fields:
| Property | Description |
|---|---|
|
Counter meter type (default) |
|
Metrics name |
|
Add tag |
|
Add tag |
Now you can use the stockrates metrics to measure the rates at which the stock transactions occur over a given time interval.
Furthermore, you can aggregate those rates by the symbol and exchange tags.
To measure the transaction volumes contained in the data.volume JSON fields, you can build a GAUGE metrics like this:
| Property | Description |
|---|---|
|
Gauge meter type |
|
Metrics name |
|
Add tag |
|
Add tag |
|
Set the Gauge to the |
Then use the stockvolumes metrics to graph, in real-time, the transaction volumes changes over time. You can aggregate those volumes by the symbol and exchange tags.
|
Warning
|
Micrometer implements the Gauges for the purpose of data sampling! There is no information about what might have occurred between two consecutive samples. Any intermediate values set on a gauge are lost by the time the gauge value is reported to a metrics backend. |
To enable one or more supported monitoring systems you need to add a configuration like this:
| Property | Description |
|---|---|
|
Enable or disable the monitoring system. (enabled by default). |
|
UIR of your Wavefront server or Wavefront Proxy. |
|
Wavefront access token. |
|
The |
See this test suite for the various ways, this consumer is used.
-
See the Analytics Sink README where this consumer is used to create a Spring Cloud Stream application where it makes a Counter sink.