Skip to content

Commit

Permalink
NMS-16033: Update docs on default RRD
Browse files Browse the repository at this point in the history
  • Loading branch information
mmahacek committed Jul 31, 2023
1 parent 6e690d2 commit ad03170
Show file tree
Hide file tree
Showing 3 changed files with 45 additions and 34 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,9 @@ The following tags are also available for an interface filter:

[cols="1,3,3"]
|===
| Tag | Description | Example
| Tag
| Description
| Example

| specific
| Specify an actual IP address to include in the package.
Expand Down Expand Up @@ -70,6 +72,11 @@ The following example illustrates collector package attributes that use some of
<3> Evaluate IPv4 rule to collect for all IPv4 interfaces in the given range.
<4> Evaluate IPv6 rule to collect for all IPv6 interfaces in the given range.

[[ga-rrd-retention]]
== Default metric retention

include::../../retention.adoc[]

[[ga-collectd-packages-services]]
== Service configuration attributes

Expand Down Expand Up @@ -104,7 +111,8 @@ For a list of collector-specific parameters and their default values, refer to t
.Common service attributes
[options="autowidth"]
|===
| Attribute | Description
| Attribute
| Description

| name
| Service name
Expand Down
34 changes: 34 additions & 0 deletions docs/modules/operation/pages/deep-dive/retention.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
When using RRD/JRB storage, data is stored in the file system using a RRD-type strategy.
With this "round robin" storage, data is kept in aggregated time buckets, with the latest bucket overwriting the oldest bucket, and multiple collection within the time period for a bucket will cause the data to be consolidated based on the type of bucket.

The aggregation buckets are defined in various config files based on a `<RRD>...</RRD>` block.

.Default RRD config
[source, xml]
----
<rrd step="300"><1>
<rra>RRA:AVERAGE:0.5:1:2016</rra><2>
<rra>RRA:AVERAGE:0.5:12:1488</rra><3>
<rra>RRA:AVERAGE:0.5:288:366</rra><4>
<rra>RRA:MAX:0.5:288:366</rra><5>
<rra>RRA:MIN:0.5:288:366</rra><6>
</rrd>
----
<1> Step of 300 seconds (5 minutes) per bucket.
This value should match the interval of the services in the package.
<2> Store the average of 1 bucket for 2016 buckets.
5 minute averages kept for 7 days.
<3> Store the average of 12 buckets for 1488 buckets.
1 hour average kept for 62 days.
<4> Store the average of 288 buckets for 366 buckets.
1 day average kept for 366 days.
<5> Store the maximum of 288 buckets for 366 buckets.
1 day max for kept 366 days.
<6> Store the minimum of 288 buckets for 366 buckets.
1 day min for kept 366 days.

IMPORTANT: If the RRD definition is modified after data has been collected, the existing RRD/JRB files on disk must manually be deleted so they can be recreated with the updated definition.

When using another storage strategy, such as Newts, Cortex, or other integration, all collected data points are persisted without aggregation.
These points will be kept until the configured TTL expires.
These storage strategies will ignore any defined `<RRA>` definitions in favor of the TTL value.
Original file line number Diff line number Diff line change
Expand Up @@ -38,38 +38,7 @@ You can also exclude IP interfaces:

== Response time configuration

The definition of polling packages lets you configure similar services with different polling intervals.
All the response time measurements are persisted in RRD files and require a definition.
Each polling package contains an RRD definition:

.RRD configuration for polling package
[source, xml]
----
<package name="example1">
<filter>IPADDR != '0.0.0.0'</filter>
<include-range begin="1.1.1.1" end="254.254.254.254" />
<include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff" />
<rrd step="300"> <1>
<rra>RRA:AVERAGE:0.5:1:2016</rra> <2>
<rra>RRA:AVERAGE:0.5:12:1488</rra> <3>
<rra>RRA:AVERAGE:0.5:288:366</rra> <4>
<rra>RRA:MAX:0.5:288:366</rra> <5>
<rra>RRA:MIN:0.5:288:366</rra> <6>
</rrd>
----

<1> Polling interval for all services in this polling package is reflected in the step size of 300 seconds.
All services in this package must be polled on a 5-minute interval, otherwise response time measurements are not persisted correctly.
<2> 1 step size is persisted 2016 times: 1 * 5 min * 2016 = 7 d, 5 min accuracy for 7 d.
<3> 12 steps average persistence 1488 times: 12 * 5 min * 1488 = 62 d, aggregated to 60 min for 62 d.
<4> 288 steps average persistence 366 times: 288 * 5 min * 366 = 366 d, aggregated to 24 h for 366 d.
<5> 288 steps maximum from 24 h persisted for 366 d.
<6> 288 steps minimum from 24 h persisted for 366 d.

WARNING: The RRD configuration and the service polling interval must be aligned.
In other cases, the persisted response time data is not correctly displayed in the response time graph.

IMPORTANT: If you change the polling interval afterwards, you must recreate existing RRD files with the new definitions.
include::../retention.adoc[]

== Service status persistence

Expand Down

0 comments on commit ad03170

Please sign in to comment.