monitor.berlin.freifunk.net Performance optimization

[nicolasberens]

The current state, especially with collectd is a bit suboptimal.

This is a draft to plan what can be done

There are a few things i am not happy with:

• routers send data every 30 seconds
• collectd works internally with a 10 second interval.
• therefore all RRD files are created with a 10 second step size
• RRDfiles only support one interval, so we need to recreate them or write a script that does it
• prometheus scrapes it every 15 seconds

To test:

• can we reduce the step size of files ?
• does it save data/writes?
• do we maybe want to drop some datapoints?
• does a higher interval for RRD files make sense? (from 30 to 60 seconds?)

Planned Goal:

• adjust settings where possible without big migrations
  • => prometheus
• rewrite the collectd config to support chains: https://github.com/collectd/collectd/wiki/Chains
  • test if its possible to change the intervall per metric
  • migrate metrics 1:1 (e.g. cange interval of cpu load, rewrite rrd files, ten change the next metric)

[nicolasberens]

using: https://github.com/AndrewPiroli/rrd-step-changer

and
for FILE in $(find . -name "*.rrd" -type f); do rrdtool dump $FILE > $FILE.tmp ; python3 rrdstep.py $FILE.tmp $FILE.small.tmp 30 60 ; rrdtool restore -f $FILE.small.tmp $FILE; done

i cloned data from monitor (10sec interval) and hds-ffraum (10sec step but data comes in every 30 seconds)

https://monitor.berlin.freifunk.net/cgp/host.php?h=monitor.berlin.freifunk.net
https://monitor.berlin.freifunk.net/cgp/host.php?h=monitor.berlin.freifunk.net-nicolastest
https://monitor.berlin.freifunk.net/cgp/host.php?h=hds-ffraum
https://monitor.berlin.freifunk.net/cgp/host.php?h=hds-ffraum-nicolastest

Looking at the data it seems like its not working propperly, and ending up stretching the data again