Influxdb deletes too much

[genofire]

Config

[influxdb]
enable   = true
address  = "http://localhost:8086"
database = "ffhb"
username = ""
password = ""
# cleaning data of measurement node,
#   which are older than 7d
delete_after = "7d"
#   how often run the cleaning
delete_interval = "1h"

Version
016e2dc

Description
Influxdb lost more then all data after 7 days :(
But this habben just on some nodes not on every one.

[genofire]

We did not handle by any nodes, we use the complete measurement look

yanic/database/database.go

Line 60 in 1bdfa29

db.client.Query(client.NewQuery(query, db.config.Influxdb.Database, "m"))

[genofire]

Issues at influxdb was closed - should be looking at and maybe closed nearly

[valcryst]

Hi, we still have problems with this issue running lastest influx > 1.4.0~n201710310800
(yanic is configured to held 365 days)
https://map.ff-en.de/graph/dashboard/db/freifunk-en-router?refresh=1m&orgId=1&from=now-30d&to=now

Why yanic does not use the build-in mechanism "RETENTION POLICY" ?
It`s highly suggested by the devs todo so.

Regards
Matthias

[genofire]

Thx for the feedback.

If it read the documentation it sound like that a "RETENTION POLICY" is for a hole database and not for a specific measurement.
(We want to have the data in the global measurement till we die.)

Or i am wrong?

[valcryst]

Sorry for the delay.
The lastest InfluxDB works perfect without deleting too much.

actualy using: InfluxDB shell version: 1.5.0~n201711120800

Yanic is configured to prune after 60d, before this new InfluxDB version it deleted everything after 14~ days.
https://map.ff-en.de/graph/dashboard/db/freifunk-en-router?refresh=1m&orgId=1&from=now-30d&to=now

This is very satisfying after this long time :)

@genofire You´re correct with this, it`s defined for one database.
How about moving "global" to an seperate database, since its not used
together with regular "node" queries?
Dont get me wrong (no critism) but i would realy like to use fully supported build-in functions.

Regards
Matthias

[ghost]

Now it doesn't delete anything anymore. The nodes config is:

[nodes]
# Cache file
# a json file to cache all data collected directly from respondd
state_path    = "/var/lib/yanic/state.json"
# prune data in RAM, cache-file and output json files (i.e. nodes.json)
# that were inactive for longer than
prune_after   = "7d"
# Export nodes and graph periodically
save_interval = "5s"
# Set node to offline if not seen within this period
offline_after = "10m"

But I still have the data of nodes from end of june in influxdb

[genofire]

@CharlemagneLasse thank you for the feedback - it still seen to be a bug in influxdb.
Which version of influxdb are you using?

[ghost]

1.5.3. But it is actually a bug in your issue report. The deletion rate is configured using "[database]" and not "[nodes]" as you said in your issue report.

[database]
# this will send delete commands to the database to prune data
# which is older than:
delete_after    = "30d"
# how often run the cleaning
delete_interval = "1h"

[genofire]

@CharlemagneLasse You was right, it was not paste the right content of the config file. Is now all okay?

[ghost]

Yes

[maurerle]

I think this issue can be closed?