Scraping fingerprints or CLI print into InfluxDB

[S54Mpower]

Hello guys,

could someone give me a direction how to scrape generated fingerprints into InfuxDB?

Had idea to use Telegraf or Prometheus but struggling with decision what would be the best way to go as I have very little experience with configuring I/O's for those exporters but I would like to try. I was also thinking with getting data from the CLI print but I have two vehicles under account so the tags would be colliding.

[S54Mpower]

Hello again,

managed to do it with the original Telegraf input.file. Before when I tried it wasn't working as I have set data format to be "influx" however Influx expects a string. Later changed it to json and it is working. If anyone would like to use this data to Grafana here below is how I managed to get it working from Telegraf.

# Parse a complete file each interval

[[inputs.file]]
files = ["/home/parallels/vehicle_fingerprint/state_bmw_0.json"]
character_encoding = "utf-8"
tag_keys = ["chargingLevelPercent","range"]
json_name_key = "BMW"
data_format = "json"
file_tag = "bmw"

Now I have to do following:
-change bimmerconnected to not create timestamp folders but just to dump json's to fingerprint folder.
-make a scheduled task that will delete files from fingerprint folder and make a command line request to generate new fignerprints every "xx" minutes.

If anyone has suggestions how to go ahead for this two task please help.

[rikroe]

Just came to my mind: how often do you plan to store this data(or execute the CLI)?

If you do it more often than every 20 minutes, you will run into quota limitations by BMW, limiting you account to a (including the MyBMW app) until the next time 0:00 UTC.

If you want the data more often, I suggest to write a small python script that creates a MyBMWAccount and then get the vehicles and stores it to json every x minutes in a loop.

[S54Mpower]

Hi @rikroe I have tried that but the data format cannot be parsed by telegraf as the status of the tags is coming under "" so Telegraf is returning metric parse error: expected field at 1:18: "[{"data": {"vin":<-- here"

example:
[{"data": {"vin":"xxxxxxxxxxxxxx".

[S54Mpower]

Just came to my mind: how often do you plan to store this data(or execute the CLI)?

If you do it more often than every 20 minutes, you will run into quota limitations by BMW, limiting you account to a (including the MyBMW app) until the next time 0:00 UTC.

If you want the data more often, I suggest to write a small python script that creates a MyBMWAccount and then get the vehicles and stores it to json every x minutes in a loop.

I was thinking to set is to 15 min refresh rate. Would you be kind and briefly tell me what would be difference between the script that you suggested and getting the bimmerconnected status? BMW server would register both as login's?

[rikroe]

Regarding the parsing error in telegraf you should be able to remove the outer quotes using some bash magic - there should be something on google/stackoverflow.

If you follow a pattern like https://github.com/bimmerconnected/bimmer_connected#example-in-non-async-code and keep the MyBMWAccount object, the library is doing less calls to BMW as most of the session data is already available.

A quickly put together script could look like this (not tested):

import asyncio
import json
import time

from bimmer_connected.account import MyBMWAccount
from bimmer_connected.api.regions import Regions

account = MyBMWAccount(USERNAME, PASSWORD, Regions.REST_OF_WORLD)

while True:
    asyncio.run(account.get_vehicles())
    vehicle = account.get_vehicle(VIN)

    with open("/tmp/bmw.json", "wb") as fp:
        json.dump(account.vehicles, cls=MyBMWJSONEncoder)
    
    time.sleep(15*60)

This should also store the JSON correctly. Otherwise it surely could be adjusted to ingest into Influx directly, but I have never used Influx or Telegraf so cannot help you there.

[S54Mpower]

Thank you very much for time to reply. I will play with it and if I come up with success will post here in case others will need.