…ing json files (jbuehl#117)

* Added utility file to take output from semonitor.py and dump to rotating files based on day or hour. No parsing, raw json

* Updated a few items per suggestions to make consistent with other things

* Added utility file to take output from semonitor.py and dump to rotating files based on day or hour. No parsing, raw json

* Updated a few items per suggestions to make consistent with other things

* Added section to se/msg.py to detect a wrong encryption key, and reset the cipher object (while logging)

* Updated se2file.py to allow the user to specify day-based output directories, off by default

* Added utility file to take output from semonitor.py and dump to rotating files based on day or hour. No parsing, raw json

* Updated a few items per suggestions to make consistent with other things

* Added section to se/msg.py to detect a wrong encryption key, and reset the cipher object (while logging)

* Updated se2file.py to allow the user to specify day-based output directories, off by default

* Fixed indent

… key exchange. (jbuehl#121)

…buehl#128)

- xxd is relatively common so don’t use a specialized utility script 
- fix basename flag that was not working on some platforms

…buehl#137)

also, halt tests on any exception

* Correct Power readings of Nan that are sent in reversed byte order

. an edge case where it appears that the solaredge inverter has a minor
  but irritating bug when encoding data for transmission

* Correct inverter Pmax power readings that should be Nan

. an edge case where it appears that the solaredge inverter has a minor
  but irritating bug when encoding data for transmission, namely that
  readings of Nan that are sent in reversed byte order

* Decode battery id to string instead of bytes.

. The battery ID in a meters_0x0030 message is 12 characters long.
  In Python 3.x struct.unpack returns a byte string when the 's'
  format is used. But byte strings are not accepted as keys for
  dictionaries, so it is necessary to decode the battery ID byte string
  to a proper unicode string to avoid problems later on.

* First attempt at providing a test for the fix_byte_strings changes.

. The 'correct' json file was created under Windows, so there may be
  problems with CR LF endings, when tests are run under a Linux shell.

* Fix value returned from parseData when unknown function encountered.

. The previous return value was raising an exception, even though the
  intent is just to ignore the unknown function part of the message.

* Convert Date and Time values in json output to the US/Pacific timezone.

. test/test.sh, as run by CircleCI, is setup to generate Date and Time
   values as per the US/Pacific timezone.  Hence my target output
  json file must also include Date and Time as per that timezone,
  in order to pass the CI tests.

* Add the .rec and .csv files expected by test/test.sh.

* Save csv and rec files created by running test/test.sh on Ubuntu

It's currently broken:

root@firewall:/opt/solaredge# ./semonitor.py -m -t 4 -c 12,H239 -s 7315977D /dev/ttyUSB0
Traceback (most recent call last):
  File "./semonitor.py", line 268, in <module>
    doCommands(args, mode, state, dataFile, recFile, outFile)
  File "./semonitor.py", line 175, in doCommands
    masterGrant(dataFile, recFile, args.slaves[0])
TypeError: masterGrant() missing 1 required positional argument: 'slaveAddr'

Co-authored-by: Ben Castricum <[email protected]>

Tested on 2022-03-12 with a SE6K inverter

The se2influx.py utility will accept semonitor.py input from stdin (allowing
you to use to pipe semonitor.py output into it) and write it to an influxdb instance.

Inverters and optimizers will be written to seperate measurements, with their
unique serial numbers acting as tags, and the rest of the semonitor fields will
be sent verbatim to your influxdb as native fields.

Lastly, se2influx (optionally) allows for writing of ingested semonitor data to
a log file on disk, with built-in log file rotation and compression.  This
allows you to eliminate any uses of "tee" in your piped commands, and permits
you to replay any data that was failed to be sent to your influxdb host (due to
host downtime, etc) at a later date by simplying cat-ing the relevant log
file(s) into the se2influx.py script.

pps.  We also include a "se2influx" script as an example of how you might
want to execute it in conjunction with semonitor.py

Co-authored-by: root <[email protected]>

It is not uncommon to setup smart symlinks for hotpluggable/USB
adapters, so they always have a known name. PySerial however seems to
refuse to follow symlinks

Signed-off-by: Olliver Schinagl <[email protected]>

To be able to track dependencies better, make development easier and
have a more reliable means of distributing and managing semonitor, it is
good to put it isolate it within a container.

Everything should still be useable as before, but care needs to be taken
when running pipelines, as if these are not properly passed into the
container, one is piping the outside of the container. The result would
be the same, but either two docker instances would need to be run or the
second command needs to be available on the host. Both are not ideal
solutions.

Signed-off-by: Olliver Schinagl <[email protected]>