Dlacoste esrf/abstract reader draft

[sergirubio]

Hi Damien,

I have corrected some issues on AbstractReader, added a simple main method to call it from shell and added the mariadb_reader as example for implementation.

Tomorrow I can make a little demo during the meeting,

Sergi

[dlacoste-esrf]

Seems really good!
Can we find a way to use the AbstracteReader as the entrypoint for clients ? maybe in the init, replace this YourDb call by calling an implementation module that could be provided as an argument ?

Another thing, for the column selection, can we use a list (that could be column names or indexes btw) instead of a single string ? I find it a bit weird as it is.

I'll start working on the timescaledb implementation on my side and we can start building on this ! That's really good, many thanks!

[sergirubio]

Hi Damien, yes, I think that the AbstractReader can be the default class and then choose the implementation via Tango properties (this is what I do, using a free property called PyTangoArchiving.Schemas where I pass the module for each DB, mariadb, sqlserver, etc) In my PyTangoArchiving implementation, my reader loads all declared implementations and executes .is_attribute_archived() on each of them; and then .get_attribute_values() on the matching one. Would this mechanism suit to you? Sergi

On 2021-09-16 11:12, dlacoste-esrf wrote: Seems really good! Can we find a way to use the AbstracteReader as the entrypoint for clients ? maybe in the _init_, replace this YourDb call by calling an implementation module that could be provided as an argument ? Another thing, for the column selection, can we use a list (that could be column names or indexes btw) instead of a single string ? I find it a bit weird as it is. I'll start working on the timescaledb implementation on my side and we can start building on this ! That's really good, many thanks! -- You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub [1], or unsubscribe [2]. Triage notifications on the go with GitHub Mobile for iOS [3] or Android [4].

Links: ------ [1] #20 (comment) [2] https://github.com/notifications/unsubscribe-auth/AEMQVKKA3GJEVMZ3DHW6A43UCGYIZANCNFSM5EBDO5EA [3] https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 [4] https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub

[dlacoste-esrf]

Sorry for the late answer. I am not sure to get what you want to do…

If we could use something like

import AbstractReader

reader=AbstractReader(mariadb_reader, config={host:"",password:""})

and we just replace mariadb_reader by any implementation of our choice, the import, and error management in case the module does not exist, would be managed at the AbstractReader level. We could rename it as well, it wouldn't be so abstract in the end…

[dlacoste-esrf]

I started working on a timescaledb implementation and it raised a little question.

For the columns, you added this argument so that we can select the columns we want to extract. I am afraid it can be problematic if the column names are implementation related…
I would create an enum or something to represent the common fields (id, time, value_r/w, error_desc, quality factor, …), pass them as a proper list, and leave the naming to the implementation, leaving an extra argument, that can be a string, for custom extra fields. Time might necessit a different treatment, as we use it to order the data, I would suggest we extract it all the time anyways.

Make me think of the error_desc actually, upon extraction should we return the message itself or simply the id ?

Thanks in advance for your feedback

[dlacoste-esrf]

I merged your modifications in my fork and pushed a first timescaledb implementation.