unhashable type: 'dict' #2
Comments
|
Hi @aerohygenx and @peekjef72, apologies if this might be the wrong guess, but I believe the reason could be that more recent versions of Grafana populate the We had to fix With kind regards, |
|
Hi @amotl , |
|
Hi @amotl , @aerohygenx Andreas was right ! the pb was with datasources return type. |
|
@aerohygenx when you have tested, tell me it is ok to close this issue. |
|
Hi Jean-Francois,
Thanks for the update.
I plan to run the test tomorrow Monday; I’ll let you know my results as soon as possible.
Regards,
Juan
From: Pik Jean-Francois ***@***.***>
Date: Sunday, September 11, 2022 at 10:24 AM
To: peekjef72/grafana-snapshots-tool ***@***.***>
Cc: Juan Ore ***@***.***>, Mention ***@***.***>
Subject: Re: [peekjef72/grafana-snapshots-tool] unhashable type: 'dict' (Issue #2)
@aerohygenx<https://github.com/aerohygenx> when you have tested, tell me it is ok to close this issue.
—
Reply to this email directly, view it on GitHub<#2 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A26SITJGVRVBADUQNVMPFUDV5XTRLANCNFSM6AAAAAAQHB7RSA>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
|
Good Day Jean_Francois,
I’m getting the following error:
```
grafana-snapshots -d "RRM2-sandbox" -F "General"
datasources OK.
panel: {'datasource': {'type': 'influxdb', 'uid': '5HyKKpank'}, 'description': 'the business id: ${business}\nthe account is: ${location}', 'fieldConfig': {'defaults': {'color': {'mode': 'palette-classic'}, 'custom': {'axisCenteredZero': False, 'axisColorMode': 'text', 'axisLabel': '', 'axisPlacement': 'auto', 'barAlignment': 0, 'drawStyle': 'line', 'fillOpacity': 0, 'gradientMode': 'none', 'hideFrom': {'legend': False, 'tooltip': False, 'viz': False}, 'lineInterpolation': 'linear', 'lineWidth': 1, 'pointSize': 5, 'scaleDistribution': {'type': 'linear'}, 'showPoints': 'auto', 'spanNulls': False, 'stacking': {'group': 'A', 'mode': 'none'}, 'thresholdsStyle': {'mode': 'off'}}, 'mappings': [], 'thresholds': {'mode': 'absolute', 'steps': [{'color': 'green', 'value': None}, {'color': 'red', 'value': 80}]}, 'unit': 's'}, 'overrides': []}, 'gridPos': {'h': 8, 'w': 14, 'x': 0, 'y': 0}, 'id': 6, 'options': {'legend': {'calcs': ['count'], 'displayMode': 'list', 'placement': 'bottom', 'showLegend': True}, 'tooltip': {'mode': 'single', 'sort': 'none'}}, 'pluginVersion': '9.1.0-76166', 'targets': [{'datasource': {'type': 'influxdb', 'uid': '5HyKKpank'}, 'query': 'from(bucket: "testjson2")\n\t|> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r["_measurement"] == "rrm2")\n |> filter(fn: (r) => r["business"] == "${business}")\n |> filter(fn: (r) => r["location"] == "${location}")\n |> filter(fn: (r) => r["_field"] == "space" or r["_field"] == "runDuration")\n |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") \n |> group(columns: [], mode:"by") \n |> drop(columns: ["_measurement", "topic", "location", "business", "uuid", "_start", "_stop"])\n |> filter(fn: (r) => if "${space}" != "ALL" then r["space"] == "${space}" else true)\n', 'refId': 'A'}], 'title': 'test sanitization duration for: ${space}', 'type': 'timeseries'}
dtsrc: {'type': 'influxdb', 'uid': '5HyKKpank'}
target: [{'datasource': {'type': 'influxdb', 'uid': '5HyKKpank'}, 'query': 'from(bucket: "testjson2")\n\t|> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r["_measurement"] == "rrm2")\n |> filter(fn: (r) => r["business"] == "${business}")\n |> filter(fn: (r) => r["location"] == "${location}")\n |> filter(fn: (r) => r["_field"] == "space" or r["_field"] == "runDuration")\n |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") \n |> group(columns: [], mode:"by") \n |> drop(columns: ["_measurement", "topic", "location", "business", "uuid", "_start", "_stop"])\n |> filter(fn: (r) => if "${space}" != "ALL" then r["space"] == "${space}" else true)\n', 'refId': 'A'}]
target expr is not defined: skipped!
panel: {'datasource': {'type': 'influxdb', 'uid': '5HyKKpank'}, 'description': 'the business id: ${business}\nthe account is: ${location}', 'fieldConfig': {'defaults': {'color': {'mode': 'thresholds'}, 'custom': {'align': 'center', 'displayMode': 'color-text', 'inspect': False}, 'mappings': [], 'thresholds': {'mode': 'absolute', 'steps': [{'color': 'green', 'value': None}, {'color': 'red', 'value': 80}]}}, 'overrides': [{'matcher': {'id': 'byName', 'options': '_start'}, 'properties': [{'id': 'custom.hidden', 'value': True}]}, {'matcher': {'id': 'byName', 'options': '_stop'}, 'properties': [{'id': 'custom.hidden', 'value': True}]}, {'matcher': {'id': 'byName', 'options': 'space'}, 'properties': [{'id': 'custom.hidden', 'value': True}]}, {'matcher': {'id': 'byName', 'options': 'runDuration'}, 'properties': [{'id': 'displayName', 'value': 'duration'}, {'id': 'unit', 'value': 's'}]}, {'matcher': {'id': 'byName', 'options': '_time'}, 'properties': [{'id': 'displayName', 'value': 'time'}]}]}, 'gridPos': {'h': 9, 'w': 12, 'x': 0, 'y': 8}, 'id': 2, 'options': {'footer': {'fields': '', 'reducer': ['count'], 'show': False}, 'showHeader': True, 'sortBy': [{'desc': True, 'displayName': 'time'}]}, 'pluginVersion': '9.1.3-5f01ebf', 'targets': [{'datasource': {'type': 'influxdb', 'uid': '5HyKKpank'}, 'query': 'from(bucket: "testjson2")\n\t|> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r["_measurement"] == "rrm2")\n |> filter(fn: (r) => r["business"] == "${business}")\n |> filter(fn: (r) => r["location"] == "${location}")\n |> filter(fn: (r) => r["_field"] == "space" or r["_field"] == "runDuration")\n |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") \n |> group(columns: [], mode:"by") \n |> drop(columns: ["_measurement", "topic", "account", "business", "uuid", "_start", "_stop"])\n |> filter(fn: (r) => if "${space}" != "ALL" then r["space"] == "${space}" else true)\n', 'refId': 'A'}], 'title': 'test sanitization dates: ${space}', 'type': 'table'}
dtsrc: {'type': 'influxdb', 'uid': '5HyKKpank'}
target: [{'datasource': {'type': 'influxdb', 'uid': '5HyKKpank'}, 'query': 'from(bucket: "testjson2")\n\t|> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r["_measurement"] == "rrm2")\n |> filter(fn: (r) => r["business"] == "${business}")\n |> filter(fn: (r) => r["location"] == "${location}")\n |> filter(fn: (r) => r["_field"] == "space" or r["_field"] == "runDuration")\n |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") \n |> group(columns: [], mode:"by") \n |> drop(columns: ["_measurement", "topic", "account", "business", "uuid", "_start", "_stop"])\n |> filter(fn: (r) => if "${space}" != "ALL" then r["space"] == "${space}" else true)\n', 'refId': 'A'}]
target expr is not defined: skipped!
Traceback (most recent call last):
File "/Users/aerojuan/.pyenv/versions/3.9.10/bin/grafana-snapshots", line 8, in <module>
sys.exit(main())
File "/Users/aerojuan/.pyenv/versions/3.9.10/lib/python3.9/site-packages/grafana_snapshots/cli.py", line 275, in main
res = data_api.get_dashboard_data()
File "/Users/aerojuan/.pyenv/versions/3.9.10/lib/python3.9/site-packages/grafana_snapshots/grafanaData.py", line 426, in get_dashboard_data
, 'transformations': panel['transformations']
KeyError: 'transformations'
```
From: Juan Ore ***@***.***>
Date: Sunday, September 11, 2022 at 3:49 PM
To: peekjef72/grafana-snapshots-tool ***@***.***>, peekjef72/grafana-snapshots-tool ***@***.***>
Cc: Mention ***@***.***>
Subject: Re: [peekjef72/grafana-snapshots-tool] unhashable type: 'dict' (Issue #2)
Hi Jean-Francois,
Thanks for the update.
I plan to run the test tomorrow Monday; I’ll let you know my results as soon as possible.
Regards,
Juan
From: Pik Jean-Francois ***@***.***>
Date: Sunday, September 11, 2022 at 10:24 AM
To: peekjef72/grafana-snapshots-tool ***@***.***>
Cc: Juan Ore ***@***.***>, Mention ***@***.***>
Subject: Re: [peekjef72/grafana-snapshots-tool] unhashable type: 'dict' (Issue #2)
@aerohygenx<https://github.com/aerohygenx> when you have tested, tell me it is ok to close this issue.
—
Reply to this email directly, view it on GitHub<#2 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A26SITJGVRVBADUQNVMPFUDV5XTRLANCNFSM6AAAAAAQHB7RSA>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
|
Hi Jean-Francois, |
|
Hi Jean-Francois, grafana-snapshots -d "RRM2-onepanel" -F "General" -f "now-30d" panel: {'datasource': {'type': 'influxdb', 'uid': '5HyKKpank'}, 'description': 'the business id: ${business}\nthe account is: ${location}', 'fieldConfig': {'defaults': {'color': {'mode': 'palette-classic'}, 'custom': {'axisCenteredZero': False, 'axisColorMode': 'text', 'axisLabel': '', 'axisPlacement': 'auto', 'barAlignment': 0, 'drawStyle': 'line', 'fillOpacity': 0, 'gradientMode': 'none', 'hideFrom': {'legend': False, 'tooltip': False, 'viz': False}, 'lineInterpolation': 'linear', 'lineWidth': 1, 'pointSize': 5, 'scaleDistribution': {'type': 'linear'}, 'showPoints': 'auto', 'spanNulls': False, 'stacking': {'group': 'A', 'mode': 'none'}, 'thresholdsStyle': {'mode': 'off'}}, 'mappings': [], 'thresholds': {'mode': 'absolute', 'steps': [{'color': 'green', 'value': None}, {'color': 'red', 'value': 80}]}, 'unit': 's'}, 'overrides': []}, 'gridPos': {'h': 8, 'w': 14, 'x': 0, 'y': 0}, 'id': 6, 'options': {'legend': {'calcs': ['count'], 'displayMode': 'list', 'placement': 'bottom', 'showLegend': True}, 'tooltip': {'mode': 'single', 'sort': 'none'}}, 'pluginVersion': '9.1.0-76166', 'targets': [{'datasource': {'type': 'influxdb', 'uid': '5HyKKpank'}, 'query': 'from(bucket: "testjson2")\n\t|> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r["_measurement"] == "rrm2")\n |> filter(fn: (r) => r["business"] == "${business}")\n |> filter(fn: (r) => r["location"] == "${location}")\n |> filter(fn: (r) => r["_field"] == "space" or r["_field"] == "runDuration")\n |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") \n |> group(columns: [], mode:"by") \n |> drop(columns: ["_measurement", "topic", "location", "business", "uuid", "_start", "_stop"])\n |> filter(fn: (r) => if "${space}" != "ALL" then r["space"] == "${space}" else true)\n', 'refId': 'A'}], 'title': 'test sanitization duration for: ${space}', 'type': 'timeseries'} OK: new snapshot 'RRM2-onepanel_202209121250' created. |
|
here is the json file generated from the export: { |
|
sorry for the delay. Hard week... |
I think you can expect every data source type to have a different behavior here (see references below), however there are some overlaps. I've exercised this aspect thoroughly when aiming to cover each data source type on submitting a health-check query to. References
|
|
Hi Andreas
Thank you.
I've checked your code and make à lot of tests... things are quite more
complicated than I though...
I plan to make some emprovements to grafana_client model.py query_factory
and datasource/smartquery().
Could you have a look to my fork please ?
Le lun. 19 sept. 2022, 15:17, Andreas Motl ***@***.***> a
écrit :
… It seems that targets are different for influxDB than for Prometheus. I
will try with a panel querying a PostgreSQL, maybe it will have the same
behavior.
I think you can expect every data source type to have a different behavior
here, however there *are* some overlaps. I've exercised this aspect
thoroughly when aiming to cover each data source type on submitting a
health-check query to.
References
- grafana-toolbox/grafana-client#19
<grafana-toolbox/grafana-client#19>
- grafana-toolbox/grafana-client#21
<grafana-toolbox/grafana-client#21>
- grafana-toolbox/grafana-client#27
<grafana-toolbox/grafana-client#27>
—
Reply to this email directly, view it on GitHub
<#2 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AQGB3MNUZLFQ4C5ADA5PXTDV7BRXBANCNFSM6AAAAAAQHB7RSA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Dear Jean-Francois,
You want me to have a look at grafana-toolbox/grafana-client@main...peekjef72:grafana-client:main, right? I think you are aiming to generalize the data query interface beyond doing health-check queries only? Thanks a stack for working on those important details to fill the corresponding gaps to make that possible. Because my iteration time was constrained by other obligations, I was sad to leave those gaps in the code base, so I really appreciate it to see you picking up the torch on it. I think the best way to work on that would be that you submit a pull request, so we can discuss the implementation there, and to stop hijacking this conversation. I am very much looking forward to your improvements. Please bear with me if I may take some time to respond. When submitting the patch, please make sure to delete the Thank you in advance and with kind regards, |
|
Dear Jean-Francois, thanks a stack for submitting grafana-toolbox/grafana-client#38. I will need some time to review and respond, so I am asking for your understanding. With kind regards, |
|
Sure, no problem.
It is a kind of first try. To start a discussion about this subject.
Le mar. 20 sept. 2022, 12:59, Andreas Motl ***@***.***> a
écrit :
… Dear Jean-Francois,
thanks a stack for submitting grafana-toolbox/grafana-client#38
<grafana-toolbox/grafana-client#38>. I will need some
time to review and respond, so I am asking for your understanding.
With kind regards,
Andreas.
—
Reply to this email directly, view it on GitHub
<#2 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AQGB3MLXYJNHOC6FGIX3FM3V7GKKTANCNFSM6AAAAAAQHB7RSA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Hi aerohygenx, Sorry for the delay. Could you please try the devs made in 0.3.0 branch on your dashboard/datasource ? For that you must download locally booth 0.3.0 branch and grafana-client Pull Request. export PYTHONPATH="PYTHONPATH:~/python/grafana-client:~/python/grafana-snapshots-tool"and use python3 grafana-snapshots-tool/local_cli.py with standard arguments. python3 local_cli.py --verbose -b ~/grafana_snapshots/ -d [dashboard_name] |
|
Hi again, on your last note, I would like to add an alternative way to install both packages within a virtualenv, using Setuppython3 -m venv .venv
source .venv/bin/activate
pip install git+https://github.com/peekjef72/[email protected]
pip install git+https://github.com/peekjef72/grafana-client@mainUsageWith kind regards, P.S.: Sorry again for not reviewing your PR grafana-toolbox/grafana-client#38 properly yet. I hope to be able to catch some time when we are back from our travels. |
|
Hi again, your improvements from grafana-toolbox/grafana-client#38 have been merged and released using grafana-toolbox/grafana-client#112. Thanks again, and apologies for the long delay. Maybe it can help on the matters discussed here? Please let me know if you need further support, by adding more patches to With kind regards, |
|
Hi Andrea, Thanks for your actions. JFPIK- |
|
Dear Jean-Francois, do you think it makes sense to integrate your improvements from the 0.3.0 branch into mainline? If you think yes, let me know if you a) appreciate a review and b) if you want me to run the subsequent release 1. With kind regards, Footnotes
|
|
Hi Andreas,
Hope you are fine.
Yes, it makes sense but with restrictions :
- pro: it is fully operational for standard datasources (prometheus, loki,
maybe sql but not sure it has been tested), and for standard panels
(timeseries, status, table)
- against:the table panels use most of the time "transformations" and most
of them are not implemented.
- pro (again): the current (master) version doesn't work on the current
version of grafana anyway !
So when I have tested in real life on a dashboard (the one that was at the
origin of the tool!), the snapshot has failed because of transformations.
:😞
For me this version 0.3.0 is an alpha.
So, yes It would be nice if you could spend time on source code review, and
maybe perform some tests or increase the scope of the test functions.
But there is still more work to arrive at a final version.
JFPIK-
Le sam. 1 juin 2024 à 01:55, Andreas Motl ***@***.***> a
écrit :
… Dear Jean-Francois,
do you think it makes sense to integrate your improvements from the 0.3.0
branch into mainline? If you think yes, let me know if you a) appreciate a
review and b) if you want me to run the release, after applying a few
maintenance tasks here and there.
With kind regards,
Andreas.
—
Reply to this email directly, view it on GitHub
<#2 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AQGB3MJEGWJFB4PXWGOPZTTZFEEXZAVCNFSM6AAAAAAQHB7RSCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBTGEZTANBWGU>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Thanks for your swift reply. So, I am slotting it into my "medium priority" box, to be implemented when time permits, or if you will approach testing and functional finalizing, and ping me about that it would be time to have another release. Actually, grafana-import also needs a release ;]. I approached it recently, but was distracted by other obligations again. |
Hi Andreas, |
Hi,
I got the following error
`grafana-snapshots -d "dash-sandbox"
datasources OK.
panel: {'datasource': {'type': 'influxdb', 'uid': '5HyLKpank'}, 'description': 'the business id: ${business}\nthe account is: ${location}', 'fieldConfig': {'defaults': {'color': {'mode': 'palette-classic'}, 'custom': {'axisCenteredZero': False, 'axisColorMode': 'text', 'axisLabel': '', 'axisPlacement': 'auto', 'barAlignment': 0, 'drawStyle': 'line', 'fillOpacity': 0, 'gradientMode': 'none', 'hideFrom': {'legend': False, 'tooltip': False, 'viz': False}, 'lineInterpolation': 'linear', 'lineWidth': 1, 'pointSize': 5, 'scaleDistribution': {'type': 'linear'}, 'showPoints': 'auto', 'spanNulls': False, 'stacking': {'group': 'A', 'mode': 'none'}, 'thresholdsStyle': {'mode': 'off'}}, 'mappings': [], 'thresholds': {'mode': 'absolute', 'steps': [{'color': 'green', 'value': None}, {'color': 'red', 'value': 80}]}, 'unit': 's'}, 'overrides': []}, 'gridPos': {'h': 8, 'w': 14, 'x': 0, 'y': 0}, 'id': 6, 'options': {'legend': {'calcs': ['count'], 'displayMode': 'list', 'placement': 'bottom', 'showLegend': True}, 'tooltip': {'mode': 'single', 'sort': 'none'}}, 'pluginVersion': '9.1.0-76166', 'targets': [{'datasource': {'type': 'influxdb', 'uid': '5HyLKpank'}, 'query': 'from(bucket: "testbucket")\n\t|> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r["_measurement"] == "rrm2")\n |> filter(fn: (r) => r["business"] == "${business}")\n |> filter(fn: (r) => r["location"] == "${location}")\n |> filter(fn: (r) => r["_field"] == "space" or r["_field"] == "runDuration")\n |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") \n |> group(columns: [], mode:"by") \n |> drop(columns: ["_measurement", "topic", "location", "business", "uuid", "_start", "_stop"])\n |> filter(fn: (r) => if "${space}" != "ALL" then r["space"] == "${space}" else true)\n', 'refId': 'A'}], 'title': 'test sanitization duration for: ${space}', 'type': 'timeseries'}
dtsrc: {'type': 'influxdb', 'uid': '5HyLKpank'}
target: [{'datasource': {'type': 'influxdb', 'uid': '5HyLKpank'}, 'query': 'from(bucket: "testbucket")\n\t|> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r["_measurement"] == "rrm2")\n |> filter(fn: (r) => r["business"] == "${business}")\n |> filter(fn: (r) => r["location"] == "${location}")\n |> filter(fn: (r) => r["_field"] == "space" or r["_field"] == "runDuration")\n |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") \n |> group(columns: [], mode:"by") \n |> drop(columns: ["_measurement", "topic", "location", "business", "uuid", "_start", "_stop"])\n |> filter(fn: (r) => if "${space}" != "ALL" then r["space"] == "${space}" else true)\n', 'refId': 'A'}]
Traceback (most recent call last):
File "/Users/aerojuan/.pyenv/versions/3.9.10/bin/grafana-snapshots", line 8, in
sys.exit(main())
File "/Users/aerojuan/.pyenv/versions/3.9.10/lib/python3.9/site-packages/grafana_snapshots/cli.py", line 268, in main
res = data_api.get_dashboard_data()
File "/Users/aerojuan/.pyenv/versions/3.9.10/lib/python3.9/site-packages/grafana_snapshots/grafanaData.py", line 313, in get_dashboard_data
if dtsrc in self.datasources and targets is not None:
TypeError: unhashable type: 'dict'`
The configuration yaml is:
`
general:
debug: true
snapshot_suffix: "_%Y%m%d%H%M"
output_path: snapshots
grafana:
default:
protocol: https
host: my.grafana.net
port: 443
token: "74777474783929"
search_api_limit: 5000
verify_ssl: true
contexts:
"dash-sandbox":
time_from: "now-7d"
time_to: "now"
`
Thanks in advance for your help.
The text was updated successfully, but these errors were encountered: