Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Cannot parse a json for keys that contain a colon (:) in the key name #1316

Closed
zwoabier opened this issue Feb 27, 2024 · 1 comment · Fixed by zwoabier/thingsboard-gateway#1 or #1327
Assignees
Labels

Comments

@zwoabier
Copy link
Contributor

Describe the bug
jsonpath_rw is able to parse keys that contain a colon (:) but the following line in the tb-gateway is preventing us from that.

expression_arr = findall(r'\$\{[${A-Za-z0-9. ^\]\[*_:]*\}', expression)

If we want to parse a key that is containing a special character like a colon we have to wrap the key in double quotes in mqtt.json config file like that
"value": "${params.\"switch:0\".aenergy.total}"
what will end in a sting like
${params."switch:0".aenergy.total}

But the above stiring does not match the regex that is passed to the findall method anymore due to the double quotes around the switch:0
The result is that the specified value regex will be set as the value itself. In my case '${params."switch:0".aenergy.total}' instead of the floating point value.

If we would add double quotes to the regex that would fix the issue like
expression_arr = findall(r'\$\{[${A-Za-z0-9. ^\]\[*_:"]*\}', expression)

Connector name (If bug in the some connector):
MQTT Connector (JSON Uplink Converter)

Error traceback (If available):

tb-gateway-1  | 2024-02-27 15:44:04 - |ERROR| - [json_mqtt_uplink_converter.py] - json_mqtt_uplink_converter - _convert_single_item - 101 - Error in converter, for config:
tb-gateway-1  | {"type": "json", "deviceNameTopicExpression": "(shelly.*?)(?=/)", "deviceTypeTopicExpression": "smart_meter", "timeout": 60000, "attributes": [{"value": "${src}", "key": "src", "type": "string"}, {"value": "${params.\"switch:0\".aenergy.total}", "key": "aenergy:total", "type": "string"}, {"value": "${params.\"switch:0\".aenergy.by_minute}", "key": "aenergy:by_minute", "type": "string"}], "timeseries": [{"value": "${params}", "key": "params", "type": "string"}, {"value": "${params.ts}", "key": "shelly_ts", "type": "int"}, {"value": "${params.\"switch:0\".aenergy.total}", "key": "aenergy:total", "type": "int"}, {"value": "${params.\"switch:0\".aenergy.by_minute}", "key": "aenergy:by_minute", "type": "int"}]}
tb-gateway-1  |  and message:
tb-gateway-1  | {'src': 'shellypro1pm-999988887777', 'dst': 'smartmeter/shellypro1pm-999988887777/events', 'method': 'NotifyStatus', 'params': {'ts': 1708948920.38, 'switch:0': {'id': 0, 'aenergy': {'by_minute': [1.0, 2.0, 3.0], 'minute_ts': 1708948919, 'total': 1.234}}}}
tb-gateway-1  |  could not convert string to float: '${params."switch:0".aenergy.total}'

Versions (please complete the following information):

  • docker container
  • Thingsboard IoT Gateway version 3.4.4
Copy link

Hi @zwoabier.
Thank you for your interest in ThingsBoard IoT Gateway.
Your issue was registered, please wait for response from engineer.

IOTGW-134

zwoabier added a commit to zwoabier/thingsboard-gateway that referenced this issue Mar 4, 2024
Fix regex and data conversion to bool
Fixes thingsboard#1316 and thingsboard#1326
@zwoabier zwoabier reopened this Mar 4, 2024
@samson0v samson0v linked a pull request Apr 9, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
3 participants