[HELP] - [tb_gateway_service.py] - tb_gateway_service - __connect_with_connectors - 849 - Config incorrect for mqtt #1287

DuXiaoChuang · 2024-01-30T02:49:22Z

Describe the issue
[tb_gateway_service.py] - tb_gateway_service - __connect_with_connectors - 849 - Config incorrect for mqtt

Configuration (Attach your configuration file)

{
  "thingsboard": {
    "host": "localhost",
    "port": 1883,
    "remoteShell": false,
    "remoteConfiguration": true,
    "statistics": {
      "enable": true,
      "statsSendPeriodInSeconds": 3600
    },
    "deviceFiltering": {
      "enable": false,
      "filterFile": "list.json"
    },
    "maxPayloadSizeBytes": 1024,
    "minPackSendDelayMS": 200,
    "minPackSizeToSend": 500,
    "checkConnectorsConfigurationInSeconds": 60,
    "handleDeviceRenaming": true,
    "security": {
      "type": "accessToken",
        "accessToken": "703v6xo03g7bjykilf4n"
    },
    "qos": 1,
    "checkingDeviceActivity": {
      "checkDeviceInactivity": false,
      "inactivityTimeoutSeconds": 200,
      "inactivityCheckPeriodSeconds": 500
    }
  },
  "storage": {
    "type": "memory",
    "read_records_count": 100,
    "max_records_count": 100000,
    "data_folder_path": "./data/",
    "max_file_count": 10,
    "max_read_records_count": 10,
    "max_records_per_file": 10000,
    "data_file_path": "./data/data.db",
    "messages_ttl_check_in_hours": 1,
    "messages_ttl_in_days": 7
  },
  "grpc": {
    "enabled": false,
    "serverPort": 9595,
    "keepaliveTimeMs": 10000,
    "keepaliveTimeoutMs": 5000,
    "keepalivePermitWithoutCalls": true,
    "maxPingsWithoutData": 0,
    "minTimeBetweenPingsMs": 10000,
    "minPingIntervalWithoutDataMs": 5000,
    "keepAliveTimeMs": 10000,
    "keepAliveTimeoutMs": 5000
  },
  "connectors": [
    {
      "type": "mqtt",
      "name": "MQTT Broker Connector",
      "configuration": "mqtt.json"
    },
    {
      "type": "modbus",
      "name": "Modbus Connector",
      "configuration": "modbus.json"
    },
     {
       "type": "modbus",
       "name": "Modbus Serial Connector",
       "configuration": "modbus_serial.json"
    },
    {
      "type": "odbc",
      "name": "ODBC Connector",
      "configuration": "odbc.json"
    },
    {
      "type": "opcua",
      "name": "OPC-UA Connector",
      "configuration": "opcua.json"
    }
  ]
}

MQTT.json：{ "broker": { "name": "Default Local Broker", "host": "broker.emqx.io", "port": 1883, "clientId": "ThingsBoard_gateway", "version": 5, "maxMessageNumberPerWorker": 10, "maxNumberOfWorkers": 100, "sendDataOnlyOnChange": false, "security": { "type": "basic", "username": "", "password": "" } }, "mapping": [ { "topicFilter": "/sensor/data", "converter": { "type": "json", "deviceNameJsonExpression": "${serialNumber}", "deviceTypeJsonExpression": "${sensorType}", "sendDataOnlyOnChange": false, "timeout": 60000, "attributes": [ { "type": "string", "key": "model", "value": "${sensorModel}" }, { "type": "string", "key": "${sensorModel}", "value": "on" } ], "timeseries": [ { "type": "double", "key": "temperature", "value": "${temp}" }, { "type": "double", "key": "humidity", "value": "${hum}" }, { "type": "string", "key": "combine", "value": "${hum}:${temp}" } ] } }, { "topicFilter": "/sensor/+/data", "converter": { "type": "json", "deviceNameTopicExpression": "(?<=sensor/)(.*?)(?=/data)", "deviceTypeTopicExpression": "Thermometer", "sendDataOnlyOnChange": false, "timeout": 60000, "attributes": [ { "type": "string", "key": "model", "value": "${sensorModel}" } ], "timeseries": [ { "type": "double", "key": "temperature", "value": "${temp}" }, { "type": "double", "key": "humidity", "value": "${hum}" } ] } }, { "topicFilter": "/sensor/raw_data", "converter": { "type": "bytes", "deviceNameExpression": "[0:4]", "deviceTypeExpression": "default", "sendDataOnlyOnChange": false, "timeout": 60000, "attributes": [ { "type": "raw", "key": "rawData", "value": "[:]" } ], "timeseries": [ { "type": "raw", "key": "temp", "value": "[4:]" } ] } }, { "topicFilter": "/custom/sensors/+", "converter": { "type": "custom", "extension": "CustomMqttUplinkConverter", "cached": true, "extension-config": { "temperatureBytes": 2, "humidityBytes": 2, "batteryLevelBytes": 1 } } } ], "connectRequests": [ { "topicFilter": "sensor/connect", "deviceNameJsonExpression": "${SerialNumber}" }, { "topicFilter": "sensor/+/connect", "deviceNameTopicExpression": "(?<=sensor/)(.*?)(?=/connect)" } ], "disconnectRequests": [ { "topicFilter": "sensor/disconnect", "deviceNameJsonExpression": "${SerialNumber}" }, { "topicFilter": "sensor/+/disconnect", "deviceNameTopicExpression": "(?<=sensor/)(.*?)(?=/disconnect)" } ], "attributeRequests": [ { "retain": false, "topicFilter": "v1/devices/me/attributes/request", "deviceNameJsonExpression": "${serialNumber}", "attributeNameJsonExpression": "${versionAttribute}, ${pduAttribute}", "topicExpression": "devices/${deviceName}/attrs", "valueExpression": "${attributeKey}: ${attributeValue}" } ], "attributeUpdates": [ { "retain": true, "deviceNameFilter": "SmartMeter.*", "attributeFilter": "uploadFrequency", "topicExpression": "sensor/${deviceName}/${attributeKey}", "valueExpression": "{\"${attributeKey}\":\"${attributeValue}\"}" } ], "serverSideRpc": [ { "deviceNameFilter": ".*", "methodFilter": "echo", "requestTopicExpression": "sensor/${deviceName}/request/${methodName}/${requestId}", "responseTopicExpression": "sensor/${deviceName}/response/${methodName}/${requestId}", "responseTimeout": 10000, "valueExpression": "${params}" }, { "deviceNameFilter": ".*", "methodFilter": "no-reply", "requestTopicExpression": "sensor/${deviceName}/request/${methodName}/${requestId}", "valueExpression": "${params}" } ], "id": "f69f14a1-d87c-4d28-840b-37dfb34a26c4" }

Connector name (If you need help with some connector/converter):
[e.g. MQTT Connector]

Error traceback (If it was raised):

'deviceName'
Traceback (most recent call last):
File "", line 2, in
KeyError: 'deviceName'



**Versions (please complete the following information):**
 - windows: [windows11]
 - Thingsboard IoT Gateway version [e.g. 3.4.4]
 - Python version[e.g. 3.12.1]

The text was updated successfully, but these errors were encountered:

github-actions · 2024-01-30T02:49:38Z

Hi @DuXiaoChuang.
Thank you for your interest in ThingsBoard IoT Gateway.
Your issue was registered, please wait for response from engineer.

IOTGW-83

devaskim · 2024-01-30T05:54:42Z

@DuXiaoChuang According to your gateway config, MQTT config is in mqtt.json

DuXiaoChuang · 2024-01-30T07:16:00Z

@DuXiaoChuang According to your gateway config, MQTT config is in mqtt.json

ok，how to reslove this problem？my gateway config is error？

MQTT.jason：

  "broker": {
    "name": "Default Local Broker",
    "host": "broker.emqx.io",
    "port": 1883,
    "clientId": "ThingsBoard_gateway",
    "version": 5,
    "maxMessageNumberPerWorker": 10,
    "maxNumberOfWorkers": 100,
    "sendDataOnlyOnChange": false,
    "security": {
      "type": "basic",
      "username": "",
      "password": ""
    }
  },
  "mapping": [
    {
      "topicFilter": "/sensor/data",
      "converter": {
        "type": "json",
        "deviceNameJsonExpression": "${serialNumber}",
        "deviceTypeJsonExpression": "${sensorType}",
        "sendDataOnlyOnChange": false,
        "timeout": 60000,
        "attributes": [
          {
            "type": "string",
            "key": "model",
            "value": "${sensorModel}"
          },
          {
            "type": "string",
            "key": "${sensorModel}",
            "value": "on"
          }
        ],
        "timeseries": [
          {
            "type": "double",
            "key": "temperature",
            "value": "${temp}"
          },
          {
            "type": "double",
            "key": "humidity",
            "value": "${hum}"
          },
          {
            "type": "string",
            "key": "combine",
            "value": "${hum}:${temp}"
          }
        ]
      }
    },
    {
      "topicFilter": "/sensor/+/data",
      "converter": {
        "type": "json",
        "deviceNameTopicExpression": "(?<=sensor/)(.*?)(?=/data)",
        "deviceTypeTopicExpression": "Thermometer",
        "sendDataOnlyOnChange": false,
        "timeout": 60000,
        "attributes": [
          {
            "type": "string",
            "key": "model",
            "value": "${sensorModel}"
          }
        ],
        "timeseries": [
          {
            "type": "double",
            "key": "temperature",
            "value": "${temp}"
          },
          {
            "type": "double",
            "key": "humidity",
            "value": "${hum}"
          }
        ]
      }
    },
    {
      "topicFilter": "/sensor/raw_data",
      "converter": {
        "type": "bytes",
        "deviceNameExpression": "[0:4]",
        "deviceTypeExpression": "default",
        "sendDataOnlyOnChange": false,
        "timeout": 60000,
        "attributes": [
          {
            "type": "raw",
            "key": "rawData",
            "value": "[:]"
          }
        ],
        "timeseries": [
          {
            "type": "raw",
            "key": "temp",
            "value": "[4:]"
          }
        ]
      }
    },
    {
      "topicFilter": "/custom/sensors/+",
      "converter": {
        "type": "custom",
        "extension": "CustomMqttUplinkConverter",
        "cached": true,
        "extension-config": {
          "temperatureBytes": 2,
          "humidityBytes": 2,
          "batteryLevelBytes": 1
        }
      }
    }
  ],
  "connectRequests": [
    {
      "topicFilter": "sensor/connect",
      "deviceNameJsonExpression": "${SerialNumber}"
    },
    {
      "topicFilter": "sensor/+/connect",
      "deviceNameTopicExpression": "(?<=sensor/)(.*?)(?=/connect)"
    }
  ],
  "disconnectRequests": [
    {
      "topicFilter": "sensor/disconnect",
      "deviceNameJsonExpression": "${SerialNumber}"
    },
    {
      "topicFilter": "sensor/+/disconnect",
      "deviceNameTopicExpression": "(?<=sensor/)(.*?)(?=/disconnect)"
    }
  ],
  "attributeRequests": [
    {
      "retain": false,
      "topicFilter": "v1/devices/me/attributes/request",
      "deviceNameJsonExpression": "${serialNumber}",
      "attributeNameJsonExpression": "${versionAttribute}, ${pduAttribute}",
      "topicExpression": "devices/${deviceName}/attrs",
      "valueExpression": "${attributeKey}: ${attributeValue}"
    }
  ],
  "attributeUpdates": [
    {
      "retain": true,
      "deviceNameFilter": "SmartMeter.*",
      "attributeFilter": "uploadFrequency",
      "topicExpression": "sensor/${deviceName}/${attributeKey}",
      "valueExpression": "{\"${attributeKey}\":\"${attributeValue}\"}"
    }
  ],
  "serverSideRpc": [
    {
      "deviceNameFilter": ".*",
      "methodFilter": "echo",
      "requestTopicExpression": "sensor/${deviceName}/request/${methodName}/${requestId}",
      "responseTopicExpression": "sensor/${deviceName}/response/${methodName}/${requestId}",
      "responseTimeout": 10000,
      "valueExpression": "${params}"
    },
    {
      "deviceNameFilter": ".*",
      "methodFilter": "no-reply",
      "requestTopicExpression": "sensor/${deviceName}/request/${methodName}/${requestId}",
      "valueExpression": "${params}"
    }
  ],
  "id": "f69f14a1-d87c-4d28-840b-37dfb34a26c4"
}```

imbeacon · 2024-01-30T07:19:21Z

Hi @DuXiaoChuang,

Could you try to use a version from the master branch? it contains changes in connect_with_connectors method.

DuXiaoChuang · 2024-01-30T07:22:33Z

嗨，

您可以尝试使用 master 分支中的版本吗？它包含connect_with_connectors方法的更改。

Hi @DuXiaoChuang,

Could you try to use a version from the master branch? it contains changes in connect_with_connectors method.

THKS，I will try;

DuXiaoChuang · 2024-01-30T08:29:44Z

hi
Can you try using the version in the master branch? It contains changes to the connect_with_connectors method.

Hi @DuXiaoChuang,
Could you try to use a version from the master branch? it contains changes in connect_with_connectors method.

THKS，I will try;

"Hello, I just downloaded the latest code from the master branch, but I'm still getting this recurring error when running it."

DuXiaoChuang · 2024-01-30T08:38:26Z

hi
Can you try using the version in the master branch? It contains changes to the connect_with_connectors method.

Hi @DuXiaoChuang,
Could you try to use a version from the master branch? it contains changes in connect_with_connectors method.

THKS，I will try;

"Hello, do I need to run the thingsboard-gateway code on a Linux system? Currently, I am debugging the thingsboard-gateway code on a Windows system."

DuXiaoChuang · 2024-02-01T02:37:03Z

Hello, I have already solved this problem, but I'm not sure if this solution is correct. I modified the --connect_ With_ The connectors method, modified the conditions, and after modification, it can connect to the mqtt connector normally.

Before modification:

if ("logLevel" in connector configuration ["config"] [config] and "name" in connector configuration ["config"] [config] and len (connector configuration ["config"] [config]. keys())>3) or\
("logLevel" not in connector configuration ["config"] [config] and "name" not in connector configuration ["config"] [config] and len (connector configuration ["config"] [config]. keys())>1):

After modification:
if ("logLevel" in connector configuration ["config"] [config] and "name" in connector configuration ["config"] [config] and len (connector configuration ["config"] [config]. keys())>3) or\("logLevel" not in connector configuration ["config"] [config] and "name" in connector configuration ["config"] [config] and len (connector configuration ["config"] [config]. keys())>1):

imbeacon · 2024-05-31T09:23:04Z

Hi @DuXiaoChuang,

This issue was closed due to inactivity, please open it if you think that it is not solved.

DuXiaoChuang added the help wanted label Jan 30, 2024

DuXiaoChuang assigned imbeacon and samson0v Jan 30, 2024

imbeacon closed this as completed May 31, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[HELP] - [tb_gateway_service.py] - tb_gateway_service - __connect_with_connectors - 849 - Config incorrect for mqtt #1287

[HELP] - [tb_gateway_service.py] - tb_gateway_service - __connect_with_connectors - 849 - Config incorrect for mqtt #1287

DuXiaoChuang commented Jan 30, 2024 •

edited

Loading

github-actions bot commented Jan 30, 2024

devaskim commented Jan 30, 2024

DuXiaoChuang commented Jan 30, 2024 •

edited by imbeacon

Loading

imbeacon commented Jan 30, 2024

DuXiaoChuang commented Jan 30, 2024

DuXiaoChuang commented Jan 30, 2024 •

edited

Loading

DuXiaoChuang commented Jan 30, 2024 •

edited

Loading

DuXiaoChuang commented Feb 1, 2024 •

edited

Loading

imbeacon commented May 31, 2024

[HELP] - [tb_gateway_service.py] - tb_gateway_service - __connect_with_connectors - 849 - Config incorrect for mqtt #1287

[HELP] - [tb_gateway_service.py] - tb_gateway_service - __connect_with_connectors - 849 - Config incorrect for mqtt #1287

Comments

DuXiaoChuang commented Jan 30, 2024 • edited Loading

github-actions bot commented Jan 30, 2024

devaskim commented Jan 30, 2024

DuXiaoChuang commented Jan 30, 2024 • edited by imbeacon Loading

imbeacon commented Jan 30, 2024

DuXiaoChuang commented Jan 30, 2024

DuXiaoChuang commented Jan 30, 2024 • edited Loading

DuXiaoChuang commented Jan 30, 2024 • edited Loading

DuXiaoChuang commented Feb 1, 2024 • edited Loading

imbeacon commented May 31, 2024

DuXiaoChuang commented Jan 30, 2024 •

edited

Loading

DuXiaoChuang commented Jan 30, 2024 •

edited by imbeacon

Loading

DuXiaoChuang commented Jan 30, 2024 •

edited

Loading

DuXiaoChuang commented Jan 30, 2024 •

edited

Loading

DuXiaoChuang commented Feb 1, 2024 •

edited

Loading