[BUG] Sliced ​​and Table FTP Connector broken with ts

[jean-louis]

Describe the bug
Using the ftp connector in sliced ​​or table mode, you get an error if the ts attribute is present. The device is not created and the data is not sent to the ThingsBoard (cloud) platform.

Attached are the configuration files and related logs.

I will open a PR with the fixes I am using locally.

Connector name:
FTP Connector

Error traceback:

tb-gateway  | 2024-11-03 23:23:40 - |ERROR| - [tb_gateway_service.py] - tb_gateway_service - __send_to_storage - 1098 - Error while sending data to storage!
tb-gateway  | Traceback (most recent call last):
tb-gateway  |   File "/thingsboard_gateway/gateway/tb_gateway_service.py", line 1093, in __send_to_storage
tb-gateway  |     self.__send_to_storage_old_formatted_data(connector_name, connector_id, data_array)
tb-gateway  |   File "/thingsboard_gateway/gateway/tb_gateway_service.py", line 1157, in __send_to_storage_old_formatted_data
tb-gateway  |     if not TBUtility.validate_converted_data(data):
tb-gateway  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tb-gateway  |   File "/thingsboard_gateway/tb_utility/tb_utility.py", line 76, in validate_converted_data
tb-gateway  |     if (entry.get("ts") is not None and len(entry.get("values")) > 0) or entry.get("ts") is None:
tb-gateway  |                                         ^^^^^^^^^^^^^^^^^^^^^^^^
tb-gateway  | TypeError: object of type 'NoneType' has no len()
tb-gateway  | 2024-11-03 23:23:40 - |ERROR| - [tb_gateway_service.py] - tb_gateway_service - __send_to_storage - 1098 - Error while sending data to storage!
tb-gateway  | Traceback (most recent call last):
tb-gateway  |   File "/thingsboard_gateway/gateway/tb_gateway_service.py", line 1093, in __send_to_storage
tb-gateway  |     self.__send_to_storage_old_formatted_data(connector_name, connector_id, data_array)
tb-gateway  |   File "/thingsboard_gateway/gateway/tb_gateway_service.py", line 1157, in __send_to_storage_old_formatted_data
tb-gateway  |     if not TBUtility.validate_converted_data(data):
tb-gateway  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tb-gateway  |   File "/thingsboard_gateway/tb_utility/tb_utility.py", line 76, in validate_converted_data
tb-gateway  |     if (entry.get("ts") is not None and len(entry.get("values")) > 0) or entry.get("ts") is None:
tb-gateway  |                                         ^^^^^^^^^^^^^^^^^^^^^^^^
tb-gateway  | TypeError: object of type 'NoneType' has no len()
tb-gateway  | 2024-11-03 23:23:40 - |ERROR| - [tb_gateway_service.py] - tb_gateway_service - __send_to_storage - 1098 - Error while sending data to storage!
tb-gateway  | Traceback (most recent call last):
tb-gateway  |   File "/thingsboard_gateway/gateway/tb_gateway_service.py", line 1093, in __send_to_storage
tb-gateway  |     self.__send_to_storage_old_formatted_data(connector_name, connector_id, data_array)
tb-gateway  |   File "/thingsboard_gateway/gateway/tb_gateway_service.py", line 1157, in __send_to_storage_old_formatted_data
tb-gateway  |     if not TBUtility.validate_converted_data(data):
tb-gateway  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tb-gateway  |   File "/thingsboard_gateway/tb_utility/tb_utility.py", line 76, in validate_converted_data
tb-gateway  |     if (entry.get("ts") is not None and len(entry.get("values")) > 0) or entry.get("ts") is None:
tb-gateway  |                                         ^^^^^^^^^^^^^^^^^^^^^^^^
tb-gateway  | TypeError: object of type 'NoneType' has no len()
tb-gateway  | 2024-11-03 23:23:40 - |ERROR| - [tb_gateway_service.py] - tb_gateway_service - __send_to_storage - 1098 - Error while sending data to storage!
tb-gateway  | Traceback (most recent call last):
tb-gateway  |   File "/thingsboard_gateway/gateway/tb_gateway_service.py", line 1093, in __send_to_storage
tb-gateway  |     self.__send_to_storage_old_formatted_data(connector_name, connector_id, data_array)
tb-gateway  |   File "/thingsboard_gateway/gateway/tb_gateway_service.py", line 1157, in __send_to_storage_old_formatted_data
tb-gateway  |     if not TBUtility.validate_converted_data(data):
tb-gateway  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tb-gateway  |   File "/thingsboard_gateway/tb_utility/tb_utility.py", line 76, in validate_converted_data
tb-gateway  |     if (entry.get("ts") is not None and len(entry.get("values")) > 0) or entry.get("ts") is None:
tb-gateway  |                                         ^^^^^^^^^^^^^^^^^^^^^^^^
tb-gateway  | TypeError: object of type 'NoneType' has no len()
tb-gateway  | 2024-11-03 23:23:40 - |ERROR| - [tb_gateway_service.py] - tb_gateway_service - __send_to_storage - 1098 - Error while sending data to storage!
tb-gateway  | Traceback (most recent call last):
tb-gateway  |   File "/thingsboard_gateway/gateway/tb_gateway_service.py", line 1093, in __send_to_storage
tb-gateway  |     self.__send_to_storage_old_formatted_data(connector_name, connector_id, data_array)
tb-gateway  |   File "/thingsboard_gateway/gateway/tb_gateway_service.py", line 1157, in __send_to_storage_old_formatted_data
tb-gateway  |     if not TBUtility.validate_converted_data(data):
tb-gateway  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tb-gateway  |   File "/thingsboard_gateway/tb_utility/tb_utility.py", line 76, in validate_converted_data
tb-gateway  |     if (entry.get("ts") is not None and len(entry.get("values")) > 0) or entry.get("ts") is None:
tb-gateway  |                                         ^^^^^^^^^^^^^^^^^^^^^^^^
tb-gateway  | TypeError: object of type 'NoneType' has no len()

Versions (please complete the following information):

• OS: Ubuntu 20.04
• Thingsboard IoT Gateway version: Docker - thingsboard/tb-gateway:latest

sliced_connector.json
sliced_connector.log
table_connector.json
table_connector.log

[jean-louis]

test_ftp_bug_sliced.txt
test_ftp_bug_table.csv