Issues with LocalCSV connector

[hveerava]

Hi all, I have just started exploring airbyte. I deployed loally on my Mac and tetsing. I could build connection with "SFTP" as source and "Local CSV" as destination. I am trying to sync data from SFTP to my local laptop. The data uploaded on SFTP is not a proper csv. Is that the reason why I get the following error? Basically I just want to move data from SFTP to local machine irrespective of JSON, CSV or any other format. Is it doable via airbyte? Thanks.
Exception occurred while trying to parse file test.csv : 2024-01-12 09:01:47 source > com.fasterxml.jackson.dataformat.csv.CsvReadException: Too many entries: expected at most 1 (value #1 (14 chars) "let's have 1-1")

[sajarin]

it seems like the issue might be related to the format of the data in your SFTP source. Airbyte's SFTP connector currently expects files with a "csv" extension to be loaded as source data. If the files are not in CSV format, they might not be considered as source data to be loaded, which could be causing the error you're seeing [0].

If you want to move data from SFTP to your local machine irrespective of the format (JSON, CSV, etc.), you might need to ensure that the data is in a format that the SFTP connector can handle. If the data is not in a proper CSV format, you might need to convert it to a proper format before trying to sync it with Airbyte.

Additionally, for local filesystem, the file probably has to be stored somewhere in the /tmp/airbyte_local folder with the same limitations as the CSV Destination so the URL should also start with /local/ [1].