Merge/Join datasets from different branches

[ntitovxperra]

Hi! I've recently started working in Appmixer and haven't found any nodes to Merge/Join data.
By Merge/Join data operations I mean something similar to SQL Joins operations, but for datasets on different branches in Appmixer.

I'll provide an example: there are 2 datasets that I got in workflow: 1. Export CSV 2. Hubspot - List Contacts.
The task is to remove all records from CSV that exist in Hubspot, checking by email.
So the output will be Export CSV dataset without filtered records.

[martin-krcmar]

Hi. There isn't a built-in Merge connector that can achieve exactly what you're asking for, but a custom solution could be developed. There are other ways to accomplish this as well. You would need to process each row from the CSV file and check against HubSpot to see if the record exists. If it does, you can add it to the output CSV file.

The first branch creates an empty CSV file, then downloads 100 contacts from HubSpot, sending them one by one to the output port. The GetRow function checks for each row in the CSV file, and if it's found (using IsNotEmpty), it's added to the CSV file.

The second branch works the opposite way. It splits the CSV file and sends each row, one by one, to the output port (GetRows), then uses GetContact to find each row in HubSpot. If a match is found, the row is added to the CSV file.

[ntitovxperra]

Thanks, but I think this solition won't work in my situation, because I have 10,000+ contacts in Hubspot. Do you think getting all of them won't crash a workflow?
Also Hubspot API has limitations and I don't see ways to use pagination in Appmixer Hubspot connector. So I guess I'll need to make direct HTTP request to Hubspot API.

[martin-krcmar]

The HubSpot ListContacts connector uses pagination, but it seems to have a limit of retrieving only 500 contacts. Therefore, the first option may not be feasible. The second option should work. How many rows are there in your CSV? Calling GetContact for each row could exhaust HubSpot's quotas, but we have retries in place. It may take some time to complete, depending on the number of rows in the CSV.