Skip to content

Commit

Permalink
docs: Add SingleStore batch variant docs (#1624)
Browse files Browse the repository at this point in the history
* Add SingleStore batch variant docs

* Add SingleStore batch variant docs
  • Loading branch information
danthelion authored Sep 12, 2024
1 parent 38f7f54 commit b174a36
Showing 1 changed file with 27 additions and 0 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# SingleStore Batch Query Connector

This connector captures data from SingleStore into Flow collections by periodically executing queries and translating
the results into JSON documents. It leverages SingleStore's MySQL wire compatibility to interact with the database.

**This connector periodically re-executes the query**. The default polling interval is set
to 24 hours to minimize this behavior's impact, but depending on table size, it may lead to duplicated data being
processed.

If the dataset has a natural cursor that can identify only new or updated rows, it should be specified by editing the
`Cursor` property of the binding. Common examples of suitable cursors include:

- Update timestamps, which are typically the best choice since they can capture all changed rows, not just new rows.
- Creation timestamps, which work for identifying newly added rows in append-only datasets but won’t capture updates.
- Serially increasing IDs, which can be used to track newly added rows.

## Setup

1. Ensure that [Estuary's IP addresses are allowlisted](/reference/allow-ip-addresses) to allow access. You can do by
following [these steps](https://docs.singlestore.com/cloud/reference/management-api/#control-access-to-the-api)
2. Grab the following details from the SingleStore workspace.
3. Workspace URL
4. Username
5. Password
6. Database
7. Configure the Connector with the appropriate values. Make sure to specify the database name under the "Advanced"
section.

0 comments on commit b174a36

Please sign in to comment.