☁ Cloud Interop: Basic run_sync() interop with Airbyte Cloud/OSS/Enterprise

[aaronsteers]

This would add the following interfaces for interacting with a hosted Airbyte REST API (Cloud/OSS/Enterprise):

1. get_cloud_connection() - Get a handle for a remotely-hosted connection.
2. get_latest_read_result() - Get the latest result from a cloud connection sync. Should have properties like success (true/false), executed_at, completed_at, etc.
3. run_sync() - Remotely execute a sync operation.

Pseudocode mockup:

Locally Executed

Executed via Hosted Airbyte

github_source = get_source( "source-github", # ... ) sql_cache = SnowflakeCache( # ... ) # Sync data to the cache read_result = github_source.read( cache=sql_cache, ) # Get the repos dataset repos_data = read_result.cache.streams["repos"] # Iterate through records for repo in repos_data: print(repo["name"])

# Get the Cloud connection object conn = get_cloud_connection( workspace_id="...", connection_id="...", api_key="...", ) # Get the latest run result read_result = conn.get_latest_read_result() # We can check basic properties of the sync: if not read_result.success: # We can invoke a Cloud sync from Python: read_result = conn.run_sync() # After the follow-on feature ships, you'd be also # be able to read from remote destinations. # https://github.com/airbytehq/PyAirbyte/issues/110 # Get the repos dataset repos_data = read_result.cache.streams["repos"] # Iterate through records for repo in repos_data: print(repo["name"])

[aaronsteers]

Deprioritized (for initial launch) in favor of:

• Docs: Add code samples and "Coming Soon" for Airbyte Cloud Interop #64