Best practices for modifying stream schema with Octavia #28766
Unanswered
alexchao
asked this question in
Connector Questions
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
We're using the Octavia CLI to keep our airbyte source, destination, and connection configurations version controlled as YAML. In the configuration for one of our mongodb-to-postgres connections, there are some fields in the stream schema whose data type I'd like to update (airbyte identified the field as a string field, but I'd like to add
format: date-time
to the definition).However, I noticed that in at least one place (this tutorial: https://airbyte.com/tutorials/version-control-airbyte-configurations), it is advised not to edit the
streams.stream
part of thesync_catalog
:I'm wondering then what the recommended workflow would be for making such updates to the stream schema. I've found that making changes in the
streams.stream
portion of the YAML file and then applying withoctavia apply
does in fact achieve the desired effect, but if it's not the recommended method, then I'm wondering what is. I so far haven't found a way to make these changes through the Web UI (i.e. for one of my Full Refresh / Overwrite streams, I cannot toggle individual fields, add new fields, or change their data type), and even if the Web UI was an option, I'm wondering whether I should then export the configuration as YAML for version control.Apologies in advance if I've missed some documentation on this topic.
Example of a change I'd like to make:
Beta Was this translation helpful? Give feedback.
All reactions