feat: Add spec_version to connector specification #48
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
flash1293
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add an optional
spec_versionproperty to the connector specification.When upgrading connector versions, the connector spec that all persisted config objects have to adhere to might change. While connectors are required to also be able to handle config objects that got created for previous versions of the connector as long as no explicit breaking change occurred, the connector itself is not the only consumer of config objects: The Airbyte UI and API clients access configuration objects as well - in these cases a spec change can have unintended consequences. For example if a field gets renamed from user_name to username), the UI won’t be able to render the configuration form correctly.
A new optional field in the connector specification allows the platform to keep track of the compatibility level of persisted actor configurations and behave accordingly.
The following changes are made to the protocol contract:
spec_versionmight be defined by a connector as part of the connector specification returned for the SPEC command.spec_versioncan be used to inform the orchestrator that persisted actor configs created for a connector version with a differentspec_versionare not considered compatible with the currentconnectionSpecificationof the connector.spec_versionis defined by a connector, all subsequent versions of the connector have to define it as well.spec_versionis not allowed to be decremented in a subsequent version of the connector.Note: Compatibility can be related to a spec change that makes persisted configuration objects from previous versions incompatible in the JSON schema sense (e.g. a field type changes from number to string) or in the logical sense (e.g. a pre-existing field has a different meaning now).