Skip to content

Commit

Permalink
airbyte-ci: build manifest only connectors correctly in up-to-date (#…
Browse files Browse the repository at this point in the history
…47483)

## What

* Problems: We saw build failure at `up-to-date` time, which let to unhandled errors.
* Solution: Manifest only connector build was not wired in the up-to-date pipeline, we used the python connector build logic on these connectors which led to these execeptions.
 
This pull request includes several changes aimed at improving the build logic and updating dependencies for the Airbyte CI connectors. The most important changes involve fixing the build logic to support any connector language, refactoring the import statements, and updating the pipeline version.

### Improvements to build logic:

* [`airbyte-ci/connectors/pipelines/README.md`](diffhunk://#diff-62eccd92928fbcd3d285983bfdaa2b0d4ca49016cb9c2f63d6d9fc968c59c541R853): Added a new entry to document the fix for build logic used in `up-to-date` to support any connector language.

### Refactoring:

* [`airbyte-ci/connectors/pipelines/pipelines/airbyte_ci/connectors/up_to_date/pipeline.py`](diffhunk://#diff-cef433180658cd9d23043da2e4577f65c9bfb7bca5e9bacca87e3b24ad3e56bbL11-R11): Refactored import statements to use `run_connector_build` instead of `BuildConnectorImages`.
* [`airbyte-ci/connectors/pipelines/pipelines/airbyte_ci/connectors/up_to_date/pipeline.py`](diffhunk://#diff-cef433180658cd9d23043da2e4577f65c9bfb7bca5e9bacca87e3b24ad3e56bbL129-R129): Updated the `run_connector_up_to_date_pipeline` function to use `run_connector_build` for building connector images.

### Dependency updates:

* [`airbyte-ci/connectors/pipelines/pyproject.toml`](diffhunk://#diff-087e2c37602bbd6824f875004abddcb4e1a374da12bf84201671ed0900882ce0L7-R7): Updated the version from `4.41.8` to `4.41.9`.
  • Loading branch information
alafanechere authored Oct 28, 2024
1 parent b4496f0 commit ecf09cc
Show file tree
Hide file tree
Showing 3 changed files with 4 additions and 3 deletions.
1 change: 1 addition & 0 deletions airbyte-ci/connectors/pipelines/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -850,6 +850,7 @@ airbyte-ci connectors --language=low-code migrate-to-manifest-only

| Version | PR | Description |
| ------- | ---------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| 4.41.9 | [#47483](https://github.com/airbytehq/airbyte/pull/47483) | Fix build logic used in `up-to-date` to support any connector language. |
| 4.41.8 | [#47447](https://github.com/airbytehq/airbyte/pull/47447) | Use `cache_ttl` for base image registry listing in `up-to-date`. |
| 4.41.7 | [#47444](https://github.com/airbytehq/airbyte/pull/47444) | Remove redundant `--ignore-connector` error from up-to-date. `--metadata-query` can be used instead. |
| 4.41.6 | [#47308](https://github.com/airbytehq/airbyte/pull/47308) | Connector testing: skip incremental acceptance test when the connector is not released. |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
from typing import TYPE_CHECKING

from jinja2 import Environment, PackageLoader, select_autoescape
from pipelines.airbyte_ci.connectors.build_image.steps.python_connectors import BuildConnectorImages
from pipelines.airbyte_ci.connectors.build_image.steps import run_connector_build
from pipelines.airbyte_ci.connectors.context import ConnectorContext
from pipelines.airbyte_ci.connectors.reports import ConnectorReport
from pipelines.airbyte_ci.steps.base_image import UpdateBaseImageMetadata
Expand Down Expand Up @@ -126,7 +126,7 @@ async def run_connector_up_to_date_pipeline(
# to fill the PR body with the correct information about what exactly got updated.
if create_pull_request:
# Building connector images is also universal across connector technologies.
build_result = await BuildConnectorImages(context).run()
build_result = await run_connector_build(context)
step_results.append(build_result)
dependency_updates: List[DependencyUpdate] = []

Expand Down
2 changes: 1 addition & 1 deletion airbyte-ci/connectors/pipelines/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "poetry.core.masonry.api"

[tool.poetry]
name = "pipelines"
version = "4.41.8"
version = "4.41.9"
description = "Packaged maintained by the connector operations team to perform CI for connectors' pipelines"
authors = ["Airbyte <[email protected]>"]

Expand Down

0 comments on commit ecf09cc

Please sign in to comment.