✨Source Gridly: Migrate Python CDK to Low-code CDK

[btkcodedev]

What

Migrating Source Gridly to Low-Code CDK
Closes #29282

How

Developed using (Configuration Based Source) low-code CDK

Recommended reading order

1. spec.yaml
2. manifest.yaml
3. schemas/*

Tests

Integration & Acceptance

Full Test Results

🚨 User Impact 🚨

• No breaking changes, migration to low-code

Migrate Connector: Source Gridly

Community member or Airbyter

• Community member? Grant edit access to maintainers (instructions)
• Secrets in the connector's spec are annotated with airbyte_secret
• Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
• Code reviews completed
• Documentation updated
  • Connector's README.md
  • Connector's bootstrap.md
  • docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
  • docs/integrations/README.md
  • airbyte-integrations/builds.md
• PR name follows PR naming conventions

[github-actions]

Before Merging a Connector Pull Request

Wow! What a great pull request you have here! 🎉

To merge this PR, ensure the following has been done/considered for each connector added or updated:

• PR name follows PR naming conventions
• Breaking changes are considered. If a Breaking Change is being introduced, ensure an Airbyte engineer has created a Breaking Change Plan.
• Connector version has been incremented in the Dockerfile and metadata.yaml according to our Semantic Versioning for Connectors guidelines
• You've updated the connector's metadata.yaml file any other relevant changes, including a breakingChanges entry for major version bumps. See metadata.yaml docs
• Secrets in the connector's spec are annotated with airbyte_secret
• All documentation files are up to date. (README.md, bootstrap.md, docs.md, etc...)
• Changelog updated in docs/integrations/<source or destination>/<name>.md with an entry for the new version. See changelog example
• Migration guide updated in docs/integrations/<source or destination>/<name>-migrations.md with an entry for the new version, if the version is a breaking change. See migration guide example
• If set, you've ensured the icon is present in the platform-internal repo. (Docs)

If the checklist is complete, but the CI check is failing,

1. Check for hidden checklists in your PR description

2. Toggle the github label checklist-action-run on/off to re-run the checklist CI.

[btkcodedev]

All tests are passing in local
Checks:

• No pagination support ✅
• No incremental support✅
• Added parent stream slices✅
• Updated version in dockerfile, docs and metadata✅

Please ping for the credentials

[marcosmarxm]

Hello 👋 , right now Airbyte's integration tests are failing for this connector and I'm going to work during the week to fix them. Due a high number of contributions we have a backlog and your contribution will be review next week. We ask to you share the integration test from your side whatever possible and if you don't have credentials to run tests to get in touch with @RealChrisSean as soon as possible. Also some common missing steps contributors does migrating to low code are (and you can double check to make sure you implemented this):

• adding paginator
• update the documentation changelog + metadata file and dockerfile version
• if the connector has start_date parameter you must implement incremental syncs

[btkcodedev]

@marcosmarxm All test are passing in local and every points are corrected
#29647 (comment)

[btkcodedev]

@marcosmarxm But I can't find Pagination support for any stream in previous python source, will try to search documentation

[marcosmarxm]

@marcosmarxm But I can't find Pagination support for any stream in previous python source, will try to search documentation

Check https://www.gridly.com/docs/api/#record

[btkcodedev]

Thanks @marcosmarxm Pagination added, All tests are passing :)

[sajarin]

I've been trying to run tests for this PR in another branch but I don't think the implementation of this connector matches the original, it seems to timeout after producing an absurd amount of records (tens of thousands) from the records stream. Unsure if there is an infinite loop somewhere or just an combinatorial explosion of the amount of records read given the tree structure of this api.

[sajarin]

Hey @btkcodedev

After looking at this connector and it's unique functionality, it does not seem possible to convert this to a low-code version. This is because this particular connector dynamically generates the schema which is not supported by the low-code cdk. Correct me if I'm wrong but the current schema files you've created for this connector are only reflective of our sandbox account and will not work in theory for someone else's connection since the schemas would be totally different.

Unfortunately, we can't accept this PR. Sometimes we get things right but sometimes we make mistake. I apologize for the time you spent developing this connector, this should have been verified before being listed as a possible contribution. Feel free to reach out if you have any questions.