Discovering Schema Failed Issue

[ahmed-buksh]

I am creating a custom connector on Airbyte. After finishing up the Python source and testing all execution I thought of testing it via connection using Airbyte interface. I created a docker image, tagged it and uploaded it to docker hub and used it on my local instance of Airbyte. I was able to add source but when I built a connection against any destination, I get error that Discovering schema failed

I am attaching the log file of error as well. In log file I can see the command which runs for checking config and when I run that command from my terminal it works fine. Not sure what is getting wrong when I am doing it via local instance.

Discover Schema Failure (2).txt
Docker Hub Image

[alexgreen496]

Also experiencing this issue with v0.55.2, but with any source connector.

[ahmed-buksh]

@alexgreen496 I am also getting the same error. It looks like that it has some other reason for this instead of any dependency on connector. By the way I am also using 0.55.x.

[ahmed-buksh]

@alexgreen496 do let me know if you are able to find any solution for this.

[ahmed-buksh]

So I was able to figure out issue. In my streams primary_id was not as per schema for some streams which raised this issue. I inspected locally running airbyte logs in detail to find this out.

[bpatwa-sanimax]

@ahmed-buksh Where can you find locally running airbyte logs?

[yannik207]

@ahmed-buksh do you have a detailed explenation? I am running into the same problem? :) and I am currently not able to fix it. the confusing thing is that it runs on lower airbyte version?

[timomarks]

I had the same issue. The issue came up after using the schema generator which creates a configured_catalog.json. I fixed it by using the configured_catalog.json from the Python CDK Tutorial. Hope it helps others:

{
  "streams": [
    {
      "stream": {
        "name": "your_stream_name",
        "json_schema": {},
        "supported_sync_modes": ["full_refresh"]
      },
      "sync_mode": "full_refresh",
      "destination_sync_mode": "overwrite"
    }
  ]
}