[source-sharepoint] connector failes check: No streams are available for source SourceMicrosoftSharePoint

[rewelle-dic]

Connector Name

source-microsoft-sharepoint

Connector Version

0.4.1

What step the error happened?

Configuring a new source connector

Relevant information

I am using Sharepoint connector to sync files from Sharepoint into Redshift.
I am using Authentication via OAuth.
Credentials provided have full read access in graph API (Files.Read.All and Sites.Read.All).
Testing via Postman, the application can list and read lists, drives and libraries successfully.

Error message:

Check fail and I get the message:

'No streams are available for source SourceMicrosoftSharePoint. This is probably an issue with the connector. Please verify that your configuration provides permissions to list and read files from the source. Contact support if you are unable to resolve this issue.'

Relevant log output

2024-08-12 08:13:40 �[46mplatform�[0m > Docker volume job log path: /tmp/workspace/7eb0180b-ae8f-4bec-9394-2848e56fb6ce/0/logs.log
2024-08-12 08:13:40 �[46mplatform�[0m > Executing worker wrapper. Airbyte version: 0.63.14
2024-08-12 08:13:40 �[46mplatform�[0m > 
2024-08-12 08:13:40 �[46mplatform�[0m > ----- START CHECK -----
2024-08-12 08:13:40 �[46mplatform�[0m > 
2024-08-12 08:13:40 �[46mplatform�[0m > Using default value for environment variable SIDECAR_KUBE_CPU_LIMIT: '2.0'
2024-08-12 08:13:40 �[46mplatform�[0m > Using default value for environment variable SOCAT_KUBE_CPU_LIMIT: '2.0'
2024-08-12 08:13:40 �[46mplatform�[0m > Using default value for environment variable SIDECAR_KUBE_CPU_REQUEST: '0.1'
2024-08-12 08:13:40 �[46mplatform�[0m > Using default value for environment variable SOCAT_KUBE_CPU_REQUEST: '0.1'
2024-08-12 08:13:41 �[46mplatform�[0m > Checking if airbyte/source-microsoft-sharepoint:0.4.1 exists...
2024-08-12 08:13:41 �[46mplatform�[0m > airbyte/source-microsoft-sharepoint:0.4.1 was found locally.
2024-08-12 08:13:41 �[46mplatform�[0m > Creating docker container = source-microsoft-sharepoint-check-7eb0180b-ae8f-4bec-9394-2848e56fb6ce-0-ywwxm with resources io.airbyte.config.ResourceRequirements@18485c8a[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=,additionalProperties={}] and allowedHosts null
2024-08-12 08:13:41 �[46mplatform�[0m > Preparing command: docker run --rm --init -i -w /data/7eb0180b-ae8f-4bec-9394-2848e56fb6ce/0 --log-driver none --name source-microsoft-sharepoint-check-7eb0180b-ae8f-4bec-9394-2848e56fb6ce-0-ywwxm --network host -v airbyte_workspace:/data -v oss_local_root:/local -e DEPLOYMENT_MODE=OSS -e WORKER_CONNECTOR_IMAGE=airbyte/source-microsoft-sharepoint:0.4.1 -e AUTO_DETECT_SCHEMA=true -e LAUNCHDARKLY_KEY= -e SOCAT_KUBE_CPU_REQUEST=0.1 -e SOCAT_KUBE_CPU_LIMIT=2.0 -e FIELD_SELECTION_WORKSPACES= -e USE_STREAM_CAPABLE_STATE=true -e AIRBYTE_ROLE=dev -e WORKER_ENVIRONMENT=DOCKER -e APPLY_FIELD_SELECTION=false -e WORKER_JOB_ATTEMPT=0 -e OTEL_COLLECTOR_ENDPOINT=http://host.docker.internal:4317 -e FEATURE_FLAG_CLIENT=config -e AIRBYTE_VERSION=0.63.14 -e WORKER_JOB_ID=7eb0180b-ae8f-4bec-9394-2848e56fb6ce airbyte/source-microsoft-sharepoint:0.4.1 check --config source_config.json
2024-08-12 08:13:41 �[46mplatform�[0m > Reading messages from protocol version 0.2.0
2024-08-12 08:13:52 �[46mplatform�[0m > Check failed
2024-08-12 08:13:52 �[46mplatform�[0m > Check connection job received output: io.airbyte.config.StandardCheckConnectionOutput@53a92f85[status=failed,message='No streams are available for source SourceMicrosoftSharePoint. This is probably an issue with the connector. Please verify that your configuration provides permissions to list and read files from the source. Contact support if you are unable to resolve this issue.',additionalProperties={}]
2024-08-12 08:13:52 �[46mplatform�[0m > 
2024-08-12 08:13:52 �[46mplatform�[0m > ----- END CHECK -----
2024-08-12 08:13:52 �[46mplatform�[0m > 

Can anyone help me with the configuration process? Because I am a bit confused why I can't find any streams in airbyte, while I can list the files with direct MSgraph api calls.
I am I missing something?

Below are the permission on the azure application.

And this is what I get when I try to add a stream

In the connector documentation here: https://docs.airbyte.com/integrations/sources/microsoft-sharepoint#step-1-set-up-sharepoint-application, the point 5. Enter Drive Name, but I can't find in the interface where to enter the Drive Name.

Any suggestions?

Thanks in advance.

[b3b0]

Getting approximately the same error in self-hosted OSS, connector version 0.5.1:

'No streams are available for source SourceMicrosoftSharePoint. This is probably an issue with the connector. Please verify that your configuration provides permissions to list and read files from the source. Contact support if you are unable to resolve this issue.'

2024-09-03 17:32:33 INFO i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: CLAIM — (workloadId = [REDACTED]) — (dataplaneId = local)
2024-09-03 17:32:43 INFO i.a.c.i.LineGobbler(voidCall):166 - 
2024-09-03 17:32:44 INFO i.a.c.i.LineGobbler(voidCall):166 - ----- START CHECK -----
2024-09-03 17:32:44 INFO i.a.c.i.LineGobbler(voidCall):166 - 
2024-09-03 17:32:46 INFO i.a.c.ConnectorWatcher(run):87 - Connector exited, processing output
2024-09-03 17:32:46 INFO i.a.c.ConnectorWatcher(run):90 - Output file jobOutput.json found
2024-09-03 17:32:46 INFO i.a.c.ConnectorWatcher(run):96 - Connector exited with 0
2024-09-03 17:32:46 INFO i.a.w.i.VersionedAirbyteStreamFactory(create):189 - Reading messages from protocol version 0.2.0
2024-09-03 17:32:46 WARN i.a.m.l.MetricClientFactory(getMetricClient):43 - MetricClient has not been initialized. Must call MetricClientFactory.CreateMetricClient before using MetricClient. Using a dummy client for now. Ignore this if Airbyte is configured to not publish any metrics.
2024-09-03 17:32:46 ERROR i.a.w.i.VersionedAirbyteStreamFactory(internalLog):304 - Check failed
2024-09-03 17:32:46 INFO i.a.c.ConnectorMessageProcessor(updateConfigFromControlMessage):231 - Checking for optional control message...
2024-09-03 17:32:46 INFO i.a.c.ConnectorMessageProcessor(updateConfigFromControlMessage):253 - Optional control message not found. Skipping...
2024-09-03 17:32:46 INFO i.a.c.ConnectorWatcher(run):134 - Writing output of [REDACTED] to the doc store
2024-09-03 17:32:46 INFO i.a.c.ConnectorWatcher(run):136 - Marking workload as successful
2024-09-03 17:32:46 INFO i.a.c.ConnectorWatcher(exitProperly):189 - Deliberately exiting process with code 0.
2024-09-03 17:32:46 INFO i.a.c.i.LineGobbler(voidCall):166 - 
2024-09-03 17:32:46 INFO i.a.c.i.LineGobbler(voidCall):166 - ----- END CHECK -----```

[yash872]

@rewelle-dic have you got any solution for this ?