Source google sheet but have AWS S3 in log???? #46389
Unanswered
ngohuuduc
asked this question in
Deployment
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I have install airbyte 1.1.0 using abctl on an Ubuntu Instance.
However i was not able to connect to google Sheet, i saw an error of 502 and then i tried my best to look into the log file located in the container (/var/log/containers/*.log).
I'm curious that I only try to access Google Sheet, why there is error of AWS S3 as below ? so weird that i'm unable to trouble shoot the issue. Please helP!
root@airbyte-abctl-control-plane:/var/log/containers# cat oogle-sheets-check-3f302c99-1dd9-4c20-a135-4413bf7cb333-0-ezlwb_airbyte-abctl_connector-sidecar-74e6bf9528998632b7cb954ab7f8f47ac9f9d2d4bc4aca2e397859dd58cbd46b.log 2024-10-05T07:01:05.810487822Z stdout F Unsetting empty environment variable 'DATA_PLANE_SERVICE_ACCOUNT_CREDENTIALS_PATH' 2024-10-05T07:01:05.810539926Z stdout F Unsetting empty environment variable 'KEYCLOAK_INTERNAL_REALM_ISSUER' 2024-10-05T07:01:05.810631833Z stdout F Unsetting empty environment variable 'DATA_PLANE_SERVICE_ACCOUNT_EMAIL' 2024-10-05T07:01:05.810698438Z stdout F Unsetting empty environment variable 'KEYCLOAK_CLIENT_ID' 2024-10-05T07:01:05.810775044Z stdout F Unsetting empty environment variable 'CONTROL_PLANE_AUTH_ENDPOINT' 2024-10-05T07:01:08.12047182Z stdout F 2024-10-05T07:01:08.118630479Z main INFO Loading mask data from '/seed/specs_secrets_mask.yaml 2024-10-05T07:01:13.088783204Z stdout F 2024-10-05 07:01:13 INFO i.m.c.e.DefaultEnvironment(<init>):168 - Established active environments: [k8s, cloud, worker-v2, control-plane, oss, local-secrets] 2024-10-05T07:01:21.388132638Z stdout F 2024-10-05 07:01:21 INFO i.a.c.ApplicationKt(main):20 - Context started 2024-10-05T07:01:21.402247876Z stdout F 2024-10-05 07:01:21 INFO i.a.c.ApplicationKt(main):21 - 12578057197.00 ns/exec (total: 12.58s, 1 executions) 2024-10-05T07:01:25.90642705Z stdout F 2024-10-05 07:01:25 INFO i.a.c.ApplicationKt(main):28 - Sidecar created 2024-10-05T07:01:25.906457152Z stdout F 2024-10-05 07:01:25 INFO i.a.c.ApplicationKt(main):29 - 8536856942.00 ns/exec (total: 17.07s, 2 executions) 2024-10-05T07:01:29.00903198Z stdout F 2024-10-05 07:01:29 INFO i.a.c.i.LineGobbler(voidCall):166 - 2024-10-05T07:01:29.090075001Z stdout F 2024-10-05T07:01:29.089122624Z pool-5-thread-1 ERROR Recursive call to appender SecretMaskRewrite 2024-10-05T07:01:31.439877562Z stdout F 2024-10-05 07:01:31 INFO i.a.c.i.LineGobbler(voidCall):166 - ----- START CHECK ----- 2024-10-05T07:01:31.442926208Z stdout F 2024-10-05 07:01:31 INFO i.a.c.i.LineGobbler(voidCall):166 - 2024-10-05T07:01:32.077989104Z stdout F 2024-10-05 07:01:32 WARN c.a.l.CommonsLog(warn):113 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant.If you are using Java 9+, you will need to include javax.xml.bind:jaxb-api as a dependency. 2024-10-05T07:01:32.902954679Z stdout F 2024-10-05 07:01:32 ERROR c.v.l.BufferPublisher(endPublish):69 - Cannot end publish with com.van.logging.aws.S3PublishHelper@686cc567 due to error 2024-10-05T07:01:32.902984481Z stdout F java.lang.RuntimeException: Cannot end publishing: Cannot publish to S3: Storage backend has reached its minimum free drive threshold. Please delete a few objects to proceed. (Service: Amazon S3; Status Code: 507; Error Code: XMinioStorageFull; Request ID: 17FB7C6D93A48F64; S3 Extended Request ID: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8; Proxy: null) 2024-10-05T07:01:32.902988982Z stdout F at com.van.logging.AbstractFilePublishHelper.end(AbstractFilePublishHelper.java:66) ~[appender-core-5.3.2.jar:?] 2024-10-05T07:01:32.902992682Z stdout F at com.van.logging.BufferPublisher.endPublish(BufferPublisher.java:67) ~[appender-core-5.3.2.jar:?] 2024-10-05T07:01:32.902995782Z stdout F at com.van.logging.LoggingEventCache.publishEventsFromFile(LoggingEventCache.java:198) ~[appender-core-5.3.2.jar:?] 2024-10-05T07:01:32.902998783Z stdout F at com.van.logging.LoggingEventCache.lambda$publishCache$0(LoggingEventCache.java:243) ~[appender-core-5.3.2.jar:?] 2024-10-05T07:01:32.903006383Z stdout F at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?] 2024-10-05T07:01:32.903010184Z stdout F at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?] 2024-10-05T07:01:32.903013084Z stdout F at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?] 2024-10-05T07:01:32.903016184Z stdout F at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?] 2024-10-05T07:01:32.903019784Z stdout F at java.base/java.lang.Thread.run(Thread.java:1583) [?:?] 2024-10-05T07:01:32.903023485Z stdout F Caused by: java.lang.RuntimeException: Cannot publish to S3: Storage backend has reached its minimum free drive threshold. Please delete a few objects to proceed. (Service: Amazon S3; Status Code: 507; Error Code: XMinioStorageFull; Request ID: 17FB7C6D93A48F64; S3 Extended Request ID: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8; Proxy: null) 2024-10-05T07:01:32.903039186Z stdout F at com.van.logging.aws.S3PublishHelper.publishFile(S3PublishHelper.java:131) ~[appender-core-5.3.2.jar:?] 2024-10-05T07:01:32.903042386Z stdout F at com.van.logging.AbstractFilePublishHelper.end(AbstractFilePublishHelper.java:61) ~[appender-core-5.3.2.jar:?] 2024-10-05T07:01:32.903045086Z stdout F ... 8 more 2024-10-05T07:01:32.903047987Z stdout F Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Storage backend has reached its minimum free drive threshold. Please delete a few objects to proceed. (Service: Amazon S3; Status Code: 507; Error Code: XMinioStorageFull; Request ID: 17FB7C6D93A48F64; S3 Extended Request ID: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8; Proxy: null) 2024-10-05T07:01:32.903051487Z stdout F at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1880) ~[aws-java-sdk-core-1.12.770.jar:?] 2024-10-05T07:01:32.903054387Z stdout F at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1418) ~[aws-java-sdk-core-1.12.770.jar:?] 2024-10-05T07:01:32.903057187Z stdout F at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1387) ~[aws-java-sdk-core-1.12.770.jar:?] 2024-10-05T07:01:32.903060088Z stdout F at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157) ~[aws-java-sdk-core-1.12.770.jar:?] 2024-10-05T07:01:32.903065288Z stdout F at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814) ~[aws-java-sdk-core-1.12.770.jar:?] 2024-10-05T07:01:32.90309629Z stdout F at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781) ~[aws-java-sdk-core-1.12.770.jar:?] 2024-10-05T07:01:32.903099991Z stdout F at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755) ~[aws-java-sdk-core-1.12.770.jar:?] 2024-10-05T07:01:32.903102891Z stdout F at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715) ~[aws-java-sdk-core-1.12.770.jar:?] 2024-10-05T07:01:32.903105591Z stdout F at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697) ~[aws-java-sdk-core-1.12.770.jar:?] 2024-10-05T07:01:32.903108491Z stdout F at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561) ~[aws-java-sdk-core-1.12.770.jar:?] 2024-10-05T07:01:32.903111192Z stdout F at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541) ~[aws-java-sdk-core-1.12.770.jar:?] 2024-10-05T07:01:32.903113892Z stdout F at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5575) ~[aws-java-sdk-s3-1.12.770.jar:?] 2024-10-05T07:01:32.903116692Z stdout F at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5522) ~[aws-java-sdk-s3-1.12.770.jar:?] 2024-10-05T07:01:32.903122993Z stdout F at com.amazonaws.services.s3.AmazonS3Client.access$300(AmazonS3Client.java:425) ~[aws-java-sdk-s3-1.12.770.jar:?] 2024-10-05T07:01:32.903125793Z stdout F at com.amazonaws.services.s3.AmazonS3Client$PutObjectStrategy.invokeServiceCall(AmazonS3Client.java:6656) ~[aws-java-sdk-s3-1.12.770.jar:?] 2024-10-05T07:01:32.903128593Z stdout F at com.amazonaws.services.s3.AmazonS3Client.uploadObject(AmazonS3Client.java:1908) ~[aws-java-sdk-s3-1.12.770.jar:?] 2024-10-05T07:01:32.903131293Z stdout F at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1868) ~[aws-java-sdk-s3-1.12.770.jar:?] 2024-10-05T07:01:32.903133994Z stdout F at com.van.logging.aws.S3PublishHelper.publishFile(S3PublishHelper.java:126) ~[appender-core-5.3.2.jar:?] 2024-10-05T07:01:32.903141294Z stdout F at com.van.logging.AbstractFilePublishHelper.end(AbstractFilePublishHelper.java:61) ~[appender-core-5.3.2.jar:?] 2024-10-05T07:01:32.903144194Z stdout F ... 8 more
Beta Was this translation helpful? Give feedback.
All reactions