Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use unbounded source in BigQueryIOStorageWrite streaming tests #28503

Merged
merged 1 commit into from
Sep 20, 2023

Conversation

Abacn
Copy link
Contributor

@Abacn Abacn commented Sep 18, 2023

Until there is a consensus of #28272, let's get in the part not related to FILE_LOAD first, which fixes BigQueryIOStorageWrite streaming tests currently using bounded source

Introduce PeriodicImpulse.stopAfter()

  • Use it in streaming BigQueryIO integration test

Please add a meaningful description for your change here


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

* Use it in streaming BigQueryIO integration test
@Abacn
Copy link
Contributor Author

Abacn commented Sep 18, 2023

Run PostCommit_Java_Dataflow

https://github.com/apache/beam/actions/runs/6224944915/job/16894302782

timeout after 4 h

@github-actions
Copy link
Contributor

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

@Abacn
Copy link
Contributor Author

Abacn commented Sep 18, 2023

Run Java_hadoop_IO_Direct PreCommit

Copy link
Contributor

@ahmedabu98 ahmedabu98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Abacn
Copy link
Contributor Author

Abacn commented Sep 19, 2023

Run PostCommit_Java_Dataflow

https://github.com/apache/beam/actions/runs/6238076552/job/16933121401

GitHub Action postcommit consistently timeout on master

@Abacn
Copy link
Contributor Author

Abacn commented Sep 19, 2023

https://ci-beam.apache.org/job/beam_PostCommit_Java_DataflowV1_PR/211/

test failures due to #28274 reverted on master. The test now consistently timing out

@Abacn
Copy link
Contributor Author

Abacn commented Sep 19, 2023

Run PostCommit_Java_Dataflow

@Abacn
Copy link
Contributor Author

Abacn commented Sep 20, 2023

java postcommit is becoming HUGE - and with high probability failing

since the test passed before I am go for merge

@Abacn Abacn merged commit 534f93a into apache:master Sep 20, 2023
22 of 25 checks passed
@Abacn Abacn deleted the periodicsequence branch September 20, 2023 17:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants