Support stateless queries. #184
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
From https://cloud.google.com/java/docs/reference/google-cloud-bigquery/2.34.1/com.google.cloud.bigquery.BigQuery > Stateless queries: query execution without corresponding job metadata Stateless queries are currently in preview. This PR introduces support for stateless queries via a new JDBC query parameter. Set `jobcreationmode` to a string that matches one of the standard `JobCreationMode` enum values: - `JOB_CREATION_MODE_UNSPECIFIED`: Unspecified JobCreationMode, defaults to `JOB_CREATION_REQUIRED`. - `JOB_CREATION_REQUIRED`: Default. Job creation is always required. - `JOB_CREATION_OPTIONAL`: Job creation is optional. Returning immediate results is prioritized. BigQuery will automatically determine if a Job needs to be created. The conditions under which BigQuery can decide to not create a Job are subject to change. If Job creation is required, JOB_CREATION_REQUIRED mode should be used, which is the default. See https://github.com/googleapis/java-bigquery/blob/v2.34.0/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/QueryJobConfiguration.java#L98-L111 For example, the following create a connection with optional job creation: DriverManager.getConnection( "jdbc:BQDriver:my-project?jobcreationmode=JOB_CREATION_OPTIONAL", driverProperties); Note that this will cause queries to fail if the provided project does not have stateless queries preview enabled.
goomrw
force-pushed
the
goomrw-bq-stateless-query
branch
from
7ca2597
to
efef4cc
Compare
fzakaria
reviewed
Dec 13, 2023
goomrw
commented
Dec 13, 2023
This adds a Java Microbenchmark Harness (JMH) benchmark that compares a small query with and without job creation. Documentation on running the microbenchmark can be found in its class' Javadoc. A small query with job creation takes almost twice as long as one with optional job creation.
Method references need # not .
goomrw
commented
Dec 14, 2023
src/test/java/net/starschema/clouddb/jdbc/ConnectionFromResources.java
Outdated
Show resolved
Hide resolved
|
I think we have a flaky test: |
|
71bc3a9 added a microbenchmark. Here's a representative summary: Queries with optional jobs are between 1.4x and 2.3x faster than required jobs! |
fzakaria
approved these changes
Dec 18, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
From
https://cloud.google.com/java/docs/reference/google-cloud-bigquery/2.34.1/com.google.cloud.bigquery.BigQuery
Stateless queries are currently in preview.
This PR introduces support for stateless queries via a new JDBC query parameter. Set
jobcreationmodeto a string that matches one of the standardJobCreationModeenum values:JOB_CREATION_MODE_UNSPECIFIED: Unspecified JobCreationMode, defaults toJOB_CREATION_REQUIRED.JOB_CREATION_REQUIRED: Default. Job creation is always required.JOB_CREATION_OPTIONAL: Job creation is optional. Returning immediate results is prioritized. BigQuery will automatically determine if a Job needs to be created. The conditions under which BigQuery can decide to not create a Job are subject to change. If Job creation is required, JOB_CREATION_REQUIRED mode should be used, which is the default.See
https://github.com/googleapis/java-bigquery/blob/v2.34.0/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/QueryJobConfiguration.java#L98-L111
For example, the following create a connection with optional job creation:
Note that this will cause queries to fail if the provided project does not have stateless queries preview enabled.