Skip to content

Commit

Permalink
feat: upgrade to latest terraform generator with prefilled fields: he…
Browse files Browse the repository at this point in the history
…nce less configuration needed
  • Loading branch information
ThomasRooney committed Oct 5, 2023
1 parent 6e55dd8 commit 09337ac
Show file tree
Hide file tree
Showing 4,616 changed files with 99,709 additions and 39,617 deletions.
The diff you're trying to view is too large. We only load the first 3000 changed files.
Empty file modified .gitattributes
100755 → 100644
Empty file.
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,16 @@ TF_REATTACH_PROVIDERS=... terraform apply

<!-- End SDK Available Operations -->



<!-- Start Dev Containers -->

<!-- End Dev Containers -->

<!-- Placeholder for Future Speakeasy SDK Sections -->



Terraform allows you to use local provider builds by setting a `dev_overrides` block in a configuration file called `.terraformrc`. This block overrides all other configured installation methods.

Terraform searches for the `.terraformrc` file in your home directory and applies any configuration settings you set.
Expand Down
Empty file modified USAGE.md
100755 → 100644
Empty file.
2 changes: 1 addition & 1 deletion airbyte.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -51281,7 +51281,7 @@ components:
type: "object"
title: "SSH Tunnel Method"
description: "Whether to initiate an SSH tunnel before connecting to the\
\ database, and if so, which kind of authentication to use."
\ database, and iftatus so, which kind of authentication to use."
oneOf:
- title: "No Tunnel"
required:
Expand Down
9 changes: 0 additions & 9 deletions docs/data-sources/destination_aws_datalake.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,6 @@ Read-Only:
- `bucket_name` (String) The name of the S3 bucket. Read more <a href="https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html">here</a>.
- `bucket_prefix` (String) S3 prefix
- `credentials` (Attributes) Choose How to Authenticate to AWS. (see [below for nested schema](#nestedatt--configuration--credentials))
- `destination_type` (String) must be one of ["aws-datalake"]
- `format` (Attributes) Format of the data output. (see [below for nested schema](#nestedatt--configuration--format))
- `glue_catalog_float_as_decimal` (Boolean) Cast float/double as decimal(38,18). This can help achieve higher accuracy and represent numbers correctly as received from the source.
- `lakeformation_database_default_tag_key` (String) Add a default tag key to databases created by this destination
Expand All @@ -67,8 +66,6 @@ Read-Only:

Read-Only:

- `credentials_title` (String) must be one of ["IAM Role"]
Name of the credentials
- `role_arn` (String) Will assume this role to write data to s3


Expand All @@ -79,17 +76,13 @@ Read-Only:

- `aws_access_key_id` (String) AWS User Access Key Id
- `aws_secret_access_key` (String) Secret Access Key
- `credentials_title` (String) must be one of ["IAM User"]
Name of the credentials


<a id="nestedatt--configuration--credentials--destination_aws_datalake_update_authentication_mode_iam_role"></a>
### Nested Schema for `configuration.credentials.destination_aws_datalake_update_authentication_mode_iam_role`

Read-Only:

- `credentials_title` (String) must be one of ["IAM Role"]
Name of the credentials
- `role_arn` (String) Will assume this role to write data to s3


Expand All @@ -100,8 +93,6 @@ Read-Only:

- `aws_access_key_id` (String) AWS User Access Key Id
- `aws_secret_access_key` (String) Secret Access Key
- `credentials_title` (String) must be one of ["IAM User"]
Name of the credentials



Expand Down
11 changes: 0 additions & 11 deletions docs/data-sources/destination_azure_blob_storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,6 @@ Read-Only:
- `azure_blob_storage_endpoint_domain_name` (String) This is Azure Blob Storage endpoint domain name. Leave default value (or leave it empty if run container from command line) to use Microsoft native from example.
- `azure_blob_storage_output_buffer_size` (Number) The amount of megabytes to buffer for the output stream to Azure. This will impact memory footprint on workers, but may need adjustment for performance and appropriate block size in Azure.
- `azure_blob_storage_spill_size` (Number) The amount of megabytes after which the connector should spill the records in a new blob object. Make sure to configure size greater than individual records. Enter 0 if not applicable
- `destination_type` (String) must be one of ["azure-blob-storage"]
- `format` (Attributes) Output data format (see [below for nested schema](#nestedatt--configuration--format))

<a id="nestedatt--configuration--format"></a>
Expand All @@ -62,16 +61,11 @@ Read-Only:

- `flattening` (String) must be one of ["No flattening", "Root level flattening"]
Whether the input json data should be normalized (flattened) in the output CSV. Please refer to docs for details.
- `format_type` (String) must be one of ["CSV"]


<a id="nestedatt--configuration--format--destination_azure_blob_storage_output_format_json_lines_newline_delimited_json"></a>
### Nested Schema for `configuration.format.destination_azure_blob_storage_output_format_json_lines_newline_delimited_json`

Read-Only:

- `format_type` (String) must be one of ["JSONL"]


<a id="nestedatt--configuration--format--destination_azure_blob_storage_update_output_format_csv_comma_separated_values"></a>
### Nested Schema for `configuration.format.destination_azure_blob_storage_update_output_format_csv_comma_separated_values`
Expand All @@ -80,14 +74,9 @@ Read-Only:

- `flattening` (String) must be one of ["No flattening", "Root level flattening"]
Whether the input json data should be normalized (flattened) in the output CSV. Please refer to docs for details.
- `format_type` (String) must be one of ["CSV"]


<a id="nestedatt--configuration--format--destination_azure_blob_storage_update_output_format_json_lines_newline_delimited_json"></a>
### Nested Schema for `configuration.format.destination_azure_blob_storage_update_output_format_json_lines_newline_delimited_json`

Read-Only:

- `format_type` (String) must be one of ["JSONL"]


29 changes: 8 additions & 21 deletions docs/data-sources/destination_bigquery.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@ Read-Only:
- `dataset_id` (String) The default BigQuery Dataset ID that tables are replicated to if the source does not specify a namespace. Read more <a href="https://cloud.google.com/bigquery/docs/datasets#create-dataset">here</a>.
- `dataset_location` (String) must be one of ["US", "EU", "asia-east1", "asia-east2", "asia-northeast1", "asia-northeast2", "asia-northeast3", "asia-south1", "asia-south2", "asia-southeast1", "asia-southeast2", "australia-southeast1", "australia-southeast2", "europe-central1", "europe-central2", "europe-north1", "europe-southwest1", "europe-west1", "europe-west2", "europe-west3", "europe-west4", "europe-west6", "europe-west7", "europe-west8", "europe-west9", "me-west1", "northamerica-northeast1", "northamerica-northeast2", "southamerica-east1", "southamerica-west1", "us-central1", "us-east1", "us-east2", "us-east3", "us-east4", "us-east5", "us-west1", "us-west2", "us-west3", "us-west4"]
The location of the dataset. Warning: Changes made after creation will not be applied. Read more <a href="https://cloud.google.com/bigquery/docs/locations">here</a>.
- `destination_type` (String) must be one of ["bigquery"]
- `loading_method` (Attributes) Loading method used to send select the way data will be uploaded to BigQuery. <br/><b>Standard Inserts</b> - Direct uploading using SQL INSERT statements. This method is extremely inefficient and provided only for quick testing. In almost all cases, you should use staging. <br/><b>GCS Staging</b> - Writes large batches of records to a file, uploads the file to GCS, then uses <b>COPY INTO table</b> to upload the file. Recommended for most workloads for better speed and scalability. Read more about GCS Staging <a href="https://docs.airbyte.com/integrations/destinations/bigquery#gcs-staging">here</a>. (see [below for nested schema](#nestedatt--configuration--loading_method))
- `project_id` (String) The GCP project ID for the project containing the target BigQuery dataset. Read more <a href="https://cloud.google.com/resource-manager/docs/creating-managing-projects#identifying_projects">here</a>.
- `raw_data_dataset` (String) The dataset to write raw tables into
Expand Down Expand Up @@ -69,21 +68,19 @@ Read-Only:
- `gcs_bucket_path` (String) Directory under the GCS bucket where data will be written.
- `keep_files_in_gcs_bucket` (String) must be one of ["Delete all tmp files from GCS", "Keep all tmp files in GCS"]
This upload method is supposed to temporary store records in GCS bucket. By this select you can chose if these records should be removed from GCS when migration has finished. The default "Delete all tmp files from GCS" value is used if not set explicitly.
- `method` (String) must be one of ["GCS Staging"]

<a id="nestedatt--configuration--loading_method--destination_bigquery_loading_method_gcs_staging--credential"></a>
### Nested Schema for `configuration.loading_method.destination_bigquery_loading_method_gcs_staging.method`
### Nested Schema for `configuration.loading_method.destination_bigquery_loading_method_gcs_staging.keep_files_in_gcs_bucket`

Read-Only:

- `destination_bigquery_loading_method_gcs_staging_credential_hmac_key` (Attributes) An HMAC key is a type of credential and can be associated with a service account or a user account in Cloud Storage. Read more <a href="https://cloud.google.com/storage/docs/authentication/hmackeys">here</a>. (see [below for nested schema](#nestedatt--configuration--loading_method--destination_bigquery_loading_method_gcs_staging--method--destination_bigquery_loading_method_gcs_staging_credential_hmac_key))
- `destination_bigquery_loading_method_gcs_staging_credential_hmac_key` (Attributes) An HMAC key is a type of credential and can be associated with a service account or a user account in Cloud Storage. Read more <a href="https://cloud.google.com/storage/docs/authentication/hmackeys">here</a>. (see [below for nested schema](#nestedatt--configuration--loading_method--destination_bigquery_loading_method_gcs_staging--keep_files_in_gcs_bucket--destination_bigquery_loading_method_gcs_staging_credential_hmac_key))

<a id="nestedatt--configuration--loading_method--destination_bigquery_loading_method_gcs_staging--method--destination_bigquery_loading_method_gcs_staging_credential_hmac_key"></a>
### Nested Schema for `configuration.loading_method.destination_bigquery_loading_method_gcs_staging.method.destination_bigquery_loading_method_gcs_staging_credential_hmac_key`
<a id="nestedatt--configuration--loading_method--destination_bigquery_loading_method_gcs_staging--keep_files_in_gcs_bucket--destination_bigquery_loading_method_gcs_staging_credential_hmac_key"></a>
### Nested Schema for `configuration.loading_method.destination_bigquery_loading_method_gcs_staging.keep_files_in_gcs_bucket.destination_bigquery_loading_method_gcs_staging_credential_hmac_key`

Read-Only:

- `credential_type` (String) must be one of ["HMAC_KEY"]
- `hmac_key_access_id` (String) HMAC key access ID. When linked to a service account, this ID is 61 characters long; when linked to a user account, it is 24 characters long.
- `hmac_key_secret` (String) The corresponding secret for the access ID. It is a 40-character base-64 encoded string.

Expand All @@ -93,10 +90,6 @@ Read-Only:
<a id="nestedatt--configuration--loading_method--destination_bigquery_loading_method_standard_inserts"></a>
### Nested Schema for `configuration.loading_method.destination_bigquery_loading_method_standard_inserts`

Read-Only:

- `method` (String) must be one of ["Standard"]


<a id="nestedatt--configuration--loading_method--destination_bigquery_update_loading_method_gcs_staging"></a>
### Nested Schema for `configuration.loading_method.destination_bigquery_update_loading_method_gcs_staging`
Expand All @@ -109,21 +102,19 @@ Read-Only:
- `gcs_bucket_path` (String) Directory under the GCS bucket where data will be written.
- `keep_files_in_gcs_bucket` (String) must be one of ["Delete all tmp files from GCS", "Keep all tmp files in GCS"]
This upload method is supposed to temporary store records in GCS bucket. By this select you can chose if these records should be removed from GCS when migration has finished. The default "Delete all tmp files from GCS" value is used if not set explicitly.
- `method` (String) must be one of ["GCS Staging"]

<a id="nestedatt--configuration--loading_method--destination_bigquery_update_loading_method_gcs_staging--credential"></a>
### Nested Schema for `configuration.loading_method.destination_bigquery_update_loading_method_gcs_staging.method`
### Nested Schema for `configuration.loading_method.destination_bigquery_update_loading_method_gcs_staging.keep_files_in_gcs_bucket`

Read-Only:

- `destination_bigquery_update_loading_method_gcs_staging_credential_hmac_key` (Attributes) An HMAC key is a type of credential and can be associated with a service account or a user account in Cloud Storage. Read more <a href="https://cloud.google.com/storage/docs/authentication/hmackeys">here</a>. (see [below for nested schema](#nestedatt--configuration--loading_method--destination_bigquery_update_loading_method_gcs_staging--method--destination_bigquery_update_loading_method_gcs_staging_credential_hmac_key))
- `destination_bigquery_update_loading_method_gcs_staging_credential_hmac_key` (Attributes) An HMAC key is a type of credential and can be associated with a service account or a user account in Cloud Storage. Read more <a href="https://cloud.google.com/storage/docs/authentication/hmackeys">here</a>. (see [below for nested schema](#nestedatt--configuration--loading_method--destination_bigquery_update_loading_method_gcs_staging--keep_files_in_gcs_bucket--destination_bigquery_update_loading_method_gcs_staging_credential_hmac_key))

<a id="nestedatt--configuration--loading_method--destination_bigquery_update_loading_method_gcs_staging--method--destination_bigquery_update_loading_method_gcs_staging_credential_hmac_key"></a>
### Nested Schema for `configuration.loading_method.destination_bigquery_update_loading_method_gcs_staging.method.destination_bigquery_update_loading_method_gcs_staging_credential_hmac_key`
<a id="nestedatt--configuration--loading_method--destination_bigquery_update_loading_method_gcs_staging--keep_files_in_gcs_bucket--destination_bigquery_update_loading_method_gcs_staging_credential_hmac_key"></a>
### Nested Schema for `configuration.loading_method.destination_bigquery_update_loading_method_gcs_staging.keep_files_in_gcs_bucket.destination_bigquery_update_loading_method_gcs_staging_credential_hmac_key`

Read-Only:

- `credential_type` (String) must be one of ["HMAC_KEY"]
- `hmac_key_access_id` (String) HMAC key access ID. When linked to a service account, this ID is 61 characters long; when linked to a user account, it is 24 characters long.
- `hmac_key_secret` (String) The corresponding secret for the access ID. It is a 40-character base-64 encoded string.

Expand All @@ -133,8 +124,4 @@ Read-Only:
<a id="nestedatt--configuration--loading_method--destination_bigquery_update_loading_method_standard_inserts"></a>
### Nested Schema for `configuration.loading_method.destination_bigquery_update_loading_method_standard_inserts`

Read-Only:

- `method` (String) must be one of ["Standard"]


Loading

0 comments on commit 09337ac

Please sign in to comment.