Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Terraform plugin framework #3889

Draft
wants to merge 21 commits into
base: main
Choose a base branch
from
Draft

Commits on Jul 4, 2024

  1. Add codegen template for tfsdk struct (#3713)

    * Add codegen template for tfsdk struct
    
    * updated map and list
    
    * -
    
    * remove tf type for list and map
    
    * comments
    
    * added test again
    parthban-db authored Jul 4, 2024
    Configuration menu
    Copy the full SHA
    a6e54e1 View commit details
    Browse the repository at this point in the history

Commits on Jul 9, 2024

  1. update (#3738)

    edwardfeng-db authored Jul 9, 2024
    Configuration menu
    Copy the full SHA
    60f0597 View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2024

  1. update (#3759)

    edwardfeng-db authored Jul 11, 2024
    Configuration menu
    Copy the full SHA
    be7d694 View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2024

  1. [Internal] Bumped terraform-plugin-framework branch to latest main (

    #3780)
    
    ## Changes
    <!-- Summary of your changes that are easy to understand -->
    Rebase the Plugin Framework branch to latest main + resolve conflicts:
    9e8fd30
    
    This needs to be merged before
    #3719
    can be merged. Rebasing the PR over main will lead to many
    commits/changes so separating them in this PR.
    
    ## Tests
    <!-- 
    How is this tested? Please see the checklist below and also describe any
    other relevant tests
    -->
    Unit tests 
    - [ ] `make test` run locally
    - [ ] relevant change in `docs/` folder
    - [ ] covered with integration tests in `internal/acceptance`
    - [ ] relevant acceptance tests are passing
    - [ ] using Go SDK
    
    ---------
    
    Signed-off-by: dependabot[bot] <[email protected]>
    Co-authored-by: vuong-nguyen <[email protected]>
    Co-authored-by: Alex Moschos <[email protected]>
    Co-authored-by: hectorcast-db <[email protected]>
    Co-authored-by: Aleksandar Dragojević <[email protected]>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    Co-authored-by: Alex Ott <[email protected]>
    Co-authored-by: Karol <[email protected]>
    Co-authored-by: touchida <[email protected]>
    Co-authored-by: Miles Yucht <[email protected]>
    Co-authored-by: Pieter Noordhuis <[email protected]>
    Co-authored-by: mkubicek <[email protected]>
    Co-authored-by: Renaud Hartert <[email protected]>
    Co-authored-by: Divyansh Vijayvergia <[email protected]>
    14 people authored Jul 17, 2024
    Configuration menu
    Copy the full SHA
    1f88025 View commit details
    Browse the repository at this point in the history
  2. [Feature] Added support to use protocol version 6 provider server for…

    … SDK plugin (#3719)
    
    ## Changes
    <!-- Summary of your changes that are easy to understand -->
    Upgrade SDK plugin to use protocol version 6 as this will be used
    further for introducing plugin framework. We need to mux them (i.e.
    support both sdkv2 and plugin framework) until all resources are
    migrated to plugin framework.
    
    Reference for PR on main branch:
    #3714,
    this PR against the main branch will be used to get the binary for
    testing with data team.
    
    ## Tests
    <!-- 
    How is this tested? Please see the checklist below and also describe any
    other relevant tests
    -->
    All Unit and Integration test passed.
    - [x] `make test` run locally
    - [ ] relevant change in `docs/` folder
    - [ ] covered with integration tests in `internal/acceptance`
    - [x] relevant acceptance tests are passing
    - [ ] using Go SDK
    tanmay-db authored Jul 17, 2024
    Configuration menu
    Copy the full SHA
    3f3ad23 View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2024

  1. Implement basic functionality for StructToSchema (#3788)

    ## Changes
    <!-- Summary of your changes that are easy to understand -->
    - Baisc functionality for `StructToSchema`, everything is `Optional` now
    - Need to do:
      - Implement equivalent for `CustomizableSchema`
      - Make tfsdk structs contain information about optional and computed
      - Handle recursive data types
    
    ## Tests
    <!-- 
    How is this tested? Please see the checklist below and also describe any
    other relevant tests
    -->
    
    - [x] `make test` run locally
    - [x] relevant change in `docs/` folder
    - [x] covered with integration tests in `internal/acceptance`
    - [x] relevant acceptance tests are passing
    - [x] using Go SDK
    edwardfeng-db authored Jul 19, 2024
    Configuration menu
    Copy the full SHA
    cf5d4bb View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2024

  1. Add optional tags to tfsdk struct + Update StructToSchema (#3801)

    ## Changes
    <!-- Summary of your changes that are easy to understand -->
    - Added new `tf` tag in the `tfsdk` structs
    - Made `StructToSchema` able to parse and handle optional vs required
    fields
    
    ## Tests
    <!-- 
    How is this tested? Please see the checklist below and also describe any
    other relevant tests
    -->
    
    - [x] `make test` run locally
    - [x] relevant change in `docs/` folder
    - [x] covered with integration tests in `internal/acceptance`
    - [x] relevant acceptance tests are passing
    - [x] using Go SDK
    edwardfeng-db authored Jul 23, 2024
    Configuration menu
    Copy the full SHA
    76b0745 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2024

  1. [Fix] Make plugin framework infra functions work for Enum types (#3813)

    ## Changes
    <!-- Summary of your changes that are easy to understand -->
    - Make the two converter functions + StructToSchema work for Enum types
    - We do not need to use `types.String` for Enum types as they do not
    have a force send zero value problem
    - Adding a unit test test case
    
    ## Tests
    <!-- 
    How is this tested? Please see the checklist below and also describe any
    other relevant tests
    -->
    
    - [x] `make test` run locally
    - [x] relevant change in `docs/` folder
    - [x] covered with integration tests in `internal/acceptance`
    - [x] relevant acceptance tests are passing
    - [x] using Go SDK
    edwardfeng-db authored Jul 24, 2024
    Configuration menu
    Copy the full SHA
    f94bd5d View commit details
    Browse the repository at this point in the history

Commits on Jul 27, 2024

  1. Make CustomizableSchema work for Plugin Framework (#3819)

    ## Changes
    <!-- Summary of your changes that are easy to understand -->
    - Added basic supports for CustomizableSchema
    - Able to add, remove fields and navigate through the schema
    - Feature support matrix:
    https://docs.google.com/document/d/1ofoLi9vlu-4L-89kBOYPbpOpPk0GZghZcC5SgnBH4ZM/edit#heading=h.wi3qt4pwk7q4
    
    ## Tests
    <!-- 
    How is this tested? Please see the checklist below and also describe any
    other relevant tests
    -->
    
    - [x] `make test` run locally
    - [x] relevant change in `docs/` folder
    - [x] covered with integration tests in `internal/acceptance`
    - [x] relevant acceptance tests are passing
    - [x] using Go SDK
    edwardfeng-db authored Jul 27, 2024
    Configuration menu
    Copy the full SHA
    1562784 View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2024

  1. [Feature] Introduced Terraform Plugin Framework (#3751)

    ## Changes
    <!-- Summary of your changes that are easy to understand -->
    This PR introduces plugin framework to the Databricks Terraform
    Provider.
    - Introduce plugin framework
    - Add core support for resource and data source. 
    - Add provider schema
    - Add support to configure databricks client
    - Add support for provider fixture
    - Add support to get raw config to be used in provider tests and further
    in unit testing framework.
    - Add Quality Monitor resource using plugin framework
    - Add UC Volumes data source using plugin framework
    - Fix bugs related in configuring client
    - Fix bugs where provider schema is used instead of resource schema
    - Fix miscellaneous bugs 
    
    ## Tests
    <!-- 
    How is this tested? Please see the checklist below and also describe any
    other relevant tests
    -->
    - Run existing provider tests also for plugin framework. All passes. 
    - Manually tested the above 2 resources E2E (apply works for data
    source, for resource -- we need customise schema support for
    `SetReadOnly`)
    
    Note: We don't have unit and integration test support as of now, so
    going ahead with commiting the .tf files for these resources so we can
    test the changes in the meantime. Once we have proper support, these can
    be converted to example or removed.
    
    <img width="569" alt="image"
    src="https://github.com/user-attachments/assets/869cacd7-a471-49db-93de-1a7bbb8b5ea7">
    
    ---------
    
    Signed-off-by: dependabot[bot] <[email protected]>
    Co-authored-by: vuong-nguyen <[email protected]>
    Co-authored-by: Alex Moschos <[email protected]>
    Co-authored-by: hectorcast-db <[email protected]>
    Co-authored-by: Aleksandar Dragojević <[email protected]>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    Co-authored-by: Alex Ott <[email protected]>
    Co-authored-by: Karol <[email protected]>
    Co-authored-by: touchida <[email protected]>
    Co-authored-by: Miles Yucht <[email protected]>
    10 people authored Jul 29, 2024
    Configuration menu
    Copy the full SHA
    51411f1 View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2024

  1. Support read only in plugin framework CustomizableSchema (#3842)

    ## Changes
    <!-- Summary of your changes that are easy to understand -->
    - Support `SetReadOnly` in `CustomizableSchema` for terraform plugin
    framework
    
    ## Tests
    <!-- 
    How is this tested? Please see the checklist below and also describe any
    other relevant tests
    -->
    
    - [x] `make test` run locally
    - [x] relevant change in `docs/` folder
    - [x] covered with integration tests in `internal/acceptance`
    - [x] relevant acceptance tests are passing
    - [x] using Go SDK
    edwardfeng-db authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    1f4a383 View commit details
    Browse the repository at this point in the history

Commits on Aug 6, 2024

  1. Bump up terraform plugin framework version (#3860)

    ## Changes
    <!-- Summary of your changes that are easy to understand -->
    - Bump up to the latest released version
    
    ## Tests
    <!-- 
    How is this tested? Please see the checklist below and also describe any
    other relevant tests
    -->
    
    - [x] `make test` run locally
    - [x] relevant change in `docs/` folder
    - [x] covered with integration tests in `internal/acceptance`
    - [x] relevant acceptance tests are passing
    - [x] using Go SDK
    edwardfeng-db authored Aug 6, 2024
    Configuration menu
    Copy the full SHA
    f3f1ef0 View commit details
    Browse the repository at this point in the history
  2. Update resource quality monitor in plugin framework (#3853)

    ## Changes
    <!-- Summary of your changes that are easy to understand -->
    - Part of making acceptance test pass:
    #3841
    - Update `ResourceQualityMonitor` to make it pass acceptance test
    - Commented out adding two new fields temporarily because we need to
    have those fields in the tfsdk struct, otherwise `.get()` and `.set()`
    don't work
    - Updated the target struct for `.get` calls to be `MonitorInfo` because
    terraform requires that to be consistent with the schema
    - Added logic at the end of `Create` and `Update` to read the latest
    `monitorInfo` and set the state
    
    ## Tests
    <!-- 
    How is this tested? Please see the checklist below and also describe any
    other relevant tests
    -->
    
    - [x] `make test` run locally
    - [x] relevant change in `docs/` folder
    - [x] covered with integration tests in `internal/acceptance`
    - [x] relevant acceptance tests are passing
    - [x] using Go SDK
    edwardfeng-db authored Aug 6, 2024
    Configuration menu
    Copy the full SHA
    3407da1 View commit details
    Browse the repository at this point in the history
  3. Make tfsdk struct enums into types.String (#3852)

    ## Changes
    <!-- Summary of your changes that are easy to understand -->
    - These enums have to be strings instead of the indirect types that they
    used to be, otherwise `.get` and `.set` does not work properly
    - First part of making the acceptance test work as specified here
    #3841
    
    ## Tests
    <!-- 
    How is this tested? Please see the checklist below and also describe any
    other relevant tests
    -->
    
    - [ ] `make test` run locally
    - [ ] relevant change in `docs/` folder
    - [ ] covered with integration tests in `internal/acceptance`
    - [ ] relevant acceptance tests are passing
    - [ ] using Go SDK
    edwardfeng-db authored Aug 6, 2024
    Configuration menu
    Copy the full SHA
    4de413d View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2024

  1. Update reflect_resource_plugin_framework to make acceptance test pass (

    …#3854)
    
    ## Changes
    <!-- Summary of your changes that are easy to understand -->
    - Subpart of
    #3841
    - Updated reflect_resource_plugin_framework to handle cases
    - When a field in the given source struct is nil, the converter function
    has to be able to handle it
      - Handle enums in tfsdk structs as types.String
    - When a field does not exist on the destination struct, ignore the
    field
    - Added unit test for the cases mentioned above
    
    ## Tests
    <!-- 
    How is this tested? Please see the checklist below and also describe any
    other relevant tests
    -->
    
    - [x] `make test` run locally
    - [x] relevant change in `docs/` folder
    - [x] covered with integration tests in `internal/acceptance`
    - [x] relevant acceptance tests are passing
    - [x] using Go SDK
    edwardfeng-db authored Aug 7, 2024
    Configuration menu
    Copy the full SHA
    05692c9 View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2024

  1. Remove AddNewField and RemoveField for customizable_schema_plugin_fra…

    …mework (#3868)
    
    ## Changes
    <!-- Summary of your changes that are easy to understand -->
    - Removing these two methods as they will cause the schema and the
    struct to be inconsistent
    
    ## Tests
    <!-- 
    How is this tested? Please see the checklist below and also describe any
    other relevant tests
    -->
    
    - [x] `make test` run locally
    - [x] relevant change in `docs/` folder
    - [x] covered with integration tests in `internal/acceptance`
    - [x] relevant acceptance tests are passing
    - [x] using Go SDK
    edwardfeng-db authored Aug 8, 2024
    Configuration menu
    Copy the full SHA
    267b3e8 View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2024

  1. Update init test and migrate imports for plugin framework (#3867)

    ## Changes
    <!-- Summary of your changes that are easy to understand -->
    - Part of
    #3841
    - Migrated imports following the documentation
    [here](https://developer.hashicorp.com/terraform/plugin/testing/migrating)
      - 
    <img width="688" alt="image"
    src="https://github.com/user-attachments/assets/f2f14175-f3f8-47b7-a19f-1f0fca13eb47">
    
    - Made `run` in `init_test` able to take both plugin framework resources
    as well as sdkv2 resources
    
    ## Tests
    <!-- 
    How is this tested? Please see the checklist below and also describe any
    other relevant tests
    -->
    
    - [x] `make test` run locally
    - [x] relevant change in `docs/` folder
    - [x] covered with integration tests in `internal/acceptance`
    - [x] relevant acceptance tests are passing
    - [x] using Go SDK
    edwardfeng-db authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    4ad0fbb View commit details
    Browse the repository at this point in the history
  2. Add quality monitor acceptance test for plugin framework (#3876)

    ## Changes
    <!-- Summary of your changes that are easy to understand -->
    - Add two more test cases replicating existing test case for quality
    monitor resource, but for the plugin framework
    - Manually executed the tests to make sure they are passing
    - Taking out the manual read check at the end of each test step as
    suggested
    [here](#3867 (comment))
    because the type casting for clients is causing issue for plugin
    framework tests
    
    ## Tests
    <!-- 
    How is this tested? Please see the checklist below and also describe any
    other relevant tests
    -->
    
    - [x] `make test` run locally
    - [x] relevant change in `docs/` folder
    - [x] covered with integration tests in `internal/acceptance`
    - [x] relevant acceptance tests are passing
    - [x] using Go SDK
    edwardfeng-db authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    6ff85a8 View commit details
    Browse the repository at this point in the history
  3. Add acceptance test for volumes data source (#3879)

    ## Changes
    <!-- Summary of your changes that are easy to understand -->
    - Make `ids` computed
    - Add test case for `data_volumes_pluginframework`
    
    ## Tests
    <!-- 
    How is this tested? Please see the checklist below and also describe any
    other relevant tests
    -->
    
    - [ ] `make test` run locally
    - [ ] relevant change in `docs/` folder
    - [ ] covered with integration tests in `internal/acceptance`
    - [ ] relevant acceptance tests are passing
    - [ ] using Go SDK
    edwardfeng-db authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    79121d6 View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2024

  1. Update tfsdk struct generation template (#3906)

    ## Changes
    <!-- Summary of your changes that are easy to understand -->
    - Addressing comments from
    databricks/universe#682431
    - Specifically
      - Removed unnecessary parts of the codegen template (map, enum)
      - Moved generated package into `internal/services`
    
    ## Tests
    <!-- 
    How is this tested? Please see the checklist below and also describe any
    other relevant tests
    -->
    
    - [x] `make test` run locally
    - [x] relevant change in `docs/` folder
    - [x] covered with integration tests in `internal/acceptance`
    - [x] relevant acceptance tests are passing
    - [x] using Go SDK
    edwardfeng-db authored Aug 15, 2024
    Configuration menu
    Copy the full SHA
    c4e9c06 View commit details
    Browse the repository at this point in the history
  2. [Internal] Rebase and Fixed merge conflicts with main branch. Also fi…

    …xed notification_destination acceptance tests. (#3892)
    
    ## Changes
    <!-- Summary of your changes that are easy to understand -->
    - Rebase to latest main
    - Fixed merge conflicts
    - Fixed notifications tests as they are incompatible with
    terraform-plugin-testing because the test use a different type instead
    of directly using the underlying type `func(*terraform.State) error`
    
    ## Tests
    <!-- 
    How is this tested? Please see the checklist below and also describe any
    other relevant tests
    -->
    Existing unit tests + Verified that no unexpected changes are there by
    creating draft PR over main:
    #3893.
    - [ ] `make test` run locally
    - [ ] relevant change in `docs/` folder
    - [ ] covered with integration tests in `internal/acceptance`
    - [ ] relevant acceptance tests are passing
    - [ ] using Go SDK
    
    ---------
    
    Signed-off-by: dependabot[bot] <[email protected]>
    Co-authored-by: vuong-nguyen <[email protected]>
    Co-authored-by: Alex Moschos <[email protected]>
    Co-authored-by: hectorcast-db <[email protected]>
    Co-authored-by: Aleksandar Dragojević <[email protected]>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    Co-authored-by: Alex Ott <[email protected]>
    Co-authored-by: Karol <[email protected]>
    Co-authored-by: touchida <[email protected]>
    Co-authored-by: Miles Yucht <[email protected]>
    Co-authored-by: Pieter Noordhuis <[email protected]>
    Co-authored-by: mkubicek <[email protected]>
    Co-authored-by: Renaud Hartert <[email protected]>
    Co-authored-by: Divyansh Vijayvergia <[email protected]>
    Co-authored-by: Arpit Jasapara <[email protected]>
    Co-authored-by: Cedric <[email protected]>
    Co-authored-by: Alex Ott <[email protected]>
    17 people authored Aug 15, 2024
    Configuration menu
    Copy the full SHA
    c5b5e5f View commit details
    Browse the repository at this point in the history