Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug on resource that is not deployed after some errors #547

Closed
irainia opened this issue Aug 30, 2022 · 2 comments
Closed

Bug on resource that is not deployed after some errors #547

irainia opened this issue Aug 30, 2022 · 2 comments
Labels
bug Something isn't working

Comments

@irainia
Copy link
Member

irainia commented Aug 30, 2022

Describe the bug

This bug happens when there's a failure when deploying resource into its service provider, such as bigquery. The steps below should explain better about this case.

To Reproduce
Steps to reproduce the behavior:

  1. create a valid resource, such as view. however, make sure that the resource will be failed when storing to the service provider, like bigquery. for example like access.
  2. deploy to optimus the newly created resource. in this state, optimus should fail the deployment process to the service provider. however, the resource itself is already stored in the database.
  3. don't update the resource spec, just update the access for it to pass the service provider
  4. re-deploy through optimus. and in this step, optimus shouldn't fail. however, optimus will skip deployment process, meaning the resource is not created.

The reason for the result of the steps above is because in Optimus, we have a mechanism to skip resource if there is no difference between the incoming one and the one in database.

Expected behavior
Resource should be stored in database, and deployed to the targetted service provider.

@irainia irainia added the bug Something isn't working label Aug 30, 2022
@irainia irainia added this to the Reliability - 2022 H2 milestone Aug 30, 2022
@sravankorumilli
Copy link
Contributor

Duplicate of #533

@sravankorumilli sravankorumilli marked this as a duplicate of #533 Sep 9, 2022
@deryrahman
Copy link
Member

deryrahman commented Sep 19, 2022

@irainia @sravankorumilli the problem of this issue is because there's a lack of atomicity property in our create / update resource system. Atomicity means all or nothing. In this case, there're 2 main operation in create / update resource (in sequential order):

  1. save the resource spec to DB
  2. actual table creation / modification on service provider

When atomicity property is not provided, for example, whenever access on service provider is problematic, only the first operation is getting executed and the second operation is not being executed. It leads to inconsistency between both operations. We need a mechanism where if one operation is failing, then all operations is not executed.

So far the approach we can take is rolling back when one operation is failing. eg, when creation / modification on service provider is failing, then, resource spec in DB should be rolled back to the previous state.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants