Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide an API to add constraints to a Delta Table via Python #106

Open
robertkossendey opened this issue Mar 13, 2023 · 6 comments
Open
Labels
enhancement New feature or request

Comments

@robertkossendey
Copy link
Collaborator

Correct me if I am wrong, but the only way to currently add table constraints is through the SQL interface.
We should provide an API that allows the user to add, maybe event update and delete constraints from a table via Python.

@robertkossendey robertkossendey added the enhancement New feature or request label Mar 13, 2023
@MrPowers
Copy link
Owner

@robertkossendey - this would be great, but it really feels like this one belongs in Delta itself. Can you open an issue in delta-io/delta and see if we can get it added there? If that's not possible, then I suppose we can add it to mack.

@robertkossendey
Copy link
Collaborator Author

@MrPowers I asked Denny about the Roadmap and whether we should work on such features for mack. I agree that this should be part of the core delta-spark but it is not on the Roadmap yet.

@cs23andris
Copy link
Contributor

Actually, because constraints end up as properties of the Delta table, you can add constraints with the following syntax. It's not elegant but it's an optional workaround:

target_table = (DeltaTable.create(spark)
  .tableName("table_with_constraints")
  .addColumn("col1", dataType = "INT", nullable = False)
  .addColumn("col2", dataType = "STRING", nullable = True)
  .addColumn("col3", dataType = "STRING", nullable = False)
  .property("delta.constraints.col1_constraint","col1 > 0" )
  .execute()
)

@robertkossendey
Copy link
Collaborator Author

Ahhh, thank you very much for letting me know @cs23andris! I always forget about the DeltaTableBuilder API. This does only work on creation though, right?

@cs23andris
Copy link
Contributor

Yes, that's true.
It seems that a DeltaTableBuilder object can only be created with create(), createIfNotExists(), replace() or createOrReplace() methods there's no alter() function.

@MrPowers
Copy link
Owner

@robertkossendey - yea, think we can go ahead and add this feature. Feel free to propose your suggested API and we can brainstorm before it gets built. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants