Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KnowledgeSourceAPI: new_source_data: fix "halferror" case for duplicated lfns in repository; improve API tests (more); manage source data status #79

Open
alebg opened this issue Nov 19, 2023 · 0 comments
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@alebg
Copy link
Collaborator

alebg commented Nov 19, 2023

In the alpha implementation, each source data of the input list (of source data) is attempted to be added and commited to the DB separately. When the repository receives a list of source data with duplicates, it will push only the first one and fail for all the rest. See TODO note in the repository function. The solution might be somehow repristining the sqla session (although trying to session.close() then with self.session.begin() will also fail with a different error).

Overall, a better solution would be managing duplicates somehow first and then commiting in batch, instead of one by one.

Also, the API test is very basic, could cover more cases for the beta.

Also, by default, the source data gets registered as status = "AVAILABLE", but we need a better way of managing this for the beta.

@alebg alebg added the enhancement New feature or request label Nov 19, 2023
@alebg alebg added this to the 1.0.0-beta milestone Nov 19, 2023
@alebg alebg self-assigned this Nov 19, 2023
@alebg alebg changed the title KnowledgeSourceRepository: new_source_data: fix "halferror" case for duplicated lfns KnowledgeSourceAPI: new_source_data: fix "halferror" case for duplicated lfns in repository; improve API tests (more) Nov 19, 2023
@alebg alebg changed the title KnowledgeSourceAPI: new_source_data: fix "halferror" case for duplicated lfns in repository; improve API tests (more) KnowledgeSourceAPI: new_source_data: fix "halferror" case for duplicated lfns in repository; improve API tests (more); manage source data status Nov 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant