Feature/3925 oai delete

[richard-jones]

• Issue: https://github.com/DOAJ/doajPM/issues/3925

---

Add deleted records to OAI-PMH

Adds withdrawn Journals and withdrawn and deleted Articles to the OAI-PMH feed with the "deleted" status. This means adding an Article Tombstone type, which records all article deletes and incorporating that data into the PMH feed.

This PR...

• has scripts to run
• has migrations to run
• adds new infrastructure
• changes the CI pipeline
• affects the public site
• affects the editorial area
• affects the publisher area
• affects the monitoring

Developer Checklist

Developers should review and confirm each of these items before requesting review

• Code meets acceptance criteria from issue
• Unit tests are written and all pass
• User Test Scripts (if required) are written and have been run through
• Project's coding standards are met
  • No deprecated methods are used
  • No magic strings/numbers - all strings are in constants or messages files
  • ES queries are wrapped in a Query object rather than inlined in the code
  • Where possible our common library functions have been used (e.g. dates manipulated via dates)
  • Cleaned up commented out code, etc
  • Urls are constructed with url_for not hard-coded
• Code documentation and related non-code documentation has all been updated
  • Core model documentation has been added to if needed: https://docs.google.com/spreadsheets/d/1lun2S9vwGbyfy3WjIjgXBm05D-3wWDZ4bp8xiIYfImM/edit
  • Events and consumers documentation has been added if needed: https://docs.google.com/spreadsheets/d/1oIeG5vg-blm2MZCE-7YhwulUlSz6TOUeY8jAftdP9JE/edit
• Migation has been created and tested
• There is a recent merge from develop (2024-09-16)

Reviewer Checklist

Reviewers should review and confirm each of these items before approval
If there are multiple reviewers, this section should be duplicated for each reviewer

• Code meets acceptance criteria from issue
• Unit tests are written and all pass
• User Test Scripts (if required) are written and have been run through
• Project's coding standards are met
  • No deprecated methods are used
  • No magic strings/numbers - all strings are in constants or messages files
  • ES queries are wrapped in a Query object rather than inlined in the code
  • Where possible our common library functions have been used (e.g. dates manipulated via dates)
  • Cleaned up commented out code, etc
  • Urls are constructed with url_for not hard-coded
• Code documentation and related non-code documentation has all been updated
  • Core model documentation has been added to if needed: https://docs.google.com/spreadsheets/d/1lun2S9vwGbyfy3WjIjgXBm05D-3wWDZ4bp8xiIYfImM/edit
  • Events and consumers documentation has been added if needed: https://docs.google.com/spreadsheets/d/1oIeG5vg-blm2MZCE-7YhwulUlSz6TOUeY8jAftdP9JE/edit
• Migation has been created and tested
• There is a recent merge from develop

Testing

There is no user script for this to be tested, but we should do the following:

1. Review the OAI feed on test
2. Withdraw a journal with articles
3. Delete an article

Then review the feed and confirm that all affected records appear on the OAI feed.

Deployment

No additional deployment considerations.

[RK206]

Overall the code looks good and working when tested.
One testcase always failing when run with complete tests for test_oaipmh. But it passed when executed alone. Not sure if it is with my environment.

doajtest/unit/test_oaipmh.py:321 (TestClient.test_09_article)
1 != 3

Expected :3
Actual   :1

There is no migration script. I believe ArticleTombstone should be indexed when application restarted. I could not able to test the migration as I did anon_import.