Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add/review Slack messaging for the new data refresh #4917

Open
stacimc opened this issue Sep 11, 2024 · 0 comments
Open

Add/review Slack messaging for the new data refresh #4917

stacimc opened this issue Sep 11, 2024 · 0 comments
Labels
💻 aspect: code Concerns the software code in the repository 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🟩 priority: low Low priority and doesn't need to be rushed 🧱 stack: catalog Related to the catalog and Airflow DAGs

Comments

@stacimc
Copy link
Collaborator

stacimc commented Sep 11, 2024

Problem

The current data refresh, run on the ingestion server, reports somewhat noisily to Slack. The new data refresh will happen almost entirely in Airflow where it is much more observable than on the ingestion server, so it's not as important to have such extensive Slack messaging -- but given that it is such an important and long running DAG, we should still consider adding some messaging at critical steps.

Description

We should look at all the places the current data refresh sends Slack notifications to double check if there are additional messages we need, but I think we should at least have a Slack message when the copy data step finishes and when the distributed reindex finishes.

Additional context

#4833

@stacimc stacimc added 🟩 priority: low Low priority and doesn't need to be rushed 💻 aspect: code Concerns the software code in the repository 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🧱 stack: catalog Related to the catalog and Airflow DAGs labels Sep 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
💻 aspect: code Concerns the software code in the repository 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🟩 priority: low Low priority and doesn't need to be rushed 🧱 stack: catalog Related to the catalog and Airflow DAGs
Projects
Status: 📋 Backlog
Development

No branches or pull requests

1 participant