[ADAPT1 - 1543] | Internal Consumer Endpoint - Enhance to Check Lag | Remove Partition With No Data | Send Notification Having Lag Percentage | Unit Test

[abhivermaaa]

[ADAPT1 - 1543] | Internal Consumer Endpoint - Enhance to Check Lag | Remove Partition With No Data | Send Notification Having Lag Percentage | Unit Test

Description :
We need to calculate lag on the topic “_hydra.consumer-groups” . We don’t commit offset for this internal topic. So, In order to calculate lag on this topic, we need to have committed offsets for which we have created a new stream named as “offset stream”.

With the offset stream, we created an consumer group offset facade which should hold all partition level offsets. We created a new endpoint : “v2/consumer-groups/hydra-internal-topic” for returning the partition level offset data for “_hydra.consumer-groups” which will take offsets from consumer group storage facade.

This PR contains code changes for :

1. removing partition data from response which are not having any records
2. calculating the partition level lag for all the partitions which are having records
3. calculating the total lag by consolidating all partition’s data
4. sending the notification to the provided channel in “HYDRA_SYSTEM_NOTIFICATION_SLACK_CHANNEL“ having lag percentage
5. adding all the unit test scenarios for this new endpoint

Jira Link :
https://pluralsight.atlassian.net/browse/ADAPT1-1543