Describing Kafka consumer behaviour when consuming from multiple topics

[KrylixZA]

Thank you for helping make the Dapr documentation better!

Please follow this checklist before submitting:

• Commits are signed with Developer Certificate of Origin (DCO - learn more)
• Read the contribution guide
• Commands include options for Linux, MacOS, and Windows within codetabs
• New file and folder names are globally unique
• Page references use shortcodes instead of markdown or URL links
• Images use HTML style and have alternative text
• Places where multiple code/command options are given have codetabs

In addition, please fill out the following to help reviewers understand this pull request:

Description

This pull request documents the behaviour of Dapr consumer groups when consuming from multiple topics in Kafka. This updates the documentation to help users understand consumer group behaviour when subscribing to multiple topics to avoid the pitfalls of unevenly balanced consumer groups across topics.

Issue reference

Closes #3707.

[KrylixZA]

In light of the changes made in Dapr 1.12.3, this may not be necessary anymore. We should confirm the behaviour of a single pub/sub component being used to subscribe to multiple topics with Dapr 1.12.3 before either merging or abandoning this pull request.

[KrylixZA]

Looking at the code changes in the fix Kafka consumer pull request on the components-contrib repo, my gut feel says the documentation herein is still relevant. Will confirm next week 👍

[msfussell]

Thanks for this contribution. Yes this should remain true even after the v1.12.3 release, but best if you can confirm anyway.

[KrylixZA]

Thanks for this contribution. Yes this should remain true even after the v1.12.3 release, but best if you can confirm anyway.

Hey Mark. Confirmed the updated documentation in this PR is still relevant in 1.12.3. Teams still need to configure multiple consumer groups (pub/sub component definitions) in order to effectively consume all partitions with all running pods 👍

This PR can merge when you are happy with it.

[msfussell]

Approved. Appreciate the great contribution to the docs

[hhunter-ms]

quick review

[KrylixZA]

Hey @msfussell / @hhunter-ms. Just an FYI: I am not authorised to merge PRs. When you are all happy and ready for this to merge, please would you mind completing the PR?

Ty :)