support consul/k8s service discovery

[leyou240]

What is the problem your feature solves, or the need it fulfills?

A clear and concise description of why this feature should be added. What is the problem? Who is
this for?
Our microservices are based on k8s, and will be dynamically expanded and reduced during daily operation and maintenance. Currently, static configuration is very inconvenient. We hope to support k8s or consul service discovery.

Describe the solution you'd like

What do you propose to resolve the problem or fulfill the need above? How would you like it to
work?

Describe alternatives you've considered

What other solutions, features, or workarounds have you considered that might also solve the issue?
What are the tradeoffs for these alternatives compared to what you're proposing?

Additional context

This could include references to documentation or papers, prior art, screenshots, or benchmark
results.

[eaufavor]

This will be an add-on business logic on top of the existing framework.

Internally we have systems on top of Pingora working that way.

Do you have a reference to a standard way of doing such thing or are you looking for guidance how to implement such custom logic.

[leyou240]

This will be an add-on business logic on top of the existing framework.

Internally we have systems on top of Pingora working that way.

Do you have a reference to a standard way of doing such thing or are you looking for guidance how to implement such custom logic.

thx, I'm looking for guidance how to implement such custom logic.

[MMADUs]

take a look at this. https://gist.github.com/Object905/6cafd5e8e56dd60670149296411a407f

[Object905]

Since publishing this gist I've updated code to be more self-contained, removed dependency on crossbeam and made it work with kube-rs>0.92.0, because it changed store internals and old version was deadlocking/not discovering right away.
Now this should be copy-pastable.
So, updated the gist too right now.

It works on my production and also achieves zero downtime upgrades of services.
As a bonus included DNS discovery that I made too.

[MMADUs]

Since publishing this gist I've updated code to be more self-contained, removed dependency on crossbeam and made it work with kube-rs>0.92.0, because it changed store internals and old version was deadlocking/not discovering right away. Now this should be copy-pastable. So, updated the gist too right now.

It works on my production and also achieves zero downtime upgrades of services. As a bonus included DNS discovery that I made too.

interesting! some question, does the dns discovery doesnt need the updater background service?

[Object905]

My use case for dns discovery doesn't account for short living dns entries (like coredns in kubernetes), yes.
Setting LoadBalancer.update_frequency should be enough for most use cases when resolving "real" domains, hickory_resolver client does some dns caching inside based on real ttl of entries, so they're not actually re-queried every time when back ends are updated.

And it will be hard to achieve zero downtime with DNS anyway. That may be remedied by retrying when handling upstream errors, but that seems to be a bit flaky anyway.

[leyou240]

@Object905 Thanks, this gist help me a lot!

[pszabop]

interesting approach. I'm working on using the -u upgrade feature to change the config (or the binary), rather than trying to keep a lot of dynamic config around in pingora.

From k8s, docker, or whatever, I generate a config file that pingora reads once, at startup. the config file has the resolved DNS names (for example)

Any drawbacks to this approach?

[simonhammes]

From k8s, docker, or whatever, I generate a config file that pingora reads once, at startup. the config file has the resolved DNS names (for example)

Any drawbacks to this approach?

Are you referring to pod or service IPs? Pod IPs are not stable

[pszabop]

If pingora upgrade is seamless, then unstable Pod IPs are not a problem. Upgrade every few minutes.

But upgrade may not be as seamless as one would hope for (e.g. the HTTP cache doesn't get upgraded. I think, so effectively a flush at every pod IP change). That's why I asked.