gci-go

Modern cloud web services developed in Go execute on top of a runtime environment. On the one hand, Go runtime provide several off-the-shelf benefits like code security and cross-platform execution. On the other side, runtime's internal routines such as automatic memory management add a non-deterministic overhead to the overall service time, increasing the tail of the service time distribution. In this context, it is well known that the Garbage Collector is among the leading causes of high tail latency.

To tackle this problem, we have developed the Garbage Collector Control Interceptor (GCI) -- a request interceptor agnostic regarding the cloud service the load it is subjected load to. GCI helps to improve the service time of cloud services by controlling GC interventions and using simple load shedding mechanisms to signal load balancers or other clients, preventing serving requests during these interventions.

Performance

• Message Push benchmark description and results can be found here.
• GCI-go on the cloud:
  • 2 instances.

Installing GCI

go get -u github.com/gcinterceptor/gci-go/...

Using GCI

Let's say you you're building your cloud service using the Go's net/http package. To start using GCI simply wrap your service endpoint with httphandler.GCI. For example, imagine your have a variable hello, which points to your endpoint http.HandlerFunc:

http.Handle("/", httphandler.GCI(hello))

A complete example here.

Would to have GCI on your favourite framework? Please send us a PR or open an issue.

Academic articles related to GCI

2017

Using Load Shedding to Fight Tail-Latency on Runtime-Based Services. Fireman, D.; Lopes, R; Brunet, J. XXIX Simpósio Brasileiro de Redes de Computadores e Sistemas Distribuídos (SBRC).

Blog posts related to Go runtime memory management/GC

• Golang’s Real-time GC in Theory and Practice
• Go GC: Prioritizing low latency and simplicity
• Go’s march to low-latency GC
• Modern garbage collection: a look at the Go GC strategy