Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add option to pass external contexts into Go SDK's HTTP calls #1475

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions go/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,3 +118,57 @@ func main() {
}
}
```
### Custom Context

If you want even greater control over the lifecycle of the HTTP requests consider providing a [context](https://pkg.go.dev/context). Contexts can be set for all requests or per request.

If you wish to include timeout functionality in your custom context then you should leverage [context.WithTimeout](https://pkg.go.dev/context#WithTimeout).
Copy link
Collaborator

@jeremytchang jeremytchang Sep 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change to:

The SDK will ignore any timeout described in the previous Timeout section for all requests if you use custom context for all requests, or per request if you use custom context per request.

You must use [context.WithTimeout](https://pkg.go.dev/context#WithTimeout) functionality to set a timeout.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my previous comment - this is inaccurate.


> Note: Setting a context will override any "Timeout" settings that you have applied!

#### Custom Context for all requests

Follow the example code snippet below if you want all requests to use the same context:
JCPistell marked this conversation as resolved.
Show resolved Hide resolved

```go
import "context"

func main() {
// sets a timeout of 5 minutes
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
defer cancel()

cfg, err := rtl.NewSettingsFromFile("path/to/looker.ini", nil)
cfg.Context = ctx

session := rtl.NewAuthSession(cfg)
sdk := v4.NewLookerSDK(session)
}
```

> Note: A context set here will also apply to background requests to fetch/refresh oauth tokens, which are normally separated from contexts set via the Timeout property.

#### Custom Context per request

Follow the example here to set a context for a specific request.
Copy link
Collaborator

@jeremytchang jeremytchang Sep 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also add "For the specific request, the custom context will replace the custom context set for all requests in the previous "Custom Context for all requests" section."


> Note: This will override any "Timeout" settings that you have applied, as well as any context set in the SDK config as outlined in the previus section!

```go
import "context"

func main() {
// sets a timeout of 5 minutes
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
defer cancel()

cfg, err := rtl.NewSettingsFromFile("path/to/looker.ini", nil)
session := rtl.NewAuthSession(cfg)
sdk := v4.NewLookerSDK(session)

sdk.Me("", &ApiSettings{Context: ctx})
}
```

> Note: Setting a context per request will NOT affect the context used for the background token fetching requests. If you have also set a context for all requests as mentioned above then that context
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also add "If you set a custom context for a specific request, the SDK will ignore any timeout described in the previous Timeout section for the specific request. You must set the timeout with the custom context for the specific request."

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't true - request timeouts will still apply - any specified context simply becomes the parent of the timeout request.

Copy link
Collaborator

@jeremytchang jeremytchang Sep 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace with

Note: If you set a custom context for a specific request, the SDK will ignore any timeout described in the previous Timeout section for the specific request. You must set the timeout with the request's custom context.

Note:
The custom context per request does NOT affect the background oauth token requests' context. You must use a custom context set on all requests to do so.

will still be used for the token requests, otherwise the SDK will fall back on using a completely separate context for the token fetching requests.
34 changes: 23 additions & 11 deletions go/rtl/auth.go
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,13 @@ func NewAuthSessionWithTransport(config ApiSettings, transport http.RoundTripper
AuthStyle: oauth2.AuthStyleInParams,
}

bgCtx := context.Background()
JCPistell marked this conversation as resolved.
Show resolved Hide resolved
if config.Context != nil {
bgCtx = config.Context
}

ctx := context.WithValue(
context.Background(),
bgCtx,
oauth2.HTTPClient,
// Will set "x-looker-appid" Header on TokenURL requests
&http.Client{Transport: appIdHeaderTransport},
Expand Down Expand Up @@ -123,18 +128,25 @@ func (s *AuthSession) Do(result interface{}, method, ver, path string, reqPars m
}
}

// create request context with timeout
var timeoutInSeconds int32 = 120 //seconds
if s.Config.Timeout != 0 {
timeoutInSeconds = s.Config.Timeout
}
if options != nil && options.Timeout != 0 {
timeoutInSeconds = options.Timeout
var ctx context.Context
if options != nil && options.Context != nil {
ctx = options.Context
} else if s.Config.Context != nil {
ctx = s.Config.Context
} else {
// create request context with timeout from options or else config or else 120 seconds
var timeoutInSeconds int32 = 120 //seconds
if s.Config.Timeout != 0 {
timeoutInSeconds = s.Config.Timeout
}
if options != nil && options.Timeout != 0 {
timeoutInSeconds = options.Timeout
}
var cncl context.CancelFunc
ctx, cncl = context.WithTimeout(context.Background(), time.Second*time.Duration(timeoutInSeconds))
defer cncl()
}

ctx, cncl := context.WithTimeout(context.Background(), time.Second*time.Duration(timeoutInSeconds))
defer cncl()

// create new request
req, err := http.NewRequestWithContext(ctx, method, u, bytes.NewBufferString(bodyString))
if err != nil {
Expand Down
158 changes: 158 additions & 0 deletions go/rtl/auth_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -635,6 +635,164 @@ func TestAuthSession_Do_Timeout(t *testing.T) {
t.Errorf("Do() call did not error with context.DeadlineExceeded, got=%v", err)
}
})

t.Run("Do() follows Context set in AuthSession config", func(t *testing.T) {
JCPistell marked this conversation as resolved.
Show resolved Hide resolved
mux := http.NewServeMux()
setupApi40Login(mux, foreverValidTestToken, http.StatusOK)
server := httptest.NewServer(mux)
defer server.Close()

mux.HandleFunc("/api"+apiVersion+path, func(w http.ResponseWriter, r *http.Request) {
time.Sleep(4 * time.Second)
})

ctx, cncl := context.WithTimeout(context.Background(), 1*time.Second)
iyabchen marked this conversation as resolved.
Show resolved Hide resolved
defer cncl()

session := NewAuthSession(ApiSettings{
BaseUrl: server.URL,
ApiVersion: apiVersion,
Context: ctx,
})

err := session.Do(nil, "GET", apiVersion, path, nil, nil, nil)

if err == nil {
t.Errorf("Do() call did not error/timeout")
} else if !errors.Is(err, context.DeadlineExceeded) {
t.Errorf("Do() call did not error with context.DeadlineExceeded, got=%v", err)
}
})

t.Run("Do() follows Context set in Do()'s options", func(t *testing.T) {
mux := http.NewServeMux()
setupApi40Login(mux, foreverValidTestToken, http.StatusOK)
server := httptest.NewServer(mux)
defer server.Close()

mux.HandleFunc("/api"+apiVersion+path, func(w http.ResponseWriter, r *http.Request) {
time.Sleep(4 * time.Second)
})

ctx, cncl := context.WithTimeout(context.Background(), 1*time.Second)
defer cncl()

session := NewAuthSession(ApiSettings{
BaseUrl: server.URL,
ApiVersion: apiVersion,
})

options := ApiSettings{
Context: ctx,
}

err := session.Do(nil, "GET", apiVersion, path, nil, nil, &options)

if err == nil {
t.Errorf("Do() call did not error/timeout")
} else if !errors.Is(err, context.DeadlineExceeded) {
t.Errorf("Do() call did not error with context.DeadlineExceeded, got=%v", err)
}
})

t.Run("Timeout set in Do()'s options overrides Authsession", func(t *testing.T) {
mux := http.NewServeMux()
setupApi40Login(mux, foreverValidTestToken, http.StatusOK)
server := httptest.NewServer(mux)
defer server.Close()

mux.HandleFunc("/api"+apiVersion+path, func(w http.ResponseWriter, r *http.Request) {
time.Sleep(4 * time.Second)
})

session := NewAuthSession(ApiSettings{
BaseUrl: server.URL,
ApiVersion: apiVersion,
Timeout: 5,
})

options := ApiSettings{
Timeout: 1, //seconds
}

err := session.Do(nil, "GET", apiVersion, path, nil, nil, &options)

if err == nil {
t.Errorf("Do() call did not error/timeout")
} else if !errors.Is(err, context.DeadlineExceeded) {
t.Errorf("Do() call did not error with context.DeadlineExceeded, got=%v", err)
}
})

t.Run("Context set in AuthSession config overrides Timeouts", func(t *testing.T) {
mux := http.NewServeMux()
setupApi40Login(mux, foreverValidTestToken, http.StatusOK)
server := httptest.NewServer(mux)
defer server.Close()

mux.HandleFunc("/api"+apiVersion+path, func(w http.ResponseWriter, r *http.Request) {
time.Sleep(4 * time.Second)
})

ctx, cncl := context.WithTimeout(context.Background(), 1*time.Second)
defer cncl()

session := NewAuthSession(ApiSettings{
BaseUrl: server.URL,
ApiVersion: apiVersion,
Context: ctx,
Timeout: 5,
})

options := ApiSettings{
Timeout: 5,
}

err := session.Do(nil, "GET", apiVersion, path, nil, nil, &options)

if err == nil {
t.Errorf("Do() call did not error/timeout")
} else if !errors.Is(err, context.DeadlineExceeded) {
t.Errorf("Do() call did not error with context.DeadlineExceeded, got=%v", err)
}
})

t.Run("Context set in options overrides config ctx and all Timeouts", func(t *testing.T) {
mux := http.NewServeMux()
setupApi40Login(mux, foreverValidTestToken, http.StatusOK)
server := httptest.NewServer(mux)
defer server.Close()

mux.HandleFunc("/api"+apiVersion+path, func(w http.ResponseWriter, r *http.Request) {
time.Sleep(4 * time.Second)
})

octx, ocncl := context.WithTimeout(context.Background(), 1*time.Second)
defer ocncl()

sctx, scncl := context.WithTimeout(context.Background(), 5*time.Second)
defer scncl()

session := NewAuthSession(ApiSettings{
BaseUrl: server.URL,
ApiVersion: apiVersion,
Context: sctx,
Timeout: 5,
})

options := ApiSettings{
Timeout: 5,
Context: octx,
}

err := session.Do(nil, "GET", apiVersion, path, nil, nil, &options)

if err == nil {
t.Errorf("Do() call did not error/timeout")
} else if !errors.Is(err, context.DeadlineExceeded) {
t.Errorf("Do() call did not error with context.DeadlineExceeded, got=%v", err)
}
})
}

func TestSetQuery(t *testing.T) {
Expand Down
5 changes: 4 additions & 1 deletion go/rtl/settings.go
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
package rtl

import (
"context"
"fmt"
"gopkg.in/ini.v1"
"os"
"strconv"
"strings"

"gopkg.in/ini.v1"
)

var defaultSectionName string = "Looker"
Expand All @@ -20,6 +22,7 @@ type ApiSettings struct {
ClientSecret string `ini:"client_secret"`
ApiVersion string `ini:"api_version"`
Headers map[string]string
Context context.Context
}

var defaultSettings ApiSettings = ApiSettings{
Expand Down
Loading