Skip to content

Commit

Permalink
feat: add ko test command to build with go test -c
Browse files Browse the repository at this point in the history
Add a new command to build images with the output of go test -c.

It allows configuring build and LD flags in .ko.yaml with different
values for build and test commands for a given importpath.
  • Loading branch information
vincent-d committed Sep 25, 2024
1 parent 55e7099 commit e14c6c5
Show file tree
Hide file tree
Showing 12 changed files with 304 additions and 5 deletions.
4 changes: 4 additions & 0 deletions .github/workflows/e2e.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -89,3 +89,7 @@ jobs:
# Check that --debug adds dlv to the image, and that dlv is runnable.
docker run --entrypoint="dlv" $(go run ./ build ./test/ --platform=${PLATFORM} --debug) version | grep "Delve Debugger"
fi
# Build and run tests in the test/ folder
testimg=$(go run ./ test ./test/test/ --platform=${PLATFORM})
docker run ${testimg} | grep "PASS"
1 change: 1 addition & 0 deletions docs/reference/ko.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,6 @@ ko [flags]
* [ko login](ko_login.md) - Log in to a registry
* [ko resolve](ko_resolve.md) - Print the input files with image references resolved to built/pushed image digests.
* [ko run](ko_run.md) - A variant of `kubectl run` that containerizes IMPORTPATH first.
* [ko test](ko_test.md) - Build and publish container images with go test from the given importpaths.
* [ko version](ko_version.md) - Print ko version.

75 changes: 75 additions & 0 deletions docs/reference/ko_test.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
## ko test

Build and publish container images with go test from the given importpaths.

### Synopsis

This sub-command builds the provided import paths into Go test binaries, containerizes them, and publishes them.

```
ko test IMPORTPATH... [flags]
```

### Examples

```
# Build and publish tests from import path references to a Docker Registry as:
# ${KO_DOCKER_REPO}/<package name>-<hash of import path>
# When KO_DOCKER_REPO is ko.local, it is the same as if --local and
# --preserve-import-paths were passed.
# If the import path is not provided, the current working directory is the
# default.
ko test github.com/foo/bar/cmd/baz github.com/foo/bar/cmd/blah
# Build and publish tests from a relative import path as:
# ${KO_DOCKER_REPO}/<package name>-<hash of import path>
# When KO_DOCKER_REPO is ko.local, it is the same as if --local and
# --preserve-import-paths were passed.
ko test ./cmd/blah
# Build and publish tests from a relative import path as:
# ${KO_DOCKER_REPO}/<import path>
# When KO_DOCKER_REPO is ko.local, it is the same as if --local was passed.
ko test --preserve-import-paths ./cmd/blah
# Build and publish tests from import path references to a Docker daemon as:
# ko.local/<import path>
# This always preserves import paths.
ko test --local github.com/foo/bar/cmd/baz github.com/foo/bar/cmd/blah
```

### Options

```
--bare Whether to just use KO_DOCKER_REPO without additional context (may not work properly with --tags).
-B, --base-import-paths Whether to use the base path without MD5 hash after KO_DOCKER_REPO (may not work properly with --tags).
--debug Include Delve debugger into image and wrap around ko-app. This debugger will listen to port 40000.
--disable-optimizations Disable optimizations when building Go code. Useful when you want to interactively debug the created container.
-h, --help help for test
--image-label strings Which labels (key=value) to add to the image.
--image-refs string Path to file where a list of the published image references will be written.
--insecure-registry Whether to skip TLS verification on the registry
-j, --jobs int The maximum number of concurrent builds (default GOMAXPROCS)
-L, --local Load into images to local docker daemon.
--oci-layout-path string Path to save the OCI image layout of the built images
--platform strings Which platform to use when pulling a multi-platform base. Format: all | <os>[/<arch>[/<variant>]][,platform]*
-P, --preserve-import-paths Whether to preserve the full import path after KO_DOCKER_REPO.
--push Push images to KO_DOCKER_REPO (default true)
--sbom string The SBOM media type to use (none will disable SBOM synthesis and upload). (default "spdx")
--sbom-dir string Path to file where the SBOM will be written.
--tag-only Include tags but not digests in resolved image references. Useful when digests are not preserved when images are repopulated.
-t, --tags strings Which tags to use for the produced image instead of the default 'latest' tag (may not work properly with --base-import-paths or --bare). (default [latest])
--tarball string File to save images tarballs
```

### Options inherited from parent commands

```
-v, --verbose Enable debug logs
```

### SEE ALSO

* [ko](ko.md) - Rapidly iterate with Go, Containers, and Kubernetes.

4 changes: 4 additions & 0 deletions pkg/build/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,10 @@ type Config struct {
Ldflags StringArray `yaml:",omitempty"`
Flags FlagArray `yaml:",omitempty"`

// TestLdflags and TestFlags will be used for the Go test command line arguments
TestLdflags StringArray `yaml:",omitempty"`
TestFlags FlagArray `yaml:",omitempty"`

// Env allows setting environment variables for `go build`
Env []string `yaml:",omitempty"`

Expand Down
47 changes: 42 additions & 5 deletions pkg/build/gobuild.go
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ type buildContext struct {
flags []string
ldflags []string
platform v1.Platform
goTest bool
}

type builder func(context.Context, buildContext) (string, error)
Expand Down Expand Up @@ -102,6 +103,7 @@ type gobuild struct {
dir string
labels map[string]string
debug bool
goTest bool
semaphore *semaphore.Weighted

cache *layerCache
Expand Down Expand Up @@ -129,6 +131,7 @@ type gobuildOpener struct {
dir string
jobs int
debug bool
goTest bool
}

func (gbo *gobuildOpener) Open() (Interface, error) {
Expand Down Expand Up @@ -159,6 +162,7 @@ func (gbo *gobuildOpener) Open() (Interface, error) {
labels: gbo.labels,
dir: gbo.dir,
debug: gbo.debug,
goTest: gbo.goTest,
platformMatcher: matcher,
cache: &layerCache{
buildToDiff: map[string]buildIDToDiffID{},
Expand Down Expand Up @@ -237,10 +241,16 @@ func (g *gobuild) IsSupportedReference(s string) error {
if dir == "." {
dir = ""
}
pkgs, err := packages.Load(&packages.Config{Dir: dir, Mode: packages.NeedName}, ref.Path())
pkgs, err := packages.Load(&packages.Config{Dir: dir, Mode: packages.NeedName, Tests: g.goTest}, ref.Path())
if err != nil {
return fmt.Errorf("error loading package from %s: %w", ref.Path(), err)
}
if g.goTest {
if len(pkgs) == 0 {
return errors.New("no package found in importpath")
}
return nil
}
if len(pkgs) != 1 {
return fmt.Errorf("found %d local packages, expected 1", len(pkgs))
}
Expand Down Expand Up @@ -356,8 +366,12 @@ func build(ctx context.Context, buildCtx buildContext) (string, error) {
return "", err
}

args := make([]string, 0, 4+len(buildArgs))
args = append(args, "build")
args := make([]string, 0, 5+len(buildArgs))
if buildCtx.goTest {
args = append(args, "test", "-c")
} else {
args = append(args, "build")
}
args = append(args, buildArgs...)
tmpDir := ""

Expand Down Expand Up @@ -659,10 +673,18 @@ func (g *gobuild) kodataPath(ref reference) (string, error) {
if dir == "." {
dir = ""
}
pkgs, err := packages.Load(&packages.Config{Dir: dir, Mode: packages.NeedFiles}, ref.Path())
pkgs, err := packages.Load(&packages.Config{Dir: dir, Mode: packages.NeedFiles, Tests: g.goTest}, ref.Path())
if err != nil {
return "", fmt.Errorf("error loading package from %s: %w", ref.Path(), err)
}
if g.goTest {
for i, p := range pkgs {
if len(p.GoFiles) != 0 {
return filepath.Join(filepath.Dir(pkgs[i].GoFiles[0]), "kodata"), nil
}
}
return "", fmt.Errorf("package loaded from %s contains no Go files", ref.path)
}
if len(pkgs) != 1 {
return "", fmt.Errorf("found %d local packages, expected 1", len(pkgs))
}
Expand Down Expand Up @@ -990,13 +1012,19 @@ func (g *gobuild) buildOne(ctx context.Context, refStr string, base v1.Image, pl

// Get the build flags.
flags := config.Flags
if g.goTest {
flags = config.TestFlags
}
if len(flags) == 0 {
// Use the default, if any.
flags = g.defaultFlags
}

// Get the build ldflags.
ldflags := config.Ldflags
if g.goTest {
ldflags = config.TestLdflags
}
if len(ldflags) == 0 {
// Use the default, if any
ldflags = g.defaultLdflags
Expand All @@ -1011,6 +1039,7 @@ func (g *gobuild) buildOne(ctx context.Context, refStr string, base v1.Image, pl
flags: flags,
ldflags: ldflags,
platform: *platform,
goTest: g.goTest,
})
if err != nil {
return nil, fmt.Errorf("build: %w", err)
Expand Down Expand Up @@ -1045,6 +1074,9 @@ func (g *gobuild) buildOne(ctx context.Context, refStr string, base v1.Image, pl

appDir := "/ko-app"
appFileName := appFilename(ref.Path())
if g.goTest {
appFileName += ".test"
}
appPath := path.Join(appDir, appFileName)

var lo layerOptions
Expand All @@ -1069,14 +1101,19 @@ func (g *gobuild) buildOne(ctx context.Context, refStr string, base v1.Image, pl
return nil, fmt.Errorf("cache.get(%q): %w", file, err)
}

comment := "go build output, at " + appPath
if g.goTest {
comment = "go test -c output, at " + appPath
}

layers = append(layers, mutate.Addendum{
Layer: binaryLayer,
MediaType: layerMediaType,
History: v1.History{
Author: "ko",
Created: g.creationTime,
CreatedBy: "ko build " + ref.String(),
Comment: "go build output, at " + appPath,
Comment: comment,
},
})

Expand Down
63 changes: 63 additions & 0 deletions pkg/build/gobuild_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1509,3 +1509,66 @@ func TestDebugger(t *testing.T) {
}
}
}

func TestGoTest(t *testing.T) {
base, err := random.Image(1024, 3)
if err != nil {
t.Fatalf("random.Image() = %v", err)
}

ng, err := NewGo(
context.Background(),
"",
WithBaseImages(func(context.Context, string) (name.Reference, Result, error) { return baseRef, base, nil }),
WithPlatforms("linux/amd64"),
WithGoTest(),
)
if err != nil {
t.Fatalf("NewGo() = %v", err)
}

for _, c := range []struct {
importpath string
expectedName string
}{
{
importpath: "github.com/google/ko/test/test/",
expectedName: "test.test",
}, {
importpath: "github.com/google/ko/pkg/build",
expectedName: "build.test",
},
} {
t.Run(c.importpath, func(t *testing.T) {
result, err := ng.Build(context.Background(), StrictScheme+c.importpath)
if err != nil {
t.Fatalf("Build() = %v", err)
}

img, ok := result.(v1.Image)
if !ok {
t.Fatalf("Build() not an Image: %T", result)
}

// Check that the entrypoint of the image is not overwritten
cfg, err := img.ConfigFile()
if err != nil {
t.Errorf("ConfigFile() = %v", err)
}
gotEntrypoint := cfg.Config.Entrypoint
wantEntrypoint := []string{
"/ko-app/" + c.expectedName,
}

if got, want := len(gotEntrypoint), len(wantEntrypoint); got != want {
t.Fatalf("len(entrypoint) = %v, want %v", got, want)
}

for i := range wantEntrypoint {
if got, want := gotEntrypoint[i], wantEntrypoint[i]; got != want {
t.Errorf("entrypoint[%d] = %v, want %v", i, got, want)
}
}
})
}
}
7 changes: 7 additions & 0 deletions pkg/build/options.go
Original file line number Diff line number Diff line change
Expand Up @@ -191,3 +191,10 @@ func WithDebugger() Option {
return nil
}
}

func WithGoTest() Option {
return func(gbo *gobuildOpener) error {
gbo.goTest = true
return nil
}
}
1 change: 1 addition & 0 deletions pkg/commands/commands.go
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ func AddKubeCommands(topLevel *cobra.Command) {
addResolve(topLevel)
addBuild(topLevel)
addRun(topLevel)
addTest(topLevel)
}

// check if kubectl is installed
Expand Down
1 change: 1 addition & 0 deletions pkg/commands/options/build.go
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ type BuildOptions struct {
Platforms []string
Labels []string
Debug bool
GoTest bool
// UserAgent enables overriding the default value of the `User-Agent` HTTP
// request header used when retrieving the base image.
UserAgent string
Expand Down
3 changes: 3 additions & 0 deletions pkg/commands/resolver.go
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,9 @@ func gobuildOptions(bo *options.BuildOptions) ([]build.Option, error) {
opts = append(opts, build.WithDebugger())
opts = append(opts, build.WithDisabledOptimizations()) // also needed for Delve
}
if bo.GoTest {
opts = append(opts, build.WithGoTest())
}
switch bo.SBOM {
case "none":
opts = append(opts, build.WithDisabledSBOM())
Expand Down
Loading

0 comments on commit e14c6c5

Please sign in to comment.