Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

flow: Add otelcol.receiver.vcenter component #5715

Merged
merged 24 commits into from
Nov 16, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
e3baf8e
flow: Add otelcol.receiver.vcenter component
marctc Nov 6, 2023
12d8ccf
flow: Add otelcol.receiver.vcenter component
marctc Nov 6, 2023
ae785aa
Merge branch 'main' into add_vcenter_receiver
marctc Nov 6, 2023
d9fe9f9
Update docs/sources/flow/reference/components/otelcol.receiver.vcente…
marctc Nov 13, 2023
abfb48a
Update docs/sources/flow/reference/components/otelcol.receiver.vcente…
marctc Nov 13, 2023
93fa7c2
Update docs/sources/flow/reference/components/otelcol.receiver.vcente…
marctc Nov 13, 2023
5f2cf69
Update docs/sources/flow/reference/components/otelcol.receiver.vcente…
marctc Nov 13, 2023
c708425
Update CHANGELOG.md
marctc Nov 13, 2023
a346827
Update docs/sources/flow/reference/components/otelcol.receiver.vcente…
marctc Nov 13, 2023
5cfef4c
Update docs/sources/flow/reference/components/otelcol.receiver.vcente…
marctc Nov 13, 2023
84b75ee
Address test feedback
marctc Nov 13, 2023
04be039
Merge branch 'main' into add_vcenter_receiver
marctc Nov 13, 2023
6465d15
clarify docs
marctc Nov 13, 2023
1de1406
fix error message
marctc Nov 15, 2023
a8aceca
Add component to all.go
marctc Nov 16, 2023
a747885
Change vsphere exporter docs
marctc Nov 16, 2023
63385ea
update docs
marctc Nov 16, 2023
80876f9
update docs
marctc Nov 16, 2023
33a62aa
update docs
marctc Nov 16, 2023
cf803c9
Metrics and attributes disabled by default
ptodev Nov 16, 2023
ea66d8f
Enable most metrics and attributes by default./
ptodev Nov 16, 2023
f3ea19e
add stability level
marctc Nov 16, 2023
fa17b65
lint
marctc Nov 16, 2023
65f7233
last tweaks
marctc Nov 16, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ Main (unreleased)

- `otelcol.processor.filter` - filters OTLP telemetry data using OpenTelemetry
Transformation Language (OTTL). (@hainenber)
- `otelcol.receiver.vcenter` - receives telemetry data from vCenter. (@marctc)
marctc marked this conversation as resolved.
Show resolved Hide resolved

### Enhancements

Expand Down
57 changes: 57 additions & 0 deletions component/otelcol/config_scrape.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
package otelcol

import (
"errors"
"fmt"
"time"

scraperhelper "go.opentelemetry.io/collector/receiver/scraperhelper"
)

var (
errNonPositiveInterval = errors.New("requires positive value")
)

// ScraperControllerArguments defines common settings for a scraper controller
// configuration.
type ScraperControllerArguments struct {
CollectionInterval time.Duration `river:"collection_interval,attr,optional"`
InitialDelay time.Duration `river:"initial_delay,attr,optional"`
Timeout time.Duration `river:"timeout,attr,optional"`
}

// DefaultScraperControllerArguments holds default settings for ScraperControllerArguments.
var DefaultScraperControllerArguments = ScraperControllerArguments{
CollectionInterval: time.Minute,
InitialDelay: time.Second,
Timeout: 0 * time.Second,
}

// SetToDefault implements river.Defaulter.
func (args *ScraperControllerArguments) SetToDefault() {
*args = DefaultScraperControllerArguments
}

// Convert converts args into the upstream type.
func (args *ScraperControllerArguments) Convert() *scraperhelper.ScraperControllerSettings {
if args == nil {
return nil
}

return &scraperhelper.ScraperControllerSettings{
CollectionInterval: args.CollectionInterval,
InitialDelay: args.InitialDelay,
Timeout: args.Timeout,
}
}

// Validate returns an error if args is invalid.
func (args *ScraperControllerArguments) Validate() error {
if args.CollectionInterval <= 0 {
return fmt.Errorf(`"collection_interval": %w`, errNonPositiveInterval)
}
if args.Timeout < 0 {
return fmt.Errorf(`"timeout": %w`, errNonPositiveInterval)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like "timeout" can be 0, whereas "collection interval" cannot? In that case the error message for timeout should be a bit different, to say the value should not be negative?

}
return nil
}
104 changes: 104 additions & 0 deletions component/otelcol/receiver/vcenter/vcenter.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
// Package vcenter provides an otelcol.receiver.vcenter component.
package vcenter

import (
"fmt"
"net/url"

"github.com/grafana/agent/component"
"github.com/grafana/agent/component/otelcol"
"github.com/grafana/agent/component/otelcol/receiver"
otel_service "github.com/grafana/agent/service/otel"
"github.com/grafana/river/rivertypes"
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/vcenterreceiver"
otelcomponent "go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/config/configopaque"
otelextension "go.opentelemetry.io/collector/extension"
)

func init() {
component.Register(component.Registration{
Name: "otelcol.receiver.vcenter",
Args: Arguments{},
NeedsServices: []string{otel_service.ServiceName},

Build: func(opts component.Options, args component.Arguments) (component.Component, error) {
fact := vcenterreceiver.NewFactory()
return receiver.New(opts, fact, args.(Arguments))
},
})
}

// Arguments configures the otelcol.receiver.vcenter component.
type Arguments struct {
Endpoint string `river:"endpoint,attr"`
Username string `river:"username,attr"`
Password rivertypes.Secret `river:"password,attr"`

ScraperControllerArguments otelcol.ScraperControllerArguments `river:",squash"`
TLS otelcol.TLSClientArguments `river:"tls,block,optional"`

// DebugMetrics configures component internal metrics. Optional.
DebugMetrics otelcol.DebugMetricsArguments `river:"debug_metrics,block,optional"`

// Output configures where to send received data. Required.
Output *otelcol.ConsumerArguments `river:"output,block"`
}

var _ receiver.Arguments = Arguments{}

var (
// DefaultArguments holds default values for Arguments.
DefaultArguments = Arguments{
ScraperControllerArguments: otelcol.DefaultScraperControllerArguments,
}
ptodev marked this conversation as resolved.
Show resolved Hide resolved
)

// SetToDefault implements river.Defaulter.
func (args *Arguments) SetToDefault() {
*args = DefaultArguments
}

// Convert implements receiver.Arguments.
func (args Arguments) Convert() (otelcomponent.Config, error) {
return &vcenterreceiver.Config{
TLSClientSetting: *args.TLS.Convert(),
ScraperControllerSettings: *args.ScraperControllerArguments.Convert(),
Endpoint: args.Endpoint,
Username: args.Username,
Password: configopaque.String(args.Password),
}, nil
}

// Validate checks to see if the supplied config will work for the receiver
func (args Arguments) Validate() error {
res, err := url.Parse(args.Endpoint)
if err != nil {
return fmt.Errorf("unable to parse url %s: %w", args.Endpoint, err)
}

if res.Scheme != "http" && res.Scheme != "https" {
return fmt.Errorf("url scheme must be http or https")
}
return nil
}

// Extensions implements receiver.Arguments.
func (args Arguments) Extensions() map[otelcomponent.ID]otelextension.Extension {
return nil
}

// Exporters implements receiver.Arguments.
func (args Arguments) Exporters() map[otelcomponent.DataType]map[otelcomponent.ID]otelcomponent.Component {
return nil
}

// NextConsumers implements receiver.Arguments.
func (args Arguments) NextConsumers() *otelcol.ConsumerArguments {
return args.Output
}

// DebugMetricsConfig implements receiver.Arguments.
func (args Arguments) DebugMetricsConfig() otelcol.DebugMetricsArguments {
return args.DebugMetrics
}
82 changes: 82 additions & 0 deletions component/otelcol/receiver/vcenter/vcenter_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
package vcenter

import (
"fmt"
"testing"
"time"

"github.com/grafana/agent/pkg/flow/componenttest"
"github.com/grafana/agent/pkg/util"
"github.com/grafana/river"
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/vcenterreceiver"
"github.com/phayes/freeport"
"github.com/stretchr/testify/require"
)

// Test ensures that otelcol.receiver.vcenter can start successfully.
func Test(t *testing.T) {
httpAddr := getFreeAddr(t)

ctx := componenttest.TestContext(t)
l := util.TestLogger(t)

ctrl, err := componenttest.NewControllerFromID(l, "otelcol.receiver.vcenter")
require.NoError(t, err)

cfg := fmt.Sprintf(`
endpoint = "%s"
username = "user"
password = "pass"

output { /* no-op */ }
`, httpAddr)

var args Arguments
require.NoError(t, river.Unmarshal([]byte(cfg), &args))

go func() {
err := ctrl.Run(ctx, args)
require.NoError(t, err)
}()

require.NoError(t, ctrl.WaitRunning(time.Second))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not quite sure what this test is testing :) Maybe if we have a dummy http server which this receiver can connect to, we could test the receiver's outputs? This kind of test could be useful in the future if we want to reproduce issues. But I wouldn't say it's necessary. If you want to, I'm happy to leave the test as it is.

}

func TestArguments_UnmarshalRiver(t *testing.T) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For unmarshal tests, I personally prefer the table driven approach. It allows you to add lots of tests with minimal overhead.

For example, I'd use the map[string]interface{} style when some config structs are internal to OTel:
https://github.com/grafana/agent/blob/main/component/otelcol/processor/span/span_test.go

If none of the config structs are internal, I'd go for specifying the OTel structs directly:
https://github.com/grafana/agent/blob/main/component/otelcol/connector/servicegraph/servicegraph_test.go

httpAddr := getFreeAddr(t)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that this address is only used for unmarshaling, it's unnecessary to look for a free address.

in := fmt.Sprintf(`
endpoint = "%s"
username = "user"
password = "pass"
collection_interval = "2m"

output { /* no-op */ }
`, httpAddr)

var args Arguments
require.NoError(t, river.Unmarshal([]byte(in), &args))
args.Convert()
ext, err := args.Convert()
require.NoError(t, err)
otelArgs, ok := (ext).(*vcenterreceiver.Config)

require.True(t, ok)

require.Equal(t, "user", otelArgs.Username)
require.Equal(t, "pass", string(otelArgs.Password))
require.Equal(t, httpAddr, otelArgs.Endpoint)

require.Equal(t, 2*time.Minute, otelArgs.ScraperControllerSettings.CollectionInterval)
require.Equal(t, time.Second, otelArgs.ScraperControllerSettings.InitialDelay)
require.Equal(t, 0*time.Second, otelArgs.ScraperControllerSettings.Timeout)

}

func getFreeAddr(t *testing.T) string {
t.Helper()

portNumber, err := freeport.GetFreePort()
require.NoError(t, err)

return fmt.Sprintf("http://localhost:%d", portNumber)
}
129 changes: 129 additions & 0 deletions docs/sources/flow/reference/components/otelcol.receiver.vcenter.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
---
aliases:
- /docs/grafana-cloud/agent/flow/reference/components/otelcol.receiver.vcenter/
- /docs/grafana-cloud/monitor-infrastructure/agent/flow/reference/components/otelcol.receiver.vcenter/
- /docs/grafana-cloud/monitor-infrastructure/integrations/agent/flow/reference/components/otelcol.receiver.vcenter/
canonical: https://grafana.com/docs/agent/latest/flow/reference/components/otelcol.receiver.vcenter/
title: otelcol.receiver.vcenter
marctc marked this conversation as resolved.
Show resolved Hide resolved
description: Learn about otelcol.receiver.vcenter
---

# otelcol.receiver.vcenter

`otelcol.receiver.vcenter` accepts telemetry data from a
vCenter or ESXi host running VMware vSphere APIs and
forwards it to other `otelcol.*` components.
marctc marked this conversation as resolved.
Show resolved Hide resolved

> **NOTE**: `otelcol.receiver.vcenter` is a wrapper over the upstream
> OpenTelemetry Collector `vcenter` receiver from the `otelcol-contrib`
> distribution. Bug reports or feature requests will be redirected to the
> upstream repository, if necessary.

Multiple `otelcol.receiver.vcenter` components can be specified by giving them
different labels.

## Usage

```river
otelcol.receiver.vcenter "LABEL" {
endpoint = "VCENTER_ENDPOINT"
username = "VCENTER_USERNAME"
password = "VCENTER_PASSWORD"

output {
metrics = [...]
}
}
```

## Arguments

`otelcol.receiver.vcenter` supports the following arguments:


Name | Type | Description | Default | Required
---- | ---- | ----------- | ------- | --------
`endpoint` | `string` | Endpoint to the vCenter Server or ESXi host that has the sdk path enabled. | | yes
marctc marked this conversation as resolved.
Show resolved Hide resolved
`username` | `string` | Username to use for authentication. | | yes
`password` | `string` | Password to use for authentication. | | yes
`collection_interval` | `duration` | This receiver collects metrics on an interval. | `"1m"` | no
marctc marked this conversation as resolved.
Show resolved Hide resolved
`initial_delay` | `duration` | Defines how long this receiver waits before starting.. | `"1s"` | no
marctc marked this conversation as resolved.
Show resolved Hide resolved
`timeout` | `duration` | Defines the timeout for the underlying HTTP client. | `"0s"` | no

`endpoint` has the following format: `<protocol>://<hostname>` (e.g. `https://vcsa.hostname.localnet`)
marctc marked this conversation as resolved.
Show resolved Hide resolved

## Blocks
marctc marked this conversation as resolved.
Show resolved Hide resolved

The following blocks are supported inside the definition of
`otelcol.receiver.vcenter`:

Hierarchy | Block | Description | Required
--------- | ----- | ----------- | --------
tls | [tls][] | Configures TLS for the server. | no
marctc marked this conversation as resolved.
Show resolved Hide resolved
debug_metrics | [debug_metrics][] | Configures the metrics that this component generates to monitor its state. | no
output | [output][] | Configures where to send received telemetry data. | yes

[tls]: #tls-block
[debug_metrics]: #debug_metrics-block
[output]: #output-block

### tls block

The `tls` block configures TLS settings used for a server. If the `tls` block
isn't provided, TLS won't be used for connections to the server.

{{< docs/shared lookup="flow/reference/components/otelcol-tls-config-block.md" source="agent" version="<AGENT VERSION>" >}}

### debug_metrics block

{{< docs/shared lookup="flow/reference/components/otelcol-debug-metrics-block.md" source="agent" version="<AGENT VERSION>" >}}

### output block

{{< docs/shared lookup="flow/reference/components/output-block.md" source="agent" version="<AGENT VERSION>" >}}

## Exported fields

`otelcol.receiver.vcenter` does not export any fields.

## Component health

`otelcol.receiver.vcenter` is only reported as unhealthy if given an invalid
configuration.

## Debug information

`otelcol.receiver.vcenter` does not expose any component-specific debug
information.

## Example

This example forwards received telemetry data through a batch processor before
finally sending it to an OTLP-capable endpoint:

```river
otelcol.receiver.vcenter "default" {
endpoint = "http://localhost:15672"
username = "otelu"
password = "password"
marctc marked this conversation as resolved.
Show resolved Hide resolved

output {
metrics = [otelcol.processor.batch.default.input]
logs = [otelcol.processor.batch.default.input]
traces = [otelcol.processor.batch.default.input]
}
}

otelcol.processor.batch "default" {
output {
metrics = [otelcol.exporter.otlp.default.input]
logs = [otelcol.exporter.otlp.default.input]
traces = [otelcol.exporter.otlp.default.input]
}
}

otelcol.exporter.otlp "default" {
client {
endpoint = env("OTLP_ENDPOINT")
}
}
Loading