-
Notifications
You must be signed in to change notification settings - Fork 487
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add otel.receiver.aws_firehose component
#6005
Open
Obito1903
wants to merge
24
commits into
grafana:main
Choose a base branch
from
Obito1903:otel-awsfirehose
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 18 commits
Commits
Show all changes
24 commits
Select commit
Hold shift + click to select a range
1732e07
Add `otel.receiver.awsfirehose` component
Obito1903 8b6deaa
Update component/otelcol/receiver/awsfirehose/awsfirehose.go
Obito1903 dfb3482
Update component/otelcol/receiver/awsfirehose/awsfirehose.go
Obito1903 afc64f7
Update according pr suggestions
Obito1903 2b1f5d9
Update docs/sources/flow/reference/components/otelcol.receiver.awsfir…
Obito1903 d217b03
Update docs/sources/flow/reference/components/otelcol.receiver.awsfir…
Obito1903 ee32667
Update docs/sources/flow/reference/components/otelcol.receiver.awsfir…
Obito1903 3158090
Update docs/sources/flow/reference/components/otelcol.receiver.awsfir…
Obito1903 aaa0b62
Update docs/sources/flow/reference/components/otelcol.receiver.awsfir…
Obito1903 69bc356
Update docs/sources/flow/reference/components/otelcol.receiver.awsfir…
Obito1903 8262acb
Update docs/sources/flow/reference/components/otelcol.receiver.awsfir…
Obito1903 d50287e
Update docs/sources/flow/reference/components/otelcol.receiver.awsfir…
Obito1903 041d825
Update docs/sources/flow/reference/components/otelcol.receiver.awsfir…
Obito1903 38a1866
Update docs/sources/flow/reference/components/otelcol.receiver.awsfir…
Obito1903 4279489
Tidy up, regenerate go.sum, docs
tpaschalis 94d4533
Fix test
tpaschalis b79ad9d
Add changelog entry
tpaschalis 4fbbae8
Merge branch 'main' into otel-awsfirehose
tpaschalis 486b376
Fix correct docs
tpaschalis 6b061ee
Fix debug metrics docs
tpaschalis 31a2b6f
Remove unused example outputs
tpaschalis d6181f1
Update docs/sources/flow/reference/components/otelcol.receiver.aws_fi…
tpaschalis 74c3a09
Fix docs, set default high cardinality labels
tpaschalis ac76d87
Merge branch 'main' into otel-awsfirehose
tpaschalis File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,88 @@ | ||
| // Package awsfirehose provides an otelcol.receiver.aws_firehose component. | ||
| package awsfirehose | ||
|
|
||
| import ( | ||
| "github.com/grafana/agent/component" | ||
| "github.com/grafana/agent/component/otelcol" | ||
| "github.com/grafana/agent/component/otelcol/receiver" | ||
| "github.com/grafana/river/rivertypes" | ||
| "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsfirehosereceiver" | ||
| otelcomponent "go.opentelemetry.io/collector/component" | ||
| "go.opentelemetry.io/collector/config/configopaque" | ||
| otelextension "go.opentelemetry.io/collector/extension" | ||
| ) | ||
|
|
||
| func init() { | ||
| component.Register(component.Registration{ | ||
| Name: "otelcol.receiver.aws_firehose", | ||
| Args: Arguments{}, | ||
|
|
||
| Build: func(opts component.Options, args component.Arguments) (component.Component, error) { | ||
| fact := awsfirehosereceiver.NewFactory() | ||
| return receiver.New(opts, fact, args.(Arguments)) | ||
| }, | ||
| }) | ||
| } | ||
|
|
||
| // Arguments configures the otelcol.receiver.awsfirehose component. | ||
| type Arguments struct { | ||
| // The type of record being received from the delivery stream. | ||
| // Each unmarshaler handles a specific type, | ||
| // so the field allows the receiver to use the correct one. | ||
| RecordType string `river:"record_type,attr,optional"` | ||
|
|
||
| // The access key to be checked on each request received. | ||
| AccessKey rivertypes.Secret `river:"access_key,attr,optional"` | ||
|
|
||
| HTTPServer otelcol.HTTPServerArguments `river:",squash"` | ||
|
|
||
| // DebugMetrics configures component internal metrics. Optional. | ||
| DebugMetrics otelcol.DebugMetricsArguments `river:"debug_metrics,block,optional"` | ||
|
|
||
| // Output configures where to send received data. Required. | ||
| Output *otelcol.ConsumerArguments `river:"output,block"` | ||
| } | ||
|
|
||
| var _ receiver.Arguments = Arguments{} | ||
|
|
||
| // DefaultArguments holds default settings for otelcol.receiver.awsfirehose. | ||
| var DefaultArguments = Arguments{ | ||
| RecordType: "cwmetrics", | ||
| HTTPServer: otelcol.HTTPServerArguments{ | ||
| Endpoint: "0.0.0.0:4433", | ||
| }, | ||
| } | ||
|
|
||
| // SetToDefault implements river.Defaulter. | ||
| func (args *Arguments) SetToDefault() { | ||
| *args = DefaultArguments | ||
| } | ||
|
|
||
| // Convert implements receiver.Arguments. | ||
| func (args Arguments) Convert() (otelcomponent.Config, error) { | ||
| return &awsfirehosereceiver.Config{ | ||
| RecordType: args.RecordType, | ||
| AccessKey: configopaque.String(args.AccessKey), | ||
| HTTPServerSettings: *args.HTTPServer.Convert(), | ||
| }, nil | ||
| } | ||
|
|
||
| // Extensions implements receiver.Arguments. | ||
| func (args Arguments) Extensions() map[otelcomponent.ID]otelextension.Extension { | ||
| return nil | ||
| } | ||
|
|
||
| // Exporters implements receiver.Arguments. | ||
| func (args Arguments) Exporters() map[otelcomponent.DataType]map[otelcomponent.ID]otelcomponent.Component { | ||
| return nil | ||
| } | ||
|
|
||
| // NextConsumers implements receiver.Arguments. | ||
| func (args Arguments) NextConsumers() *otelcol.ConsumerArguments { | ||
| return args.Output | ||
| } | ||
|
|
||
| // DebugMetricsConfig implements receiver.Arguments. | ||
| func (args Arguments) DebugMetricsConfig() otelcol.DebugMetricsArguments { | ||
| return args.DebugMetrics | ||
| } |
86 changes: 86 additions & 0 deletions
86
component/otelcol/receiver/awsfirehose/awsfirehose_test.go
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,86 @@ | ||
| package awsfirehose_test | ||
|
|
||
| import ( | ||
| "fmt" | ||
| "testing" | ||
| "time" | ||
|
|
||
| "github.com/grafana/agent/component/otelcol/receiver/awsfirehose" | ||
| "github.com/grafana/agent/pkg/flow/componenttest" | ||
| "github.com/grafana/agent/pkg/util" | ||
| "github.com/grafana/river" | ||
| "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsfirehosereceiver" | ||
| "github.com/phayes/freeport" | ||
| "github.com/stretchr/testify/require" | ||
| ) | ||
|
|
||
| func TestRun(t *testing.T) { | ||
| httpAddr := getFreeAddr(t) | ||
|
|
||
| ctx := componenttest.TestContext(t) | ||
| l := util.TestLogger(t) | ||
|
|
||
| ctrl, err := componenttest.NewControllerFromID(l, "otelcol.receiver.aws_firehose") | ||
| require.NoError(t, err) | ||
|
|
||
| cfg := fmt.Sprintf(` | ||
| endpoint = "%s" | ||
|
|
||
| output { /* no-op */ } | ||
| `, httpAddr) | ||
|
|
||
| var args awsfirehose.Arguments | ||
| require.NoError(t, river.Unmarshal([]byte(cfg), &args)) | ||
|
|
||
| go func() { | ||
| err := ctrl.Run(ctx, args) | ||
| require.NoError(t, err) | ||
| }() | ||
|
|
||
| require.NoError(t, ctrl.WaitRunning(time.Second)) | ||
| } | ||
|
|
||
| func TestArguments_UnmarshalRiver(t *testing.T) { | ||
| t.Run("grpc", func(t *testing.T) { | ||
| httpAddr := getFreeAddr(t) | ||
| in := fmt.Sprintf(` | ||
| endpoint = "%s" | ||
| cors { | ||
| allowed_origins = ["https://*.test.com", "https://test.com"] | ||
| } | ||
|
|
||
| record_type = "cwmetrics" | ||
|
|
||
| debug_metrics { | ||
| disable_high_cardinality_metrics = true | ||
| } | ||
|
|
||
| output { /* no-op */ } | ||
| `, httpAddr) | ||
|
|
||
| var args awsfirehose.Arguments | ||
| require.NoError(t, river.Unmarshal([]byte(in), &args)) | ||
| require.Equal(t, args.DebugMetricsConfig().DisableHighCardinalityMetrics, true) | ||
| ext, err := args.Convert() | ||
| require.NoError(t, err) | ||
| otelArgs, ok := (ext).(*awsfirehosereceiver.Config) | ||
|
|
||
| require.True(t, ok) | ||
|
|
||
| // Check the arguments | ||
| require.Equal(t, otelArgs.HTTPServerSettings.Endpoint, httpAddr) | ||
| require.Equal(t, len(otelArgs.HTTPServerSettings.CORS.AllowedOrigins), 2) | ||
| require.Equal(t, otelArgs.HTTPServerSettings.CORS.AllowedOrigins[0], "https://*.test.com") | ||
| require.Equal(t, otelArgs.HTTPServerSettings.CORS.AllowedOrigins[1], "https://test.com") | ||
| require.Equal(t, otelArgs.RecordType, "cwmetrics") | ||
| }) | ||
| } | ||
|
|
||
| func getFreeAddr(t *testing.T) string { | ||
| t.Helper() | ||
|
|
||
| portNumber, err := freeport.GetFreePort() | ||
| require.NoError(t, err) | ||
|
|
||
| return fmt.Sprintf("localhost:%d", portNumber) | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why does this PR change the
loki.sourcecomponent docs? It's probably by accident?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I accidentally did this as I was trying to make the check for compatible components pass, I'll revert.