feat: allow to discard pipeline events when flusher_http's queue is full #1220
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
shunjiazhu
commented
Nov 6, 2023
•
shunjiazhu
commented
- 当前的flusher http的queue满时，会导致flusher的Export函数阻塞住，如果配置的http上报过慢会导致反压，允许用户在这种情况丢弃数据。
- 添加4个自监控指标，用来追踪丢点，重试，和发送延迟。
yyuuttaaoo
reviewed
Nov 6, 2023
| | AsyncIntercept | Boolean | 否 | 异步过滤数据，默认为否 | ||
| | QueueCapacity | Int | 否 | 内部channel的缓存大小，默认为1024 | | ||
| | AsyncIntercept | Boolean | 否 | 异步过滤数据，默认为否 | | ||
| | DropEventWhenQueueFull | Boolean | 否 | 当Channel满时是否丢弃数据，默认为是，防止影响整个流水线 | |
There was a problem hiding this comment.
另外，这个功能是不是应该往上放一层，就是放在flusher的queue那里？这个功能是不是通用的？
There was a problem hiding this comment.
嗯，长期来看确实应该应该提供这样一个统一的功能组件。
liuhaoyang
reviewed
Nov 10, 2023
plugins/flusher/http/flusher_http.go
Outdated
| @@ -139,7 +141,7 @@ func (f *FlusherHTTP) Init(context pipeline.Context) error { | |||
|
|
|||
| f.buildVarKeys() | |||
| f.fillRequestContentType() | |||
|
|
|||
| f.droppedEvents = helper.NewCounterMetricAndRegister("http_flusher_dropped_events", context) | |||
There was a problem hiding this comment.
这个是统计 drop 的 events 还是 group
plugins/flusher/http/flusher_http.go
Outdated
| default: | ||
| f.counter.Done() | ||
| f.droppedEvents.Add(1) | ||
| logger.Warningf(f.context.GetRuntimeContext(), "FLUSHER_FLUSH_ALARM", "http flusher dropped a event due to the queue is full") |
There was a problem hiding this comment.
嗯，改了名字了。不过发现这边的metric和我们的预期差别挺大的，只有最后一个flusher_http的指标被注册了下来。
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.