Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(otelcolLogs): Split kubelet from systemd logs with a separate feature flag #3841

Closed
wants to merge 11 commits into from
Closed

feat(otelcolLogs): Split kubelet from systemd logs with a separate feature flag #3841

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
8c45f3c
Add kubelet.enabled to split kubelet from system.enabled flag
jagan2221 Aug 7, 2024
f2d0ca2
Update README.md
jagan2221 Aug 19, 2024
16f245a
Update config.yaml
jagan2221 Aug 19, 2024
4cc3c9c
Update values.yaml
jagan2221 Aug 19, 2024
e233e15
Update config.yaml
jagan2221 Sep 12, 2024
f47a793
Update config.yaml
jagan2221 Sep 12, 2024
06135eb
Create 3840.added.txt
jagan2221 Sep 12, 2024
d3d008b
Rename 3840.added.txt to 3841.added.txt
jagan2221 Sep 12, 2024
02dcaa6
Update config.yaml
jagan2221 Sep 13, 2024
961f23c
Update config.yaml
jagan2221 Sep 13, 2024
8b1494a
Pull master branch (#3842)
jagan2221 Sep 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .changelog/3841.added.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
feat(otelcol logs) : Split kubelet from systemd logs with a separate feature flag
1 change: 1 addition & 0 deletions deploy/helm/sumologic/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ The following table lists the configurable parameters of the Sumo Logic chart an
| `sumologic.logs.multiline.additional` | List of additional conditions and expressions to match first line of multiline logs. See [Multiline](https://help.sumologic.com/docs/send-data/kubernetes/collecting-logs/#conditional-multiline-log-parsing) for more information. | `[]` |
| `sumologic.logs.systemd.enabled` | Enable collecting systemd logs from Kubernets nodes. | `true` |
| `sumologic.logs.systemd.units` | List of systemd units to collect logs from. | See [values.yaml] |
| `sumologic.logs.kubelet.enabled` | Enable collecting kubelet logs from Kubernets nodes. | `true` |
| `sumologic.logs.container.keep_time_attribute` | When set to `true`, preserves the `time` attribute, which is a string representation of the `timestamp` attribute. | `false` |
| `sumologic.logs.container.sourceHost` | Set the \_sourceHost metadata field in Sumo Logic. | `""` |
| `sumologic.logs.container.sourceName` | Set the \_sourceName metadata field in Sumo Logic. | `"%{namespace}.%{pod}.%{container}"` |
Expand Down
12 changes: 8 additions & 4 deletions deploy/helm/sumologic/conf/logs/collector/otelcol/config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ processors:
send_batch_size: 1000
timeout: 1s

{{- if .Values.sumologic.logs.systemd.enabled }}
{{- if or .Values.sumologic.logs.systemd.enabled .Values.sumologic.logs.kubelet.enabled }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just curious,

what is the reason to split kubelet logs from systemd?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#3168
This PR has the history @chan-tim-sumo , This was added in OSC-KP Epic and seems dominik was the one previously working on it.
Scenario is like a customer would want only kubelet logs and not other systemd logs, Now, we can easily toggle kubelet and systemd logs with a flag in values yaml.

## copy _SYSTEMD_UNIT, SYSLOG_FACILITY, _HOSTNAME and PRIORITY from body to attributes
## so they can be used by metadata processors same way like for fluentd
## build fluent.tag attribute as `host.{_SYSTEMD_UNIT}`
Expand Down Expand Up @@ -58,7 +58,7 @@ receivers:
{{ tpl (.Files.Get "conf/logs/collector/common/filelog_receiver.yaml") . | nindent 2 }}
{{- end }}

{{- if .Values.sumologic.logs.systemd.enabled }}
{{- if or .Values.sumologic.logs.systemd.enabled .Values.sumologic.logs.kubelet.enabled }}
journald:
directory: /var/log/journal
## This is not a full equivalent of fluent-bit filtering as fluent-bit filters by `_SYSTEMD_UNIT`
Expand All @@ -67,6 +67,7 @@ receivers:
{{- if .Values.sumologic.logs.systemd.units }}
{{ toYaml .Values.sumologic.logs.systemd.units | nindent 6 }}
{{- else }}
{{- if .Values.sumologic.logs.systemd.enabled }}
- addon-config.service
- addon-run.service
- cfn-etcd-environment.service
Expand All @@ -90,7 +91,6 @@ receivers:
- flanneld.service
- format-etcd2-volume.service
- kube-node-taint-and-uncordon.service
- kubelet.service
- ldconfig.service
- locksmithd.service
- logrotate.service
Expand All @@ -117,6 +117,10 @@ receivers:
- var-lib-etcd2.service
{{- end }}
{{- end }}
{{- if .Values.sumologic.logs.kubelet.enabled }}
- kubelet.service
{{- end }}
{{- end }}

service:
extensions:
Expand All @@ -136,7 +140,7 @@ service:
receivers:
- filelog/containers
{{- end }}
{{- if .Values.sumologic.logs.systemd.enabled }}
{{ if or .Values.sumologic.logs.systemd.enabled .Values.sumologic.logs.kubelet.enabled}}
logs/systemd:
exporters:
- otlphttp
Expand Down
24 changes: 13 additions & 11 deletions deploy/helm/sumologic/conf/logs/otelcol/config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ exporters:

{{- end }}

{{ if .Values.sumologic.logs.systemd.enabled }}
{{ if or .Values.sumologic.logs.systemd.enabled .Values.sumologic.logs.kubelet.enabled}}
sumologic/systemd:
endpoint: ${SUMO_ENDPOINT_DEFAULT_LOGS_SOURCE}
log_format: json
Expand Down Expand Up @@ -108,7 +108,7 @@ extensions:
pprof: {}

processors:
{{ if .Values.sumologic.logs.systemd.enabled }}
{{ if or .Values.sumologic.logs.systemd.enabled .Values.sumologic.logs.kubelet.enabled}}
attributes/extract_systemd_source_fields:
actions:
- action: extract
Expand Down Expand Up @@ -143,7 +143,7 @@ processors:
value: kubelet.service
{{ end }}

{{ if .Values.sumologic.logs.systemd.enabled }}
{{ if .Values.sumologic.logs.kubelet.enabled }}
filter/exclude_kubelet_hostname:
logs:
exclude:
Expand All @@ -153,7 +153,7 @@ processors:
value: {{ .Values.sumologic.logs.kubelet.excludeHostRegex | default "$^" | quote }}
{{ end }}

{{ if .Values.sumologic.logs.systemd.enabled }}
{{ if .Values.sumologic.logs.kubelet.enabled }}
filter/exclude_kubelet_priority:
logs:
exclude:
Expand All @@ -163,7 +163,7 @@ processors:
value: {{ .Values.sumologic.logs.kubelet.excludePriorityRegex | default "$^" | quote }}
{{ end }}

{{ if .Values.sumologic.logs.systemd.enabled }}
{{ if .Values.sumologic.logs.kubelet.enabled }}
filter/exclude_kubelet_syslog:
logs:
exclude:
Expand All @@ -173,7 +173,7 @@ processors:
value: {{ .Values.sumologic.logs.kubelet.excludeFacilityRegex | default "$^" | quote }}
{{ end }}

{{ if .Values.sumologic.logs.systemd.enabled }}
{{ if .Values.sumologic.logs.kubelet.enabled }}
filter/exclude_kubelet_unit:
logs:
exclude:
Expand Down Expand Up @@ -233,7 +233,7 @@ processors:
value: .+
{{ end }}

{{ if .Values.sumologic.logs.systemd.enabled }}
{{ if or .Values.sumologic.logs.systemd.enabled .Values.sumologic.logs.kubelet.enabled}}
filter/include_fluent_tag_host:
logs:
include:
Expand All @@ -243,7 +243,7 @@ processors:
value: host\..+
{{ end }}

{{ if .Values.sumologic.logs.systemd.enabled }}
{{ if .Values.sumologic.logs.kubelet.enabled }}
filter/include_kubelet:
logs:
include:
Expand Down Expand Up @@ -273,7 +273,7 @@ processors:
- _collector
{{ end }}

{{ if .Values.sumologic.logs.systemd.enabled }}
{{ if or .Values.sumologic.logs.systemd.enabled .Values.sumologic.logs.kubelet.enabled}}
groupbyattrs/systemd:
keys:
- _sourceName
Expand Down Expand Up @@ -435,7 +435,7 @@ processors:
source_name: {{ .Values.sumologic.logs.container.sourceName | quote }}
{{ end }}

{{ if .Values.sumologic.logs.systemd.enabled }}
{{ if .Values.sumologic.logs.kubelet.enabled }}
source/kubelet:
collector: {{ .Values.sumologic.collectorName | default .Values.sumologic.clusterName | quote }}
source_host: "%{_sourceHost}"
Expand Down Expand Up @@ -557,7 +557,7 @@ service:
- otlp
{{ end }}

{{ if .Values.sumologic.logs.systemd.enabled }}
{{ if .Values.sumologic.logs.kubelet.enabled }}
logs/otlp/kubelet:
exporters:
{{ include "logs.otelcol.exporters" (dict "Values" .Values "Type" "systemd") | nindent 8}}
Expand Down Expand Up @@ -588,7 +588,9 @@ service:
{{- end }}
receivers:
- otlp
{{- end }}

{{ if .Values.sumologic.logs.systemd.enabled}}
logs/otlp/systemd:
exporters:
{{ include "logs.otelcol.exporters" (dict "Values" .Values "Type" "systemd") | nindent 8}}
Expand Down
1 change: 1 addition & 0 deletions deploy/helm/sumologic/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -416,6 +416,7 @@ sumologic:
excludeUnitRegex: ""

kubelet:
enabled: true
otelcol:
## Extra processors for kubelet logs. See https://help.sumologic.com/docs/send-data/kubernetes/collecting-logs/ for details.
extraProcessors: []
Expand Down
Loading