Skip to content

Latest commit

 

History

History
654 lines (485 loc) · 32.8 KB

fluentbit.md

File metadata and controls

654 lines (485 loc) · 32.8 KB

API Docs

This Document documents the types introduced by the fluentbit Operator.

Note this document is generated from code comments. When contributing a change to this document please do so by changing the code comments.

Table of Contents

ClusterFilter

ClusterFilter defines a cluster-level Filter configuration.

Field Description Scheme
metadata metav1.ObjectMeta
spec Specification of desired Filter configuration. FilterSpec

Back to TOC

ClusterFilterList

ClusterFilterList contains a list of ClusterFilter

Field Description Scheme
metadata metav1.ListMeta
items []ClusterFilter

Back to TOC

ClusterFluentBitConfig

ClusterFluentBitConfig is the Schema for the cluster-level fluentbitconfigs API

Field Description Scheme
metadata metav1.ObjectMeta
spec FluentBitConfigSpec

Back to TOC

ClusterFluentBitConfigList

ClusterFluentBitConfigList contains a list of ClusterFluentBitConfig

Field Description Scheme
metadata metav1.ListMeta
items []ClusterFluentBitConfig

Back to TOC

ClusterInput

ClusterInput is the Schema for the inputs API

Field Description Scheme
metadata metav1.ObjectMeta
spec InputSpec

Back to TOC

ClusterInputList

ClusterInputList contains a list of ClusterInput

Field Description Scheme
metadata metav1.ListMeta
items []ClusterInput

Back to TOC

ClusterMultilineParser

ClusterMultilineParser is the Schema for the cluster-level multiline parser API

Field Description Scheme
metadata metav1.ObjectMeta
spec MultilineParserSpec

Back to TOC

ClusterMultilineParserList

ClusterMultilineParserList contains a list of ClusterMultilineParser

Field Description Scheme
metadata metav1.ListMeta
items []ClusterMultilineParser

Back to TOC

ClusterOutput

ClusterOutput is the Schema for the cluster-level outputs API

Field Description Scheme
metadata metav1.ObjectMeta
spec OutputSpec

Back to TOC

ClusterOutputList

ClusterOutputList contains a list of ClusterOutput

Field Description Scheme
metadata metav1.ListMeta
items []ClusterOutput

Back to TOC

ClusterParser

ClusterParser is the Schema for the cluster-level parsers API

Field Description Scheme
metadata metav1.ObjectMeta
spec ParserSpec

Back to TOC

ClusterParserList

ClusterParserList contains a list of ClusterParser

Field Description Scheme
metadata metav1.ListMeta
items []ClusterParser

Back to TOC

Collector

Collector is the Schema for the fluentbits API

Field Description Scheme
metadata metav1.ObjectMeta
spec CollectorSpec
status CollectorStatus

Back to TOC

CollectorList

CollectorList contains a list of Collector

Field Description Scheme
metadata metav1.ListMeta
items []Collector

Back to TOC

CollectorService

CollectorService defines the service of the FluentBit

Field Description Scheme
name Name is the name of the FluentBit service. string
annotations Annotations to add to each Fluentbit service. map[string]string
labels Labels to add to each FluentBit service map[string]string

Back to TOC

CollectorSpec

CollectorSpec defines the desired state of FluentBit

Field Description Scheme
image Fluent Bit image. string
args Fluent Bit Watcher command line arguments. []string
imagePullPolicy Fluent Bit image pull policy. corev1.PullPolicy
imagePullSecrets Fluent Bit image pull secret []corev1.LocalObjectReference
resources Compute Resources required by container. corev1.ResourceRequirements
nodeSelector NodeSelector map[string]string
affinity Pod's scheduling constraints. *corev1.Affinity
tolerations Tolerations []corev1.Toleration
fluentBitConfigName Fluentbitconfig object associated with this Fluentbit string
secrets The Secrets are mounted into /fluent-bit/secrets/. []string
runtimeClassName RuntimeClassName represents the container runtime configuration. string
priorityClassName PriorityClassName represents the pod's priority class. string
volumes List of volumes that can be mounted by containers belonging to the pod. []corev1.Volume
volumesMounts Pod volumes to mount into the container's filesystem. []corev1.VolumeMount
annotations Annotations to add to each Fluentbit pod. map[string]string
serviceAccountAnnotations Annotations to add to the Fluentbit service account map[string]string
securityContext SecurityContext holds pod-level security attributes and common container settings. *corev1.PodSecurityContext
hostNetwork Host networking is requested for this pod. Use the host's network namespace. If this option is set, the ports that will be used must be specified. Default to false. bool
pvc PVC definition *corev1.PersistentVolumeClaim
rbacRules RBACRules represents additional rbac rules which will be applied to the fluent-bit clusterrole. []rbacv1.PolicyRule
disableService By default will build the related service according to the globalinputs definition. bool
bufferPath The path where buffer chunks are stored. *string
ports Ports represents the pod's ports. []corev1.ContainerPort
service Service represents configurations on the fluent-bit service. CollectorService
schedulerName SchedulerName represents the desired scheduler for the Fluentbit collector pods string

Back to TOC

Decorder

Field Description Scheme
decodeField If the content can be decoded in a structured message, append that structure message (keys and values) to the original log message. string
decodeFieldAs Any content decoded (unstructured or structured) will be replaced in the same key/value, no extra keys are added. string

Back to TOC

Filter

Filter is the Schema for namespace level filter API

Field Description Scheme
metadata metav1.ObjectMeta
spec FilterSpec

Back to TOC

FilterItem

Field Description Scheme
grep Grep defines Grep Filter configuration. *filter.Grep
recordModifier RecordModifier defines Record Modifier Filter configuration. *filter.RecordModifier
kubernetes Kubernetes defines Kubernetes Filter configuration. *filter.Kubernetes
modify Modify defines Modify Filter configuration. *filter.Modify
nest Nest defines Nest Filter configuration. *filter.Nest
parser Parser defines Parser Filter configuration. *filter.Parser
lua Lua defines Lua Filter configuration. *filter.Lua
throttle Throttle defines a Throttle configuration. *filter.Throttle
rewriteTag RewriteTag defines a RewriteTag configuration. *filter.RewriteTag
aws Aws defines a Aws configuration. *filter.AWS
multiline Multiline defines a Multiline configuration. *filter.Multiline
customPlugin CustomPlugin defines a Custom plugin configuration. *custom.CustomPlugin

Back to TOC

FilterList

FilterList contains a list of Filters

Field Description Scheme
metadata metav1.ListMeta
items []Filter

Back to TOC

FilterSpec

FilterSpec defines the desired state of ClusterFilter

Field Description Scheme
match A pattern to match against the tags of incoming records. It's case-sensitive and support the star (*) character as a wildcard. string
matchRegex A regular expression to match against the tags of incoming records. Use this option if you want to use the full regex syntax. string
logLevel string
filters A set of filter plugins in order. []FilterItem

Back to TOC

FluentBit

FluentBit is the Schema for the fluentbits API

Field Description Scheme
metadata metav1.ObjectMeta
spec FluentBitSpec
status FluentBitStatus

Back to TOC

FluentBitConfig

FluentBitConfig is the Schema for the API

Field Description Scheme
metadata metav1.ObjectMeta
spec NamespacedFluentBitCfgSpec

Back to TOC

FluentBitConfigList

FluentBitConfigList contains a list of Collector

Field Description Scheme
metadata metav1.ListMeta
items []FluentBitConfig

Back to TOC

FluentBitConfigSpec

FluentBitConfigSpec defines the desired state of ClusterFluentBitConfig

Field Description Scheme
service Service defines the global behaviour of the Fluent Bit engine. *Service
inputSelector Select input plugins metav1.LabelSelector
filterSelector Select filter plugins metav1.LabelSelector
outputSelector Select output plugins metav1.LabelSelector
parserSelector Select parser plugins metav1.LabelSelector
multilineParserSelector Select multiline parser plugins metav1.LabelSelector
namespace If namespace is defined, then the configmap and secret for fluent-bit is in this namespace. If it is not defined, it is in the namespace of the fluentd-operator *string
configFileFormat ConfigFileFormat defines the format of the config file, default is "classic", available options are "classic" and "yaml" *string

Back to TOC

FluentBitList

FluentBitList contains a list of FluentBit

Field Description Scheme
metadata metav1.ListMeta
items []FluentBit

Back to TOC

FluentBitService

FluentBitService defines the service of the FluentBit

Field Description Scheme
name Name is the name of the FluentBit service. string
annotations Annotations to add to each Fluentbit service. map[string]string
labels Labels to add to each FluentBit service map[string]string

Back to TOC

FluentBitSpec

FluentBitSpec defines the desired state of FluentBit

Field Description Scheme
disableService DisableService tells if the fluentbit service should be deployed. bool
image Fluent Bit image. string
args Fluent Bit Watcher command line arguments. []string
command Fluent Bit Watcher command. []string
imagePullPolicy Fluent Bit image pull policy. corev1.PullPolicy
imagePullSecrets Fluent Bit image pull secret []corev1.LocalObjectReference
internalMountPropagation MountPropagation option for internal mounts *corev1.MountPropagationMode
positionDB Storage for position db. You will use it if tail input is enabled. corev1.VolumeSource
containerLogRealPath Container log path string
resources Compute Resources required by container. corev1.ResourceRequirements
nodeSelector NodeSelector map[string]string
affinity Pod's scheduling constraints. *corev1.Affinity
tolerations Tolerations []corev1.Toleration
fluentBitConfigName Fluentbitconfig object associated with this Fluentbit string
namespaceFluentBitCfgSelector NamespacedFluentBitCfgSelector selects the namespace FluentBitConfig associated with this FluentBit metav1.LabelSelector
secrets The Secrets are mounted into /fluent-bit/secrets/. []string
runtimeClassName RuntimeClassName represents the container runtime configuration. string
priorityClassName PriorityClassName represents the pod's priority class. string
volumes List of volumes that can be mounted by containers belonging to the pod. []corev1.Volume
volumesMounts Pod volumes to mount into the container's filesystem. []corev1.VolumeMount
disableLogVolumes DisableLogVolumes removes the hostPath mounts for varlibcontainers, varlogs and systemd. bool
annotations Annotations to add to each Fluentbit pod. map[string]string
serviceAccountAnnotations Annotations to add to the Fluentbit service account map[string]string
labels Labels to add to each FluentBit pod map[string]string
securityContext SecurityContext holds pod-level security attributes and common container settings. *corev1.PodSecurityContext
containerSecurityContext ContainerSecurityContext holds container-level security attributes. *corev1.SecurityContext
hostNetwork Host networking is requested for this pod. Use the host's network namespace. If this option is set, the ports that will be used must be specified. Default to false. bool
envVars EnvVars represent environment variables that can be passed to fluentbit pods. []corev1.EnvVar
livenessProbe LivenessProbe represents the pod's liveness probe. *corev1.Probe
readinessProbe ReadinessProbe represents the pod's readiness probe. *corev1.Probe
initContainers InitContainers represents the pod's init containers. []corev1.Container
ports Ports represents the pod's ports. []corev1.ContainerPort
rbacRules RBACRules represents additional rbac rules which will be applied to the fluent-bit clusterrole. []rbacv1.PolicyRule
dnsPolicy Set DNS policy for the pod. Defaults to "ClusterFirst". Valid values are 'ClusterFirstWithHostNet', 'ClusterFirst', 'Default' or 'None'. corev1.DNSPolicy
metricsPort MetricsPort is the port used by the metrics server. If this option is set, HttpPort from ClusterFluentBitConfig needs to match this value. Default is 2020. int32
service Service represents configurations on the fluent-bit service. FluentBitService
schedulerName SchedulerName represents the desired scheduler for fluent-bit pods. string
terminationGracePeriodSeconds Optional duration in seconds the pod needs to terminate gracefully. Value must be non-negative integer. *int64

Back to TOC

InputSpec

InputSpec defines the desired state of ClusterInput

Field Description Scheme
alias A user friendly alias name for this input plugin. Used in metrics for distinction of each configured input. string
logLevel string
dummy Dummy defines Dummy Input configuration. *input.Dummy
tail Tail defines Tail Input configuration. *input.Tail
systemd Systemd defines Systemd Input configuration. *input.Systemd
nodeExporterMetrics NodeExporterMetrics defines Node Exporter Metrics Input configuration. *input.NodeExporterMetrics
prometheusScrapeMetrics PrometheusScrapeMetrics defines Prometheus Scrape Metrics Input configuration. *input.PrometheusScrapeMetrics
fluentBitMetrics FluentBitMetrics defines Fluent Bit Metrics Input configuration. *input.FluentbitMetrics
customPlugin CustomPlugin defines Custom Input configuration. *custom.CustomPlugin
forward Forward defines forward input plugin configuration *input.Forward
openTelemetry OpenTelemetry defines the OpenTelemetry input plugin configuration *input.OpenTelemetry
http HTTP defines the HTTP input plugin configuration *input.HTTP
mqtt MQTT defines the MQTT input plugin configuration *input.MQTT
collectd Collectd defines the Collectd input plugin configuration *input.Collectd
statsd StatsD defines the StatsD input plugin configuration *input.StatsD
nginx Nginx defines the Nginx input plugin configuration *input.Nginx
syslog Syslog defines the Syslog input plugin configuration *input.Syslog
tcp TCP defines the TCP input plugin configuration *input.TCP
kubernetesEvents KubernetesEvents defines the KubernetesEvents input plugin configuration *input.KubernetesEvents
processors Processors defines the processors configuration *plugins.Config

Back to TOC

MultilineParser

MultilineParser is the Schema of namespace-level multiline parser API

Field Description Scheme
metadata metav1.ObjectMeta
spec MultilineParserSpec

Back to TOC

MultilineParserList

MultilineParserList contains a list of MultilineParser

Field Description Scheme
metadata metav1.ListMeta
items []MultilineParser

Back to TOC

NamespacedFluentBitCfgSpec

NamespacedFluentBitCfgSpec defines the desired state of FluentBit

Field Description Scheme
filterSelector Select filter plugins metav1.LabelSelector
outputSelector Select output plugins metav1.LabelSelector
parserSelector Select parser plugins metav1.LabelSelector
service Service defines the global behaviour of the Fluent Bit engine. *Service
clusterParserSelector Select cluster level parser config metav1.LabelSelector
multilineParserSelector Select multiline parser plugins metav1.LabelSelector
clusterMultilineParserSelector Select cluster level multiline parser config metav1.LabelSelector

Back to TOC

Output

Output is the schema for namespace level output API

Field Description Scheme
metadata metav1.ObjectMeta
spec OutputSpec

Back to TOC

OutputList

OutputList contains a list of Outputs

Field Description Scheme
metadata metav1.ListMeta
items []Output

Back to TOC

OutputSpec

OutputSpec defines the desired state of ClusterOutput

Field Description Scheme
match A pattern to match against the tags of incoming records. It's case sensitive and support the star (*) character as a wildcard. string
matchRegex A regular expression to match against the tags of incoming records. Use this option if you want to use the full regex syntax. string
alias A user friendly alias name for this output plugin. Used in metrics for distinction of each configured output. string
logLevel Set the plugin's logging verbosity level. Allowed values are: off, error, warn, info, debug and trace, Defaults to the SERVICE section's Log_Level string
azureBlob AzureBlob defines AzureBlob Output Configuration *output.AzureBlob
azureLogAnalytics AzureLogAnalytics defines AzureLogAnalytics Output Configuration *output.AzureLogAnalytics
cloudWatch CloudWatch defines CloudWatch Output Configuration *output.CloudWatch
retry_limit RetryLimit represents configuration for the scheduler which can be set independently on each output section. This option allows to disable retries or impose a limit to try N times and then discard the data after reaching that limit. string
es Elasticsearch defines Elasticsearch Output configuration. *output.Elasticsearch
file File defines File Output configuration. *output.File
forward Forward defines Forward Output configuration. *output.Forward
http HTTP defines HTTP Output configuration. *output.HTTP
kafka Kafka defines Kafka Output configuration. *output.Kafka
null Null defines Null Output configuration. *output.Null
stdout Stdout defines Stdout Output configuration. *output.Stdout
tcp TCP defines TCP Output configuration. *output.TCP
loki Loki defines Loki Output configuration. *output.Loki
syslog Syslog defines Syslog Output configuration. *output.Syslog
influxDB InfluxDB defines InfluxDB Output configuration. *output.InfluxDB
datadog DataDog defines DataDog Output configuration. *output.DataDog
firehose Firehose defines Firehose Output configuration. *output.Firehose
kinesis Kinesis defines Kinesis Output configuration. *output.Kinesis
stackdriver Stackdriver defines Stackdriver Output Configuration *output.Stackdriver
splunk Splunk defines Splunk Output Configuration *output.Splunk
opensearch OpenSearch defines OpenSearch Output configuration. *output.OpenSearch
opentelemetry OpenTelemetry defines OpenTelemetry Output configuration. *output.OpenTelemetry
prometheusExporter PrometheusExporter_types defines Prometheus exporter configuration to expose metrics from Fluent Bit. *output.PrometheusExporter
prometheusRemoteWrite PrometheusRemoteWrite_types defines Prometheus Remote Write configuration. *output.PrometheusRemoteWrite
s3 S3 defines S3 Output configuration. *output.S3
gelf Gelf defines GELF Output configuration. *output.Gelf
customPlugin CustomPlugin defines Custom Output configuration. *custom.CustomPlugin
processors Processors defines the processors configuration *plugins.Config

Back to TOC

Parser

Parser is the Schema for namespace level parser API

Field Description Scheme
metadata metav1.ObjectMeta
spec ParserSpec

Back to TOC

ParserList

ParserList contains a list of Parsers

Field Description Scheme
metadata metav1.ListMeta
items []Parser

Back to TOC

ParserSpec

ParserSpec defines the desired state of ClusterParser

Field Description Scheme
json JSON defines json parser configuration. *parser.JSON
regex Regex defines regex parser configuration. *parser.Regex
ltsv LTSV defines ltsv parser configuration. *parser.LSTV
logfmt Logfmt defines logfmt parser configuration. *parser.Logfmt
decoders Decoders are a built-in feature available through the Parsers file, each Parser definition can optionally set one or multiple decoders. There are two type of decoders type: Decode_Field and Decode_Field_As. []Decorder

Back to TOC

Script

Field Description Scheme
Name string
Content string

Back to TOC

Service

Field Description Scheme
daemon If true go to background on start *bool
flushSeconds Interval to flush output *int64
graceSeconds Wait time on exit *int64
hcErrorsCount the error count to meet the unhealthy requirement, this is a sum for all output plugins in a defined HC_Period, example for output error: [2022/02/16 10:44:10] [ warn] [engine] failed to flush chunk '1-1645008245.491540684.flb', retry in 7 seconds: task_id=0, input=forward.1 > output=cloudwatch_logs.3 (out_id=3) *int64
hcRetryFailureCount the retry failure count to meet the unhealthy requirement, this is a sum for all output plugins in a defined HC_Period, example for retry failure: [2022/02/16 20:11:36] [ warn] [engine] chunk '1-1645042288.260516436.flb' cannot be retried: task_id=0, input=tcp.3 > output=cloudwatch_logs.1 *int64
hcPeriod The time period by second to count the error and retry failure data point *int64
healthCheck enable Health check feature at http://127.0.0.1:2020/api/v1/health Note: Enabling this will not automatically configure kubernetes to use fluentbit's healthcheck endpoint *bool
httpListen Address to listen string
httpPort Port to listen *int32
httpServer If true enable statistics HTTP server *bool
logFile File to log diagnostic output string
logLevel Diagnostic level (error/warning/info/debug/trace) string
parsersFile Optional 'parsers' config file (can be multiple) string
parsersFiles backward compatible []string
storage Configure a global environment for the storage layer in Service. It is recommended to configure the volume and volumeMount separately for this storage. The hostPath type should be used for that Volume in Fluentbit daemon set. *Storage
emitterName Per-namespace re-emitter configuration string
emitterMemBufLimit string
emitterStorageType string

Back to TOC

Storage

Field Description Scheme
path Select an optional location in the file system to store streams and chunks of data/ string
sync Configure the synchronization mode used to store the data into the file system string
checksum Enable the data integrity check when writing and reading data from the filesystem string
backlogMemLimit This option configure a hint of maximum value of memory to use when processing these records string
maxChunksUp If the input plugin has enabled filesystem storage type, this property sets the maximum number of Chunks that can be up in memory *int64
metrics If http_server option has been enabled in the Service section, this option registers a new endpoint where internal metrics of the storage layer can be consumed string
deleteIrrecoverableChunks When enabled, irrecoverable chunks will be deleted during runtime, and any other irrecoverable chunk located in the configured storage path directory will be deleted when Fluent-Bit starts. string

Back to TOC