-
Notifications
You must be signed in to change notification settings - Fork 56
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Publish updated main docs, 1693488820
- Loading branch information
1 parent
cd37e14
commit c640b7f
Showing
309 changed files
with
1,488 additions
and
506 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -81,7 +81,7 @@ | |
| <li>5.1: <a href="#pg-c0708ac9ee8c8fbda19457c73798a929">The Logging custom resource</a></li><li>5.2: <a href="#pg-73731e87b595ac2d32428566811fc127">Configure Fluentd</a></li><li>5.3: <a href="#pg-e0c4f740879ef5ce79b6d0d086ef5e14">Configure syslog-ng</a></li><li>5.4: <a href="#pg-e55dcde4fa90369f23e1167b78e14986">Fluent Bit log collector</a></li><li>5.5: <a href="#pg-d8f4a300f3d0e2b8000fec50bc64befb">Multiple Fluent Bit agents in the cluster</a></li><li>5.6: <a href="#pg-34e5e120deddba15a35b3c3d1135cbb2">TLS encryption</a></li><li>5.7: <a href="#pg-ad05d57db5d08bb5a16b343b608e91fa">Security</a></li><ul> | ||
| </ul></ul><li>6: <a href="#pg-2468a881210107c399941aefc2a95a23">Operation</a></li><ul> | ||
| <li>6.1: <a href="#pg-85dfc8f2bc8d8124c30cdb1fdddb3b6c">Logging operator troubleshooting</a></li><ul> | ||
| <li>6.1.1: <a href="#pg-741171b799e509ff0eabf397d862c757">Troubleshooting Fluent Bit</a></li><li>6.1.2: <a href="#pg-08d978e8b4bd5dff112fd9e079aaba48">Troubleshooting Fluentd</a></li><li>6.1.3: <a href="#pg-3f9ff6b76f6626e8e1fb3f91c1ad7b0f">Troubleshooting syslog-ng</a></li><li>6.1.4: <a href="#pg-ff4769d67e22b03ee37bf3b31cd60c80">Running on KinD</a></li></ul><li>6.2: <a href="#pg-f43e816fa3bb01ec74b11406f7aa77ce">Monitor your logging pipeline with Prometheus Operator</a></li><li>6.3: <a href="#pg-362f849752620b4cf0d070e2fc884699">Alerting</a></li><li>6.4: <a href="#pg-e1b978ea58e795d55d2b6d7ad7aabb23">Readiness probe</a></li><li>6.5: <a href="#pg-791b523329c1485057fb827f9ed4b80b">Collect Fluentd errors</a></li><li>6.6: <a href="#pg-5ac19d15e22096d860ccfc7422b529f9">Optimization</a></li><li>6.7: <a href="#pg-af5ba8326f5b4c35d5dc6de4337c19b0">Scaling</a></li><li>6.8: <a href="#pg-2ea22a017fd9fef2575c36200cd7f9b8">CPU and memory requirements</a></li></ul><li>7: <a href="#pg-b9683b78eca24c9572e4995734221f3f">For developers</a></li><li>8: <a href="#pg-956da29f025cba37fca9fe6b80ab5d48">Frequently asked questions</a></li><li>9: <a href="#pg-0c44f8157757ae9807433efcf75c4e52">License</a></li></ul><div class="content"> | ||
| <li>6.1.1: <a href="#pg-741171b799e509ff0eabf397d862c757">Troubleshooting Fluent Bit</a></li><li>6.1.2: <a href="#pg-08d978e8b4bd5dff112fd9e079aaba48">Troubleshooting Fluentd</a></li><li>6.1.3: <a href="#pg-3f9ff6b76f6626e8e1fb3f91c1ad7b0f">Troubleshooting syslog-ng</a></li><li>6.1.4: <a href="#pg-ff4769d67e22b03ee37bf3b31cd60c80">Running on KinD</a></li></ul><li>6.2: <a href="#pg-f43e816fa3bb01ec74b11406f7aa77ce">Monitor your logging pipeline with Prometheus Operator</a></li><li>6.3: <a href="#pg-362f849752620b4cf0d070e2fc884699">Alerting</a></li><li>6.4: <a href="#pg-e1b978ea58e795d55d2b6d7ad7aabb23">Readiness probe</a></li><li>6.5: <a href="#pg-791b523329c1485057fb827f9ed4b80b">Collect Fluentd errors</a></li><li>6.6: <a href="#pg-5ac19d15e22096d860ccfc7422b529f9">Optimization</a></li><li>6.7: <a href="#pg-af5ba8326f5b4c35d5dc6de4337c19b0">Scaling</a></li><li>6.8: <a href="#pg-2ea22a017fd9fef2575c36200cd7f9b8">CPU and memory requirements</a></li></ul><li>7: <a href="#pg-58416e0a7f68f24b8e7d9b1834b3036f">Images used by Logging operator</a></li><li>8: <a href="#pg-b9683b78eca24c9572e4995734221f3f">For developers</a></li><li>9: <a href="#pg-956da29f025cba37fca9fe6b80ab5d48">Frequently asked questions</a></li><li>10: <a href="#pg-0c44f8157757ae9807433efcf75c4e52">License</a></li></ul><div class="content"> | ||
| <p>Welcome to the Logging operator documentation!</p><h2 id="overview">Overview</h2><p>The Logging operator solves your logging-related problems in Kubernetes environments by automating the deployment and configuration of a Kubernetes logging pipeline.</p><ol> | ||
| <li>The operator deploys and configures a log collector (currently a Fluent Bit DaemonSet) on every node to collect container and application logs from the node file system.</li><li>Fluent Bit queries the Kubernetes API and enriches the logs with metadata about the pods, and transfers both the logs and the metadata to a log forwarder instance.</li><li>The log forwarder instance receives, filters, and transforms the incoming the logs, and transfers them to one or more destination outputs. The Logging operator supports <a href="https://www.fluentd.org/">Fluentd</a> and syslog-ng (via the <a href="https://axoflow.com/docs/axosyslog-core/">AxoSyslog syslog-ng distribution</a>) as log forwarders.</li></ol><p>Your logs are always transferred on authenticated and encrypted channels.</p><p>This operator helps you bundle logging information with your applications: you can describe the behavior of your application in its charts, the Logging operator does the rest.</p><p align="center"><img src="img/logging_operator_flow.png" alt="How Logging operator works"></p><h2 id="feature-highlights">Feature highlights</h2><ul> | ||
| <li>Namespace isolation</li><li>Native Kubernetes label selectors</li><li>Secure communication (TLS)</li><li>Configuration validation</li><li>Multiple flow support (multiply logs for different transformations)</li><li>Multiple <a href="/docs/configuration/plugins/outputs/">output</a> support (store the same logs in multiple storage: S3, GCS, ES, Loki and more…)</li><li>Multiple logging system support (multiple Fluentd, Fluent Bit deployment on the same cluster)</li><li>Support for both syslog-ng and Fluentd as the central log routing component</li></ul><h2 id="architecture">Architecture</h2><p>The Logging operator manages the log collectors and log forwarders of your logging infrastructure, and the routing rules that specify where you want to send your different log messages.</p><p>The <strong>log collectors</strong> are endpoint agents that collect the logs of your Kubernetes nodes and send them to the log forwarders. Logging operator currently uses Fluent Bit as log collector agents.</p><p>The <strong>log forwarder</strong> instance receives, filters, and transforms the incoming logs, and transfers them to one or more destination outputs. The Logging operator supports Fluentd and syslog-ng as log forwarders. Which log forwarder is best for you depends on your logging requirements. For tips, see <a href="/docs/configuration/fluentd-vs-syslog-ng/">Which log forwarder to use</a>.</p><p>You can filter and process the incoming log messages using the <a href="/docs/configuration/flow/"><strong>flow</strong></a> custom resource of the log forwarder to route them to the appropriate <a href="/docs/configuration/output/"><strong>output</strong></a>. The outputs are the destinations where you want to send your log messages, for example, Elasticsearch, or an Amazon S3 bucket. You can also define cluster-wide outputs and flows, for example, to use a centralized output that namespaced users can reference but cannot modify. Note that flows and outputs are specific to the type of log forwarder you use (Fluentd or syslog-ng).</p><p>You can configure the Logging operator using the following Custom Resource Definitions.</p><ul> | ||
|
|
@@ -4729,7 +4729,23 @@ <h1 id="pg-2ea22a017fd9fef2575c36200cd7f9b8">6.8 - CPU and memory requirements</ | |
| </span></span></span><span style="display:flex"><span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#204a87;font-weight:700">cpu</span><span style="color:#000;font-weight:700">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#0000cf;font-weight:700">1</span><span style="color:#f8f8f8;text-decoration:underline"> | ||
| </span></span></span><span style="display:flex"><span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#204a87;font-weight:700">memory</span><span style="color:#000;font-weight:700">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">1Gi</span><span style="color:#f8f8f8;text-decoration:underline"> | ||
| </span></span></span></code></pre></div></div><div class="td-content" style="page-break-before:always"> | ||
| <h1 id="pg-b9683b78eca24c9572e4995734221f3f">7 - For developers</h1><p>This documentation helps to set-up a developer environment and writing plugins for the Logging operator.</p><h2 id="setting-up-kind">Setting up Kind</h2><ol> | ||
| <h1 id="pg-58416e0a7f68f24b8e7d9b1834b3036f">7 - Images used by Logging operator</h1><p>Logging operator uses the following image versions.</p><h2 id="logging-operator-version-43">Logging operator version 4.3</h2><table> | ||
| <thead> | ||
| <tr> | ||
| <th>Image repository</th><th>GitHub repository</th><th>Version</th></tr></thead><tbody> | ||
| <tr> | ||
| <td>ghcr.io/kube-logging/node-exporter</td><td><a href="https://github.com/kube-logging/node-exporter-image">https://github.com/kube-logging/node-exporter-image</a></td><td>v0.6.1</td></tr><tr> | ||
| <td>ghcr.io/kube-logging/config-reloader</td><td><a href="https://github.com/kube-logging/config-reloader">https://github.com/kube-logging/config-reloader</a></td><td>v0.0.5</td></tr><tr> | ||
| <td>ghcr.io/kube-logging/fluentd-drain-watch</td><td><a href="https://github.com/kube-logging/fluentd-drain-watch">https://github.com/kube-logging/fluentd-drain-watch</a></td><td>v0.2.0</td></tr><tr> | ||
| <td>k8s.gcr.io/pause</td><td></td><td>3.2</td></tr><tr> | ||
| <td>docker.io/busybox</td><td><a href="https://github.com/docker-library/busybox">https://github.com/docker-library/busybox</a></td><td>latest</td></tr><tr> | ||
| <td>ghcr.io/axoflow/axosyslog</td><td><a href="https://github.com/axoflow/axosyslog-docker/">https://github.com/axoflow/axosyslog-docker/</a></td><td>4.3.0</td></tr><tr> | ||
| <td>docker.io/fluent/fluent-bit</td><td><a href="https://github.com/fluent/fluent-bit-docker-image">https://github.com/fluent/fluent-bit-docker-image</a></td><td>2.1.4</td></tr><tr> | ||
| <td>ghcr.io/kube-logging/fluentd</td><td><a href="https://github.com/kube-logging/fluentd-images">https://github.com/kube-logging/fluentd-images</a></td><td>v1.15-ruby3</td></tr><tr> | ||
| <td>ghcr.io/kube-logging/syslog-ng-exporter</td><td><a href="https://github.com/kube-logging/syslog_ng_exporter">https://github.com/kube-logging/syslog_ng_exporter</a></td><td>v0.0.16</td></tr><tr> | ||
| <td>ghcr.io/kube-logging/syslogng-reload</td><td><a href="https://github.com/kube-logging/syslogng-reload-image">https://github.com/kube-logging/syslogng-reload-image</a></td><td>v1.3.1</td></tr><tr> | ||
| <td>banzaicloud/eventrouter</td><td><a href="https://github.com/kube-logging/event-router">https://github.com/kube-logging/event-router</a></td><td>v0.1.0</td></tr></tbody></table></div><div class="td-content" style="page-break-before:always"> | ||
| <h1 id="pg-b9683b78eca24c9572e4995734221f3f">8 - For developers</h1><p>This documentation helps to set-up a developer environment and writing plugins for the Logging operator.</p><h2 id="setting-up-kind">Setting up Kind</h2><ol> | ||
| <li> | ||
| <p>Install Kind on your computer</p><div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash"><span style="display:flex"><span>go get sigs.k8s.io/[email protected] | ||
| </span></span></code></pre></div></li><li> | ||
|
|
@@ -4789,7 +4805,7 @@ <h1 id="pg-b9683b78eca24c9572e4995734221f3f">7 - For developers</h1><p>This docu | |
| </span></span></span><span style="display:flex"><span><span style="color:#8f5902;font-style:italic"></span><span style="color:#000">Buffer</span> <span style="color:#ce5c00;font-weight:700">*</span><span style="color:#000">Buffer</span> <span style="color:#4e9a06">`json:"buffer,omitempty"`</span> | ||
| </span></span></code></pre></div><h3 id="generate-docs-for-your-plugin">Generate docs for your Plugin</h3><div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash"><span style="display:flex"><span>make docs | ||
| </span></span></code></pre></div></div><div class="td-content" style="page-break-before:always"> | ||
| <h1 id="pg-956da29f025cba37fca9fe6b80ab5d48">8 - Frequently asked questions</h1><h2 id="how-can-i-run-the-unreleased-master-version">How can I run the unreleased master version?</h2><ol> | ||
| <h1 id="pg-956da29f025cba37fca9fe6b80ab5d48">9 - Frequently asked questions</h1><h2 id="how-can-i-run-the-unreleased-master-version">How can I run the unreleased master version?</h2><ol> | ||
| <li> | ||
| <p>Clone the logging-operator repo.</p><div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash"><span style="display:flex"><span>git clone [email protected]:kube-logging/logging-operator.git | ||
| </span></span></code></pre></div></li><li> | ||
|
|
@@ -4800,7 +4816,7 @@ <h1 id="pg-956da29f025cba37fca9fe6b80ab5d48">8 - Frequently asked questions</h1> | |
| <p>Helm v3</p><div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash"><span style="display:flex"><span> helm upgrade --install --wait --create-namespace --namespace logging logging ./charts/logging-operator --set image.tag<span style="color:#ce5c00;font-weight:700">=</span>master | ||
| </span></span></code></pre></div></li></ul></li></ol><h2 id="how-can-i-support-the-project">How can I support the project?</h2><ul> | ||
| <li>Give a star to <a href="https://github.com/kube-logging/logging-operator">this repository</a></li><li>Add your company to the <a href="https://github.com/kube-logging/logging-operator/blob/master/ADOPTERS.md">adopters</a> list</li></ul></div><div class="td-content" style="page-break-before:always"> | ||
| <h1 id="pg-0c44f8157757ae9807433efcf75c4e52">9 - License</h1><p>Copyright (c) 2017-2019 <a href="https://banzaicloud.com">Banzai Cloud, Inc.</a> | ||
| <h1 id="pg-0c44f8157757ae9807433efcf75c4e52">10 - License</h1><p>Copyright (c) 2017-2019 <a href="https://banzaicloud.com">Banzai Cloud, Inc.</a> | ||
| Copyright (c) 2020-2023 <a href="https://cisco.com">Cisco Systems, Inc.</a> | ||
| Copyright (c) 2023- <a href="https://github.com/kube-logging/">kube-logging authors</a></p><p>Licensed under the Apache License, Version 2.0 (the “License”); | ||
| you may not use this file except in compliance with the License. | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.