From 1c24739c79edd6e466bfa134830cf98c3d6a7bb8 Mon Sep 17 00:00:00 2001 From: "mergify[bot]" <37929162+mergify[bot]@users.noreply.github.com> Date: Tue, 10 Oct 2023 10:31:08 -0400 Subject: [PATCH] Add list of Elastic Agent inputs for standalone configuration (#561) (#575) (cherry picked from commit 389af46d87b3da7939ec42538cc1a25944f1ebaa) Co-authored-by: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com> --- .../inputs/input-configuration.asciidoc | 2 +- .../configuration/inputs/inputs-list.asciidoc | 408 ++++++++++++++++++ docs/en/ingest-management/index.asciidoc | 2 + 3 files changed, 411 insertions(+), 1 deletion(-) create mode 100644 docs/en/ingest-management/elastic-agent/configuration/inputs/inputs-list.asciidoc diff --git a/docs/en/ingest-management/elastic-agent/configuration/inputs/input-configuration.asciidoc b/docs/en/ingest-management/elastic-agent/configuration/inputs/input-configuration.asciidoc index 47d0a5e10..94b5d96c1 100644 --- a/docs/en/ingest-management/elastic-agent/configuration/inputs/input-configuration.asciidoc +++ b/docs/en/ingest-management/elastic-agent/configuration/inputs/input-configuration.asciidoc @@ -33,7 +33,7 @@ By default {agent} collects system metrics, such as CPU, memory, network, and fi ----------------------------------------------------------------------- <1> A unique ID for the input. -<2> A generic type describing the data. +<2> The name of the input. Refer to <> for the list of what's available. <3> The name of the `output` to use. If not specified, `default` will be used. <4> Package specification. <5> A user-defined namespace. diff --git a/docs/en/ingest-management/elastic-agent/configuration/inputs/inputs-list.asciidoc b/docs/en/ingest-management/elastic-agent/configuration/inputs/inputs-list.asciidoc new file mode 100644 index 000000000..b7657e197 --- /dev/null +++ b/docs/en/ingest-management/elastic-agent/configuration/inputs/inputs-list.asciidoc @@ -0,0 +1,408 @@ +[[elastic-agent-inputs-list]] += {agent} inputs + +When you <> for standalone {agents}, the following values are supported for the input `type` parameter. + +*Expand any section to view the available inputs:* + +// Auditbeat +[[elastic-agent-inputs-list-auditbeat]] +[%collapsible] +.Audit the activities of users and processes on your systems +==== + +|=== +|Input |Description |Learn more + +|`audit/auditd` +|Receives audit events from the Linux Audit Framework that is a part of the Linux kernel. +|{auditbeat-ref}/auditbeat-module-auditd.html[Auditd Module] ({auditbeat} docs) + +|`audit/file_integrity` +|Sends events when a file is changed (created, updated, or deleted) on disk. The events contain file metadata and hashes. +|{auditbeat-ref}/auditbeat-module-file_integrity.html[File Integrity Module] ({auditbeat} docs) + +|`audit/system` +|beta[] Collects various security related information about a system. All datasets send both periodic state information (e.g. all currently running processes) and real-time changes (e.g. when a new process starts or stops). +|{auditbeat-ref}/auditbeat-module-system.html[System Module] ({auditbeat} docs) + +|=== + +==== + +// Metricbeat +[[elastic-agent-inputs-list-metricbeat]] +[%collapsible] +.Collect metrics from operating systems and services running on your servers +==== + +|=== +|Input |Description |Learn more + +|`activemq/metrics` +|Periodically fetches JMX metrics from Apache ActiveMQ. +|{metricbeat-ref}/metricbeat-module-activemq.html[ActiveMQ module] ({metricbeat} docs) + +|`apache/metrics` +|Periodically fetches metrics from https://httpd.apache.org/[Apache HTTPD] servers. +|{metricbeat-ref}/metricbeat-module-apache.html[Apache module] ({metricbeat} docs) + +|`aws/metrics` +|Periodically fetches monitoring metrics from AWS CloudWatch using https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_GetMetricData.html[GetMetricData API] for AWS services. +|{metricbeat-ref}/metricbeat-module-aws.html[AWS module] ({metricbeat} docs) + +|`awsfargate/metrics` +|beta[] Retrieves various metadata, network metrics, and Docker stats about tasks and containers. +|{metricbeat-ref}/metricbeat-module-awsfargate.html[AWS Fargate module] ({metricbeat} docs) + +|`azure/metrics` +|Collects and aggregates Azure logs and metrics from a variety of sources into a common data platform where it can be used for analysis, visualization, and alerting. +|{metricbeat-ref}/metricbeat-module-azure.html[Azure module] ({metricbeat} docs) + +|`beat/metrics` +|Collects metrics about any Beat or other software based on libbeat. +|{metricbeat-ref}/metricbeat-module-beat.html[Beat module] ({metricbeat} docs) + +|`cloudfoundry/metrics` +|Connects to Cloud Foundry loggregator to gather container, counter, and value metrics into a common data platform where it can be used for analysis, visualization, and alerting. +|{metricbeat-ref}/metricbeat-module-cloudfoundry.html[Cloudfoundry module] ({metricbeat} docs) + +|`containerd/metrics` +|beta[] Collects cpu, memory and blkio statistics about running containers controlled by containerd runtime. +|{metricbeat-ref}/metricbeat-module-containerd.html[Containerd module] ({metricbeat} docs) + +|`docker/metrics` +|Fetches metrics from https://www.docker.com/[Docker] containers. +|{metricbeat-ref}/metricbeat-module-docker.html[Docker module] ({metricbeat} docs) + +|`elasticsearch/metrics` +|Collects metrics about {es}. +|{metricbeat-ref}/metricbeat-module-elasticsearch.html[Elasticsearch module] ({metricbeat} docs) + +|`enterprisesearch/metrics` +|Periodically fetches metrics and health information from Elastic {ents} instances using HTTP APIs. +|{metricbeat-ref}/metricbeat-module-enterprisesearch.html[{ents} module] ({metricbeat} docs) + +|`etcd/metrics` +|This module targets Etcd V2 and V3. When using V2, metrics are collected using https://coreos.com/etcd/docs/latest/v2/api.html[Etcd v2 API]. When using V3, metrics are retrieved from the `/metrics`` endpoint as intended for https://coreos.com/etcd/docs/latest/metrics.html[Etcd v3]. +|{metricbeat-ref}/metricbeat-module-etcd.html[Etcd module] ({metricbeat} docs) + +|`gcp/metrics` +|Periodically fetches monitoring metrics from Google Cloud Platform using https://cloud.google.com/monitoring/api/metrics_gcp[Stackdriver Monitoring API] for Google Cloud Platform services. +|{metricbeat-ref}/metricbeat-module-gcp.html[Google Cloud Platform module] ({metricbeat} docs) + +|`haproxy/metrics` +|Collects stats from http://www.haproxy.org/[HAProxy]. It supports collection from TCP sockets, UNIX sockets, or HTTP with or without basic authentication. +|{metricbeat-ref}/[HAProxy module] ({metricbeat} docs) + +|`http/metrics` +|Used to call arbitrary HTTP endpoints for which a dedicated Metricbeat module is not available. +|{metricbeat-ref}/metricbeat-module-http.html[HTTP module] ({metricbeat} docs) + +|`iis/metrics` +|Periodically retrieve IIS web server related metrics. +|{metricbeat-ref}/metricbeat-module-iis.html[IIS module] ({metricbeat} docs) + +|`jolokia/metrics` +|Collects metrics from https://jolokia.org/reference/html/agents.html[Jolokia agents] running on a target JMX server or dedicated proxy server. +|{metricbeat-ref}/metricbeat-module-jolokia.html[Jolokia module] ({metricbeat} docs) + +|`kafka/metrics` +|Collects metrics from the https://kafka.apache.org/intro[Apache Kafka] event streaming platform. +|{metricbeat-ref}/metricbeat-module-kafka.html[Kafka module] ({metricbeat} docs) + +|`kibana/metrics` +|Collects metrics about {Kibana}. +|{metricbeat-ref}/metricbeat-module-kibana.html[{kib} module] ({metricbeat} docs) + +|`kubernetes/metrics` +|As one of the main pieces provided for Kubernetes monitoring, this module is capable of fetching metrics from several components. +|{metricbeat-ref}/metricbeat-module-kubernetes.html[Kubernetes module] ({metricbeat} docs) + +|`linux/metrics` +|beta[] Reports on metrics exclusive to the Linux kernel and GNU/Linux OS. +|{metricbeat-ref}/metricbeat-module-linux.html[Linux module] ({metricbeat} docs) + +|`logstash/metrics` +|collects metrics about {ls}. +|{metricbeat-ref}/metricbeat-module-logstash.html[{ls} module] ({metricbeat} docs) + +|`memcached/metrics` +|Collects metrics about the https://memcached.org/[memcached] memory object caching system. +|{metricbeat-ref}/metricbeat-module-memcached.html[Memcached module] ({metricbeat} docs) + +|`mongodb/metrics` +|Periodically fetches metrics from https://www.mongodb.com/[MongoDB] servers. +|{metricbeat-ref}/metricbeat-module-mongodb.html[MongoDB module] ({metricbeat} docs) + +|`mssql/metrics` +|The https://www.microsoft.com/en-us/sql-server/sql-server-2017[Microsoft SQL 2017] Metricbeat module. It is still under active development to add new Metricsets and introduce enhancements. +|{metricbeat-ref}/metricbeat-module-mssql.html[MSSQL module] ({metricbeat} docs) + +|`mysql/metrics` +|Periodically fetches metrics from https://www.mysql.com/[MySQL] servers. +|{metricbeat-ref}/metricbeat-module-mysql.html[MySQL module] ({metricbeat} docs) + +|`nats/metrics` +|Uses the https://nats.io/documentation/managing_the_server/monitoring/[Nats monitoring server APIs] to collect metrics. +|{metricbeat-ref}/metricbeat-module-nats.html[NATS module] ({metricbeat} docs) + +|`nginx/metrics` +|Periodically fetches metrics from https://nginx.org/[Nginx] servers. +|{metricbeat-ref}/metricbeat-module-nginx.html[Nginx module] ({metricbeat} docs) + +|`oracle/metrics` +|The https://www.oracle.com/[Oracle] module for Metricbeat. It is under active development with feedback from the community. A single Metricset for Tablespace monitoring is added so the community can start gathering metrics from their nodes and contributing to the module. +|{metricbeat-ref}/metricbeat-module-oracle.html[Oracle module] ({metricbeat} docs) + +|`postgresql/metrics` +|Periodically fetches metrics from https://www.postgresql.org/[PostgreSQL] servers. +|{metricbeat-ref}/metricbeat-module-postgresql.html[PostgresSQL module] ({metricbeat} docs) + +|`prometheus/metrics` +|Periodically scrapes metrics from https://prometheus.io/docs/instrumenting/exporters/[Prometheus exporters]. +|{metricbeat-ref}/metricbeat-module-prometheus.html[Prometheus module] ({metricbeat} docs) + +|`rabbitmq/metrics` +|Uses the http://www.rabbitmq.com/management.html[HTTP API] created by the management plugin to collect RabbitMQ metrics. +|{metricbeat-ref}/metricbeat-module-rabbitmq.html[RabbitMQ module] ({metricbeat} docs) + +|`redis/metrics` +|Periodically fetches metrics from http://redis.io/[Redis] servers. +|{metricbeat-ref}/metricbeat-module-redis.html[Redis module] ({metricbeat} docs) + +|`sql/metrics` +|Allows you to execute custom queries against an SQL database and store the results in {es}. +|{metricbeat-ref}/metricbeat-module-sql.html[SQL module] ({metricbeat} docs) + +|`stan/metrics` +|Uses https://github.com/nats-io/nats-streaming-server/blob/master/server/monitor.go[STAN monitoring server APIs] to collect metrics. +|{metricbeat-ref}/metricbeat-module-stan.html[Stan module] ({metricbeat} docs) + +|`statsd/metrics` +|Spawns a UDP server and listens for metrics in StatsD compatible format. +|{metricbeat-ref}/metricbeat-module-statsd.html[Statsd module] ({metricbeat} docs) + +|`syncgateway/metrics` +|beta[] Monitor a Sync Gateway instance by using its REST API. +|{metricbeat-ref}/metricbeat-module-syncgateway.html[SyncGateway module] ({metricbeat} docs) + +|`system/metrics` +|Allows you to monitor your server metrics, including CPU, load, memory, network, processes, sockets, filesystem, fsstat, uptime, and more. +|{metricbeat-ref}/metricbeat-module-system.html[System module] ({metricbeat} docs) + +|`traefik/metrics` +|Periodically fetches metrics from a https://traefik.io/[Traefik] instance. +|{metricbeat-ref}/metricbeat-module-traefik.html[Traefik module] ({metricbeat} docs) + +|`uwsgi/metrics` +|By default, collects the uWSGI stats metricset, using https://uwsgi-docs.readthedocs.io/en/latest/StatsServer.html[StatsServer]. +|{metricbeat-ref}/metricbeat-module-uwsgi.html[uWSGI module] ({metricbeat} docs) + +|`vsphere/metrics` +|Uses the https://github.com/vmware/govmomi[Govmomi] library to collect metrics from any Vmware SDK URL (ESXi/VCenter). +|{metricbeat-ref}/metricbeat-module-vsphere.html[vSphere module] ({metricbeat} docs) + +|`windows/metrics` +|Collects metrics from Windows systems. +|{metricbeat-ref}/metricbeat-module-windows.html[Windows module] ({metricbeat} docs) + +|`zookeeper/metrics` +|Fetches statistics from the ZooKeeper service. +|{metricbeat-ref}/metricbeat-module-zookeeper.html[ZooKeeper module] ({metricbeat} docs) + +|=== + +==== + +// Filebeat +[[elastic-agent-inputs-list-filebeat]] +[%collapsible] +.Forward and centralize log data +==== + +|=== +|Input |Description |Learn more + +|`aws-cloudwatch` +|Stores log files +from Amazon Elastic Compute Cloud(EC2), AWS CloudTrail, Route53, and other sources. +|{filebeat-ref}/filebeat-input-aws-cloudwatch.html[AWS CloudWatch input] ({filebeat} docs) + +|`aws-s3` +|Retrieves logs from S3 objects that are pointed to by S3 notification events read from an SQS queue or directly polling list of S3 objects in an S3 bucket. +|{filebeat-ref}/filebeat-input-aws-s3.html[AWS S3 input] ({filebeat} docs) + +|`azure-blob-storage` +|Reads content from files stored in containers which reside on your Azure Cloud. +|{filebeat-ref}/filebeat-input-azure-blob-storage.html[Azure Blob Storage] ({filebeat} docs) + +|`azure-eventhub` +|Reads messages from an azure eventhub. +|{filebeat-ref}/filebeat-input-azure-eventhub.html[Azure eventhub input] ({filebeat} docs) + +|`cel` +|Reads messages from a file path or HTTP API with a variety of payloads using the https://opensource.google.com/projects/cel[Common Expression Language (CEL)] and the https://pkg.go.dev/github.com/elastic/mito/lib[mito] CEL extension libraries. +|{filebeat-ref}/filebeat-input-cel.html[Common Expression Language input] ({filebeat} docs) + +|`cloudfoundry` +|Gets HTTP access logs, container logs and error logs from Cloud Foundry. +|{filebeat-ref}/filebeat-input-cloudfoundry.html[Cloud Foundry input] ({filebeat} docs) + +|`cometd` +|Streams the real-time events from a Salesforce generic subscription Push Topic. +|{filebeat-ref}/filebeat-input-cometd.html[CometD input] ({filebeat} docs) + +|`container` +|Reads containers log files. +|{filebeat-ref}/filebeat-input-container.html[Container input] ({filebeat} docs) + +|`docker` +|Alias for `container`. +|- + +|`log/docker` +|Alias for `container`. +|n/a + +|`entity-analytics` +|Collects identity assets, such as users, from external identity providers. +|{filebeat-ref}/filebeat-input-entity-analytics.html[Entity Analytics input] ({filebeat} docs) + +|`event/file` +|Alias for `log`. +|n/a + +|`event/tcp` +|Alias for `tcp`. +|n/a + +|`filestream` +|Reads lines from active log files. Replaces and imporoves on the `log` input. +|{filebeat-ref}/filebeat-input-filestream.html[filestream input] ({filebeat} docs) + +|`gcp-pubsub` +|Reads messages from a Google Cloud Pub/Sub topic subscription. +|{filebeat-ref}/filebeat-input-gcp-pubsub.html[GCP Pub/Sub input] ({filebeat} docs) + +|`gcs` +|beta[] Reads content from files stored in buckets which reside on your Google Cloud. +|{filebeat-ref}/filebeat-input-gcs.html[Google Cloud Storage input] ({filebeat} docs) + +|`http_endpoint` +|beta[] Initializes a listening HTTP server that collects incoming HTTP POST requests containing a JSON body. +|{filebeat-ref}/filebeat-input-http_endpoint.html[HTTP Endpoint input] ({filebeat} docs) + +|`httpjson` +|Read messages from an HTTP API with JSON payloads. +|{filebeat-ref}/filebeat-input-httpjson.html[HTTP JSON input] ({filebeat} docs) + +|`journald` +|beta[] A system service that collects and stores logging data. +|{filebeat-ref}/filebeat-input-journald.html[Journald input] ({filebeat} docs) + +|`kafka` +|Reads from topics in a Kafka cluster. +|{filebeat-ref}/filebeat-input-kafka.html[Kafka input] ({filebeat} docs) + +|`log` +|DEPRECATED: Please use the `filestream` input instead. +|n/a + +|`logfile` +|Alias for `log`. +|n/a + +|`log/redis_slowlog` +|Alias for `redis`. +|n/a + +|`log/syslog` +|Alias for `syslog`. +|n/a + +|`mqtt` +|Reads data transmitted using lightweight messaging protocol for small and mobile devices, optimized for high-latency or unreliable networks. +|{filebeat-ref}/filebeat-input-mqtt.html[MQTT input] ({filebeat} docs) + +|`netflow` +|Reads NetFlow and IPFIX exported flows and options records over UDP. +|{filebeat-ref}/filebeat-input-netflow.html[NetFlow input] ({filebeat} docs) + +|`o365audit` +|beta[] Retrieves audit messages from Office 365 and Azure AD activity logs. +|{filebeat-ref}/filebeat-input-o365audit.html[Office 365 Management Activity API input] ({filebeat} docs) + +|`osquery` +|Collects and decodes the result logs written by https://osquery.readthedocs.io/en/latest/introduction/using-osqueryd/[osqueryd] in the JSON format. +| - + +|`redis` +|beta[] Reads entries from Redis slowlogs. +|{filebeat-ref}/[Redis input] ({filebeat} docs) + +|`syslog` +|Reads Syslog events as specified by RFC 3164 and RFC 5424, over TCP, UDP, or a Unix stream socket. +|{filebeat-ref}/filebeat-input-syslog.html[Syslog input] ({filebeat} docs) + +|`tcp` +|Reads events over TCP. +|{filebeat-ref}/filebeat-input-tcp.html[TCP input] ({filebeat} docs) + +|`udp` +|Reads events over UDP. +|{filebeat-ref}/filebeat-input-udp.html[UDP input] ({filebeat} docs) + +|`unix` +|beta[] Reads events over a stream-oriented Unix domain socket. +|{filebeat-ref}/[Unix input] ({filebeat} docs) + +|`winlog` +|Reads from one or more event logs using Windows APIs, filters the events based on user-configured criteria, then sends the event data to the configured outputs ({es} or {ls}). +|{winlogbeat-ref}[Winlogbeat Overview] ({winlogbeat} docs) + +|=== + +==== + +// Heartbeat +[[elastic-agent-inputs-list-heartbeat]] +[%collapsible] +.Monitor the status of your services +==== + +|=== +|Input |Description |Learn more + +|`synthetics/http` +|Connect via HTTP and optionally verify that the host returns the expected response. +|{heartbeat-ref}/monitor-http-options.html[HTTP options] ({heartbeat} docs) + +|`synthetics/icmp` +|Use ICMP (v4 and v6) Echo Requests to check the configured hosts. +|{heartbeat-ref}/monitor-icmp-options.html[ICMP options] ({heartbeat} docs) + +|`synthetics/tcp` +|Connect via TCP and optionally verify the endpoint by sending and/or receiving a custom payload. +|{heartbeat-ref}/monitor-tcp-options.html[TCP options] ({heartbeat} docs) + +|=== + +==== + +// Packetbeat +[[elastic-agent-inputs-list-packetbeat]] +[%collapsible] +.View network traffic between the servers of your network +==== + +|=== +|Input |Description |Learn more + +|`packet` +|Sniffs the traffic between your servers, parses the application-level protocols on the fly, and correlates the messages into transactions. +|{packetbeat-ref}/packetbeat-overview.html[Packetbeat overview] ({packetbeat} docs) + +|=== + +==== \ No newline at end of file diff --git a/docs/en/ingest-management/index.asciidoc b/docs/en/ingest-management/index.asciidoc index cbd4ac3f7..0cfd084da 100644 --- a/docs/en/ingest-management/index.asciidoc +++ b/docs/en/ingest-management/index.asciidoc @@ -167,6 +167,8 @@ include::elastic-agent/configuration/inputs/input-configuration.asciidoc[levelof include::elastic-agent/configuration/inputs/simplified-input-configuration.asciidoc[leveloffset=+3] +include::elastic-agent/configuration/inputs/inputs-list.asciidoc[leveloffset=+3] + include::elastic-agent/elastic-agent-dynamic-inputs.asciidoc[leveloffset=+3] include::elastic-agent/configuration/providers/elastic-agent-providers.asciidoc[leveloffset=+2]