From 59ecbdb3df4a6014558c4ca06ac064d6f538fa25 Mon Sep 17 00:00:00 2001 From: wuzexian <1027914152@qq.com> Date: Wed, 14 Aug 2024 10:30:41 +0800 Subject: [PATCH] add elasticsearch.md --- .../pipeline-connectors/elasticsearch.md | 242 ++++++++++++++++++ 1 file changed, 242 insertions(+) create mode 100644 docs/content/docs/connectors/pipeline-connectors/elasticsearch.md diff --git a/docs/content/docs/connectors/pipeline-connectors/elasticsearch.md b/docs/content/docs/connectors/pipeline-connectors/elasticsearch.md new file mode 100644 index 0000000000..c86924a9d9 --- /dev/null +++ b/docs/content/docs/connectors/pipeline-connectors/elasticsearch.md @@ -0,0 +1,242 @@ +--- +title: "Elasticsearch" +weight: 5 +type: docs +aliases: +- /connectors/pipeline-connectors/elasticsearch +--- + + +# Elasticsearch Connector + +This article introduces the Elasticsearch Connector + +## Example + +```yaml +source: + type: values + name: ValuesSource + +sink: + type: elasticsearch + name: Elasticsearch Sink + hosts: http://localhost:9200 + version: 8 + username: elastic + password: changeme + +pipeline: + parallelism: 1 +``` +## Connector Options + +
Option | +Required | +Default | +Type | +Description | +
---|---|---|---|---|
type | +required | +(none) | +String | +Specify the Sink to use, here is 'elasticsearch' . |
+
name | +optional | +(none) | +String | +Name of Pipeline | +
hosts | +required | +(none) | +String | +The comma-separated list of Elasticsearch hosts to connect to, e.g., http://localhost:9200. | +
username | +optional | +(none) | +String | +The username for Elasticsearch authentication. | +
password | +optional | +(none) | +String | +The password for Elasticsearch authentication. | +
version | +optional | +7 | +Integer | +The version of Elasticsearch to connect to (6, 7, or 8). | +
batch.size.max | +optional | +500 | +Integer | +The maximum number of actions to buffer for each bulk request. | +
inflight.requests.max | +optional | +5 | +Integer | +The maximum number of concurrent requests that the sink will try to execute. | +
buffered.requests.max | +optional | +1000 | +Integer | +The maximum number of requests to keep in the in-memory buffer. | +
batch.size.max.bytes | +optional | +5242880 (5MB) | +Long | +The maximum size of batch requests in bytes. | +
buffer.time.max.ms | +optional | +5000 | +Long | +The maximum time to wait for incomplete batches before flushing (in milliseconds). | +
record.size.max.bytes | +optional | +10485760 (10MB) | +Long | +The maximum size of a single record in bytes. | +
Flink CDC Type | +Elasticsearch Type | +Note | +
---|---|---|
CHAR / VARCHAR / STRING | +text | ++ |
BOOLEAN | +boolean | ++ |
BINARY / VARBINARY | +binary | ++ |
DECIMAL | +scaled_float | +Elasticsearch doesn't have a native decimal type. scaled_float is used as an approximation. | +
TINYINT | +byte | ++ |
SMALLINT | +short | ++ |
INTEGER | +integer | ++ |
BIGINT | +long | ++ |
FLOAT | +float | ++ |
DOUBLE | +double | ++ |
DATE | +date | ++ |
TIME | +keyword | +Stored as string in format HH:mm:ss.SSSSSS | +
TIMESTAMP | +date | +Stored with microsecond precision | +
ARRAY | +array | ++ |
MAP / ROW | +object | ++ |