- Dead letter index (DLI): do not send retryable errors to the DLI 793.
- Dead letter index (DLI): align the
errorfield in documents to the ECS format; the field now provideserror.messageanderror.type793.
- Remove call to EC2:DescribeRegions API in the cloudwatch-logs input 811.
- Cache EC2:DescribeRegions API response to avoid throttling and improve performance 803.
- Add dead letter index for ES outputs 733.
- Prevent duplicate _id events from reaching the replay queue 729.
- Enable multiple outputs for each input 725.
- Report misconfigured input ids as an error instead of warning, and place those messages in the replaying queue #711.
- Add documentation and optimise performance for
root_fields_to_add_to_expanded_event#642
- Go beyond 4096b limit on CF Parameter for event triggers on SAR deployment #627
- Add outputs for Lambda function ARN and IAM Role ARN #552
- Add user agent with information about ESF version and host environment: #537
- Remove calls to
sqs.DeleteMessageand refactor storage decorators: #544
- Fix regression when both
json_content_type: singleandexpand_event_list_from_fieldare set: #553
- Move
_idfield to@metadata._idin logstash output: #507
- Allow the possibility to set a prefix for role and policy when deploying with the
publish_lambda.shscript: #399
- Explicitly set
SqsManagedSseEnabledin CF template for replay and continuing queues for stack created before September/October 2022: #353
- Add
root_fields_to_add_to_expanded_eventinput setting to merge fields at root level when expanding a list of events from field: #290
- Reduced logging verbosity: #287
- Fix potential
PolicyLengthExceededExceptionwhen usingcloudwatch-logsinput: #283
- Fix events mutation across different outputs of the same input, proper handling of multiple outputs in the replay queue handler, proper handling of non json content where no json object start was ever met in the storage json collector: #260
- Fix throttling exception in
cloudwatch-logsinput related to DescribeLogStreams: #276
- Fix wrong resolved
expand_event_list_from_fieldwith AWS CloudTrail due to race condition: #244 - Fix shipper cache to include output type in the key: #245
- Added support for Logstash as output (Technical Preview): #210
- Changed event ID format to use a SHA3 384bit hash of AWS-provided ids: #227
- Fix
kinesis-data-streamdata payload type decoding and empty fields in message attributes insqscontinuation: #228 - Handle missing matching
_idfrom failed actions in elasticsearch output: #230
- Allow for extra customisation on event triggers and vpc at deployment: #201
- Improve instrumented performance bumping elastic-apm to 6.14.0: #220
- Changed ID generation logic to use AWS-provided ids: #214
- Allow to connect to ES with self-signed certificate through certificate fingerprint assertion: #173
- Allow to use a CloudWatch Logs Log Stream ARN as ID for
cloudwatch-logsinput type: #160 - Respect batch settings of original input in replay queue: #157
- Add
json_content_typeinput setting as optional hint for json content auto discovery: #145 - Switch timeout handling for
kinesis-data-streaminput to continuing queue like any other input: #146 - Handle continuation from offset in the list when dealing with
expand_event_list_from_field: #147 - Improve json parser and dumper performance: #148
- Fix multiline documentation to properly render: #137
- Add support for collecting multiline messages in a single event: #135
- Handle proper ARN format for CloudWatch Logs Log Group in the macro, as received from
ElasticServerlessForwarderCloudWatchLogsEventsCloudFormation Parameter : #130
- Handle properly gzip content regardless of the content type in S3 storage: #128
- Add support for expanding an events list from a json field: #124
- Promote to GA: #126
- Removed deprecated
es_index_or_datastream_nameconfig param: #126
- Add support for AWS IaC deployment with CloudFormation and terraform: #115
- Deprecate
es_index_or_datastream_nameconfig param in favour ofes_datastream_nameone: #115
- Handle properly
cloudwatch-logscontent payload: #113
- Add support for AWS CloudTrail logs: #110
- Handle properly flushing in
kinesis-data-streaminput type, handle properly empty messages in include exclude filters, handle properly empty lines inJsonCollectordecorator, identify properly CloudWatch Logs payload: #107
- Make the integration scope discovery available at Input component: #106
- Handle failure of replayed messages: #105
- Handle properly messages in the continuing queue originated from the continuing queue itself: #104
- Add support for providing
S3_CONFIG_FILEenv variable as cloudformation param: #103
- Add support for collecting json content spanning multiple lines: #99
- Add support for include/exclude filter: #97
- Extract
fieldssubfields at event root and make metadata for CloudWatch Logs in event in sync with Elastic Agent: #98
- Support handling of continuing queue with batch size greater than 1: #95
- Add support for CloudWatch Logs subscription filter input: #94
- Add support for direct
sqsinput: #91
- Set default
S3_CONFIG_FILEenv variable to "s3://": #90
- Remove
aws.lambda,aws.snsandaws.s3_storage_lensmetrics datasets auto-discovery #82
- Add support for sending data to an index or alias on top of datastream for the Elasticsearch output (
datasetandnamespaceconfig params replaced byes_index_or_datastream_name): #73
- Set HTTP compression always on and max retries to not exceed 15 mins in the ES client #69
- Add support for
kinesis-data-streaminput: #66
- Expose
batch_max_actionsandbatch_max_bytesconfig params for ES shipper: #65
- Handle batches of SQS records: #63
- Replay queue for ES ingestion phase failure: #60
- Routing support for AWS Services logs: #58
- Let the Lambda fail and end up in built-in retry mechanism in case of errors before ingestion phase: #57
- Support for tags: #45
- General performance refactoring after stress test outcome: #48
- Support for Secrets Manager: #42
- Integration tests for AWS Lambda handler: #24
- Proper handling of empty lines in
by_linesdecorator: #26
- Support for cloud_id and api_key in elasticsearch client: #14
- Tests for remaining packages: #12
- Optimise storage memory usage: #11
- Tests for share packages: #10
- Refactoring in offset marker: #9
- Support for type checking and related refactoring: #8
- Config support on AWS Lambda handler: #7
- First draft of the AWS Lambda handler with no config: #2