[Feature] Add support for audit log writing to data streams

[tmanninger]

I am using opensearch 2.11.

My auditlog config:

plugins.security.audit.config.index: opensearch-security-auditlog
plugins.security.audit.type: internal_opensearch

"opensearch-security-auditlog" is a datastream.

When opensearch is storing auditlog to the datastream, i get the following error:

java.lang.IllegalArgumentException: only write ops with an op_type of create are allowed in data streams

How can i change the op_type of the auditlogs?

[stephen-crawford]

[Triage] Hi @tmanninger, thank you for filing this issue. Right now we do not support the data stream API to allow this use case. Let me mark this as a feature request since it will involve adding this support to the audit logging.

That being said, sounds like we can close this issue when:

• Add data stream audit log support
• Add associated tests
• Add documentation

Are complete.

[tmanninger]

Any Feedback, any roadmap available for this future request?

[stephen-crawford]

Hi @tmanninger, I am not aware of any active efforts to provide this support. If you are eager to see this feature implemented the fastest route would likely be for you to open a pull request adding the feature. Feel free to open a PR with the change and I will try to review it as quickly as I can.

[tmanninger]

How should i integrate this feature?
Add an option
plugins.security.audit.config.is_datastream: true
or
plugins.security.audit.config.op_type: create
?

[tmanninger]

@scrawfor99 i created an PR:
#4257

I changed to OP_TYPE to create, because audit logs should never be updated.
It's tested and works in my test environment.
Looks this ok?