S3 SQS Metadata

[rhysxevans]

Hi

We have the requirement to extract more metadata from the SQS notifications, at present only the bucket and object names are exported. Is this possible to get more metadata out ? https://docs.aws.amazon.com/AmazonS3/latest/userguide/notification-content-structure.html , in particular we need "eventTime" and "eventName", this is to allow for "latency" reporting from time file uploaded to file processed, and eventTime from my understanding is the time the s3 operation completed.

Any help is appreciated

Thanks

[dlvenable]

@rhysxevans , The current s3 source does not support this.

I think this could be supported as a feature enhancement. One thing we might want to change is where the S3 metadata is placed. Currently we put the S3 bucket and key in an object in the event - s3. This means that it saves to the sinks unless you remove it. We may wish to change this behavior to put the S3 metadata in the Event metadata instead. Then you could extract it using a processor.

If you'd like to create a feature request, please open a GitHub issue describing what you'd be looking for.

Also, is the metadata you are asking for part of the S3 object? It looks like it comes from the SQS message.

[rhysxevans]

Hi, thanks for the response. I have raised a feature request here #3641