Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for python deaggregation of Java KPL aggregated records #81

Open
redparham opened this issue Jan 23, 2020 · 1 comment
Open

Comments

@redparham
Copy link

This project focuses mainly on supporting Kinesis (de)aggregation in the context of aws lambda. Would it be possible to make the deggregation code more generic so that it can be used to deaggregate records produced by Java KPL?

We have a use case where we produce using Java KPL and consume using python via boto3 client.get_records. There is no builtin support for processing these aggregated records from python: https://docs.aws.amazon.com/streams/latest/dev/kinesis-kpl-integration.html. The code in deaggregator.py iter_deaggregate_records(records) is coupled to kinesis/lambda but with some refactoring, I think it could satisfy my use case.

@IanMeyers
Copy link
Contributor

Agreed. Separating the structure of the input event from the base64 decoding and deaggregating should be trivial to implement in the code with backward compatibility. Happy to take a PR on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants