Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TextInputFormat with EMR Streaming #17

Open
refaelos opened this issue Nov 13, 2016 · 3 comments
Open

TextInputFormat with EMR Streaming #17

refaelos opened this issue Nov 13, 2016 · 3 comments

Comments

@refaelos
Copy link

refaelos commented Nov 13, 2016

Hey,

@carlomedas

Since EMR Streaming is using the old format of FileInputFormat class (required the old mapred package name), we can't find a way to read the compressed files within the EMR Streaming steps.

Is there a wrapper to FourMzTextInputFormat using the older api?

@refaelos refaelos changed the title TextInputFormat with EMR TextInputFormat with EMR Streaming Nov 13, 2016
@refaelos
Copy link
Author

I managed to create the input format suitable for the old api (usable with EMR Streaming).

On our fork - https://github.com/soomla/4mc

@carlomedas
Copy link
Collaborator

OK please submit pull request if you like, if it's not impacting other existing formats, I'll merge it along.

@carlomedas carlomedas reopened this Nov 14, 2016
@refaelos
Copy link
Author

@carlomedas I can do the PR but I think that it needs to support Mc input format as well first. Don't you think?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants