Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for parameterized PailStructures #17

Closed
wants to merge 70 commits into from
Closed

Added support for parameterized PailStructures #17

wants to merge 70 commits into from

Conversation

derrickburns
Copy link

  1. Made PailTap#_pailRoot protected instead of private.
  2. Added PailTap#getPaths() method to provide paths to all subPail roots.
  3. Added Jackson 2.0 annotations to PailStructure to support Jackson serialization.
  4. Retry rename() in AbstractPail#close() because failure is costly!
  5. Serialize PailStructure into pail.meta files using Jackson 2.0. This allows PailStructures to be parameterized. This is done in a backward compatible way. Old meta files with PailStructures that take no arguments still work. However, new pail.meta files will be created by emitting not just the PailStructure class name, but a Jackson 2.0 serialization of the PailStructure. This is typically very compact, for example:

---
format: SequenceFile
structure: '{"@class":"com.rincaro.mapreduce.store.timeseriesstore.TemplateTimeSeriesPailStructure","template":"${type}"}'
args: {}

This declares a PailStructure that takes a single argument, "template", with value "${type}".

Here is the corresponding constructor with Jackson 2.0 annotations:

    @JsonCreator
    public TemplateTimeSeriesPailStructure( @JsonProperty("template") String template ) {
        mTemplate = template;
    }

    public String getTemplate() {
        return mTemplate;
    }
  1. Added pom.xml. If you don't want it, just ignore it. :)

@kul
Copy link

kul commented Sep 13, 2013

👍

@sritchie
Copy link
Collaborator

@sorenmacbeth, what do you think? big change here.

@kul
Copy link

kul commented Sep 13, 2013

@sorenmacbeth Please sort this out. the number and kind of pull requests already show that this kind of feature is most desirable for api users. 😄

@kul
Copy link

kul commented Sep 14, 2013

The pull request looks like a dead end as it seems that author has developed it into a person project. Whole code organization is changed even the Build files for scala are deleted. although this could have been the most elegant way to support dynamic pail structures. 😢 commits can be cherry picked but, seems like rewriting support for Jackson would be more easier!

@derrickburns
Copy link
Author

Yes, I wrote the pull request. After no action was taken, I dealt with the many other issues that arose. 

Cherry picking the Jackson support would be easy....


Sent from Mailbox for iPhone

On Fri, Sep 13, 2013 at 10:01 PM, kul [email protected] wrote:

The pull request looks like a dead end as it seems that author has developed it into a person project. Even the Build files for scala are deleted. although this could have been the most elegant way to support dynamic pail structures. 😢 Although commits can be cherry picked but, seems like rewriting support for Jackson would be more easier!

Reply to this email directly or view it on GitHub:
#17 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants