Add support for serialization of object members (using prefixing of parent name)

[fenjen]

Let's assume we have two classes:

class A {
  String s1 = "A_string1";
  String s2 = "A_string2";
  @JsonIgnore B b;
}

class B {
  String s1 = "B_string1";
  String s2 = "B_string2";
}

If we'd serialize an instance of A the result is A_string1,A_string2
The serialization of an instance of B would look like this B_string1,B_string2

Now If we'd remove the @JsonIgnore from class A, we'd get the Exception:
com.fasterxml.jackson.core.JsonGenerationException: CSV generator does not support Object values for properties

Why couldn't we just serialize b in a -- the result would be quite intuitive: A_string1,A_string2,B_string1,B_string2

This would enable jackson-csv to serialize object members as it's possible in XML and JSON.

[tatu-at-salesforce]

Limitation was added so we can figure out what would be the best way to deal with embedded values.
Just serializing them in order could work well, so I will consider this a request to add handling of structured simple values.

[fenjen]

Thank you!
Transitive members should work as well: they'd have to be serialized recursively.

[cowtowncoder]

Yes, understood. Sorry for long delay, but it is possible that I could tackle this for next release (2.3).

[cowtowncoder]

Oh. Forgot to add one obvious related thing: use of @JsonUnwrapped is supported (at least since 2.2.2), and this does allow inclusion of members of embedded objects (including recursive inclusion). While not as convenient as automatically handling this (which I hope to tackle as per above), at least it makes it possible; and probably commonly used with mix-in annotations.

[shaon002]

using @JsonUnwrapped didn't work for me. Wondering when this support will be added?

[cowtowncoder]

Did not work in what way?

[shaon002]

My initial purpose was to serialize the following POJO class

public class Application {
String name;
String description;
// failed to serialize this member
HashMap <String, String> customAttr;
....
}
Does jackson support this case

[cowtowncoder]

Ah. The problem is with Maps; unwrapping with @JsonUnwrapped will not work with things other than POJOs, as it requires knowledge of properties that class has (which is not defined for Maps).

Could you add a new issue entry, asking for support to handle Maps? It is theoretically doable, but requires different kind of support than POJOs, because Maps can have any keys, not just ones defined as properties.

[5kilodope]

@JsonUnwrapped is a bit of a problem if we want to use the pojos with other formats (json,xml) because those formats have no problem with nested elements but would also respect the @JsonUnwrapped annotation per default.

I would also like to give some input ;-)
following Pojo:

class Customer {
  Integer id;
  Address billing;
  Address shipping;
}
class Address {
  String first_name;
  String last_name;
  Contact contact;
}
class Contact {
  String phone;
}

Why not just always use the name of the parent-element as prefix in the CSV. So the Pojo would be serialized like this (first line is csv-schema):

id|billing:first_name|billing:last_name|billing:contact:phone|shipping:first_name|shipping:last_name|shipping:contact:phone|
7|max|headroom|01648288383|julia|headroom|0164828383

[cowtowncoder]

@5kilodope At conceptual level that would make sense I think. The challenge is just the interaction between data-binding and streaming parser, whether that could be made to work considering that data-binding is format agnostic.
I think separator (or even strategy) to use needs to be configurable (some might prefer period, or slash), but that's a minor thing.

Come to think of this, it might well be possible if streaming parser and generator handled this transparently.
Or... perhaps by handling it with CsvSchema construction.

If done by streaming layer, this would simply mean that generator would "output" JsonToken.START_OBJECT / END_OBJECT by enabling/changing prefix to use, to basically output fully-qualified name. And parser would strip these out, based on similar matching. So that data-binding would not see fully-qualified names; these would be transparently handled at streaming layer.

If done via schema, it could work more similar to @JsonUnwrapped, and basically expose fully-qualified names at data-binding. But this could also require automatic enabling of wrapping, unwrapping.

There are some drawbacks to both approaches, but I think former (handle it at CSV-specific streaming) might be better all around. One challenge there would be need for heuristics, to assume such naming convention, but it seems like perhaps lesser evil. It would also be more performant, and aside from theoretical possibility of name collision (user explicitly uses prefix notation), would probably be more robust.

[5kilodope]

@cowtowncoder

thanks for your feedback. I see it is not that easy, but we already use JSON/XML Databinding for our little Import/Export Tool. And it would therefore be nice if CSV could be also used, and could be equally powerful for pojo-mapping.

JSON/XML already provide a complete schema within their data.
In CSV the schema is the first line of the data.

So theoretically CSV could be equally powerful, if the schema in the first line is used.

Hope you find a good solution on that ;-)

[cowtowncoder]

@5kilodope Yes, first line can be optionally used as the source for schema already (if not, caller must provide it).
And now that mention it, schema-defined name could/should play part in the detection.

I agree in that there's much more that could be done with CSV, using little bit of naming convention, at least for nested Objects. Arrays are bit trickier, but even there it might be possible to use in-field separators.

[kringol]

Is there any pseudo-easy way to achieve this or similar nowadays by using a list of JsonPointer paths for colums and let the csvmapper write those nodes to the csv, provided that the resulting node of the pointer is of a serializable type and not an object.
JsonNode already supports doing node.at("/rootNode/objectField/name" or even with array positions

[cowtowncoder]

No. Jackson stays out of transformation business as a boundary. You can achieve this outside of core databind by creating a JsonNode, transforming it, but there is nothing automated and I have no plans to add anything that requires building of intermediate tree structures.

But as I said @JsonUnwrapped does work for flattening, for cases where that is applicable.

[zadeswapnilzade]

this is going to be in next release ?

[cowtowncoder]

@zadeswapnilzade There is no active development for this feature at this point in time.

[don-han]

@JsonUnwrapped doesn't work if we are using the same POJO for deserializing from JSON and serializing to CSV as @5kilodope pointed out.

It seems like mix-ins might be able to solve this problem, but given that the conversion between JSON and CSV are probably the main use for this library, I think this would be a very valuable feature.

[cowtowncoder]

@don-han Certainly would be valuable, if someone had time to work on this.

[jmatthewpryor]

One simple, but manual solution might be to allow dot notation in the Schema Builder

CsvSchema schema = CsvSchema.builder()
        .addColumn("person.firstName")
        .addColumn("person.firstName")
        .addColumn("person.age", CsvSchema.ColumnType.NUMBER)
        .build();

That would allow for writing CSV from complex objects

[cowtowncoder]

@jmatthewpryor there are couple of related challenges: but perhaps something like this could work, similar to how jackson-dataformat-properties does things (by keeping track of prefix by object nesting).

[alianos-]

@don-han mixins seems deprecated.

[cowtowncoder]

@alianos- Mix-ins are certainly not deprecated as a feature. But maybe you mean something else?

[alianos-]

If I write mapper.addMixInAnnotations(Person.class, PersonFormat.class); my IDE marks the method as deprecated. Tbh I didn't look more into it since I didn't think it can help with the issue at hand.

[ptahchiev]

+1 for this :)

[ptahchiev]

Guys, I have created a pull-request for this here: FasterXML/jackson-dataformats-text#97 however I have one test failing. I feel like I'm almost there, perhaps @cowtowncoder can help me out :)

[ptahchiev]

I have updated the PR and now all the tests are passing. However, I am not yet 100% sure what in what I have implemented. @cowtowncoder can you please have a look at it and if there's something else to be done, please let me know.