WIP Allow setting custom FIX tags in encoders #462

[pcdv]

Adds support for custom tags in encoders (second part of #462)

• only int and CharSequence values can be set for now (other types can be added easily)
• TODO: the customTagsBuffer has a fixed size for now (64bytes), needs to be extendable (or bigger)
• TODO: should the feature be opt-in? it probably has negligible impact if unused (one extra if in encode())

Bonus: added an .editorconfig file to reduce checkstyle errors due to formatting

[RichardWarburton]

I don't really understand the motivation for a change like this. At the moment we have a strongly typed API around encoders with code-gen for the classes in question. If you want to add a field then the best thing to do would be to add it to your XML dictionary. Either the field exists or it doesn't. We've got shared codecs to support the case of multiple different FIX Dictionaries that need to be combined over a shared interface.

[pcdv]

I agree that the strongly typed API is one of the strengths of the current design, but it makes it a bit rigid: modifying the XML dictionary implies a regeneration of the codecs + modification of the code that uses them + a new build/CI/release cycle. The goal would be to add a little bit of flexibility to the system without losing the benefits of the strongly typed API.

We are migrating our legacy FIX routing system to an Aeron/Artio solution, and due to the number of client-side and exchange-side counterparties that we support, there are a lot of custom mapping rules that we frequently have to adapt. This can be done quickly with a change in gateway configuration, without need for a rebuild.

Adding every client-specific field to a XML dictionary means either polluting a shared dictionary with a lot of fields, or having to maintain a lot of client-specific dictionaries and the associated code. We are still early in the migration process, so for the moment this is not a huge issue. But when we start migrating more users, we will probably face some problems.

Anyway, this PR is just for discussion, and if not acceptable we will still have the option of using alternative codecs in Artio if we really start suffering from this.

Thanks

[pcdv]

If the PR is too intrusive, it can be reduced to the bare minimum: keep a buffer field (+ length, or use DirectBufferVector) and a setter in CommonEncoderImpl. The contents of buffer will be appended to the message body in encode(). The user would be responsible for building the contents of the buffer, in a FIX-compatible format.

The injection of the appending code in encode() can even be made optional in the CodecConfiguration if we want zero performance impact on existing codecs.

[RichardWarburton]

Hi,

"Adding every client-specific field to a XML dictionary means either polluting a shared dictionary with a lot of fields, or having to maintain a lot of client-specific dictionaries and the associated code. We are still early in the migration process, so for the moment this is not a huge issue. But when we start migrating more users, we will probably face some problems."

You don't need to do this, the official solution to this problem is https://github.com/real-logic/artio/wiki/Codecs#shared-codecs which takes a bunch of dictionaries and builds a shared dictionary from them in an algorithmic manner. It also does things like unify different enum types from different dictionaries that clash.

That doesn't let you alter your FIX encoding purely through configuration - though I do wonder whether that's something that we want to support in Artio generally, or a wider use-case that paying clients have.

regards,

Richard

[wojciech-adaptive]

Hi @pcdv, I implemented something similar here and was wondering if it would cover the use case you had here?

[pcdv]

Hi @wojciech-adaptive ,

Thanks, the encoder part looks like it could be useful.

It is not obvious to me whether there is any support for "AnyFields" in the decoder part. Do you confirm it is not the case?

[wojciech-adaptive]

Correct, only encoding for now.