Custom build issue for SecurityIDSource tag-td = 22

[sushil023]

Describe the bug
As per our requirement we have made changes to FIX50SP2.xml file added some new messages and tags and new supported values for those tags. What ever changes we done in the xml file after doind mvn clean install in their corresponding generated source files those changes are reflecting. But only for tag-d 22 that is SecurityIDSource we added 2 new supported values to it 403 and 403 . After rebuild the code in SecurityIDSouce.java file those new supported values not reflecting.

To Reproduce
Open the FIX50SP2.modified.xml file and under add below new values

and then rebuild it using mvn clean install after installation done check in SecurityIDSource.java file new 2 values 403 and 404 adding or not.

Expected behavior
After rebuild in SecurityIDSource.java file 403 and 404 two new values with its entries should be show.

system information:

• OS: Windows11
• Java version JDK21
• QFJ Version latest master branch code

Additional context
Add any other context about the problem here.

[chrjohn]

To which file did you make changes? You mention one time FIX50SP2 and one time FIX50SP2.modified

[sushil023]

I changed to FIX50SP2.modified.xml file , as in our application we are using the file as FIX50SP2.xml file so taken the modified FIX50SP2.xml file content and replace the content to FIX50SP2.modified.xml file content in Quickfixj source code to rebuild it.

[chrjohn]

@sushil023 , you are correct, I does not work for me either.

@david-gibbs-ig , could you maybe take a look? I'm pretty sure that this worked before 3.0.0-SNAPSHOT. I added two enums and rebuilt, but the changes are not reflected in the code. Thank you in advance. :)

diff --git a/quickfixj-messages/quickfixj-messages-fix50sp2/src/main/resources/FIX50SP2.modified.xml b/quickfixj-messages/quickfixj-messages-fix50sp2/src/main/resources/FIX50SP2.modified.xml
index 069cf483..21362bd5 100644
--- a/quickfixj-messages/quickfixj-messages-fix50sp2/src/main/resources/FIX50SP2.modified.xml
+++ b/quickfixj-messages/quickfixj-messages-fix50sp2/src/main/resources/FIX50SP2.modified.xml
@@ -5502,6 +5502,8 @@
       <value enum="K" description="ISDA_FPML_PRODUCT_URL"/>
       <value enum="L" description="LETTER_OF_CREDIT"/>
       <value enum="M" description="MARKETPLACE_ASSIGNED_IDENTIFIER"/>
+      <value enum="403" description="FOO"/>
+      <value enum="404" description="BAR"/>
     </field>
     <field number="23" name="IOIID" type="STRING"/>
     <field number="25" name="IOIQltyInd" type="CHAR">

Edit: when building on 2.3.x I can see the changed enums in the file ./quickfixj-core/target/generated-sources/quickfix/field/SecurityIDSource.java which is the only file with that name.

On 3.0.0-SNAPSHOT there are the following files but none of them has the changed enums:

./quickfixj-messages/quickfixj-messages-fix50sp2/target/generated-sources/quickfix/field/SecurityIDSource.java
./quickfixj-messages/quickfixj-messages-fix50/target/generated-sources/quickfix/field/SecurityIDSource.java
./quickfixj-messages/quickfixj-messages-fixlatest/target/generated-sources/quickfix/field/SecurityIDSource.java
./quickfixj-messages/quickfixj-messages-fix50sp1/target/generated-sources/quickfix/field/SecurityIDSource.java

When building I can see the following:

[INFO] --- quickfixj-codegenerator:3.0.0-SNAPSHOT:generate (fix50sp2) @ quickfixj-messages-all ---
[INFO] Successfully created an instance of the QuickFIX source generator
[INFO] Initialising code generator task
[INFO] Processing /home/cj/macd/workspace/quickfixj-new/quickfix-j-git/quickfixj/quickfixj-messages/quickfixj-messages-fix50sp2/src/main/resources/FIX50SP2.modified.xml
[INFO] FIX50SP2.modified.xml: generating field classes in /home/cj/macd/workspace/quickfixj-new/quickfix-j-git/quickfixj/quickfixj-messages/quickfixj-messages-all/target/generated-sources/quickfix/field/

The named file in the output above has my changes but the file in the directory that is listed above does NOT have my changes although the file timestamp is up-to-date.
Any thoughts?

[sushil023]

Similar issue is there for tag-id 460 “Product” if adding new value to it not reflecting..!!!

[chrjohn]

Similar issue is there for tag-id 460 “Product” if adding new value to it not reflecting..!!!

Well, I'd say that is expected when the other tags did not work. Could you use 2.3.x to generate the classes?

[david-gibbs-ig]

Please have a look at ./quickfixj-messages/readme.md. It's a bit of a read. I think it explains the project build correctly but I think I could have been more clear about the generation of Fields . I hope I don't make any mistakes in the following explanation, I'm trying to answer quickly so you can have a solution.

Please have a look at the README.md in the root of the project. Have a look under ## QuickFIX/J Runtime. This tries to explain that, to implement custom Rules of Engagement, one only needs to build separately the Application Message Packages. You need to have a build of the quickjfixj-core and quickjfixj-base and quickfixj-messages-fixt11 packages but the application message packages such as quickfixj-messages-fix50sp2 can be built separately.
In this case you can track quickfixj repo for the quickjfixj-core, quickjfixj-base and quickfixj-messages-fixt11 packages for changes and you may optionally maintain quickfixj-messages-fix50sp2 independently. The application message packages that you want need only be provided at runtime. quickjfixj-core, quickjfixj-base and quickfixj-messages-fixt11 do not have compile time dependence on quickfixj-messages-fix50sp2. There is a little more to it but that is the basic idea. You can use different mvn co-ordinates for the quickfixj-messages-fix50sp2 that you build, as long as the jar is in the classpath (without conflicting jars) .

The way that 3.0.0-SNAPSHOT works is similar to the older versions with regard to the FIX Protocol Fields . The QuickFIX/J build treats these as "universal". All the Fields are generated and are "shared" by different versions of the application protocols (4.0 through 5.0sp2). You can see this if you look at the generated classes in quickfixj-messages-all . For both old and new ways of building ordering can be important but I think that there is a lot of flexibility in the new way of building. The following excerpt from the readme is relevant.

The code generation for the FIX Protocol specification versions is done in the quickfixj-messages-all module. It proceeds in order from oldest to most recent (latest), with the class generated from most recent specification of a Field over-writing any prior classes. This behaviour can be controlled using the overwrite parameter on the quickfixj-codegenerator maven plugin and by changing the order of processing. Consequently, in the reference QuickFIX/J build, almost all the Field classes are those generated from the FIX Latest specification as in the orchestration published by the quickfixj-orchestration module.

So we see that the definition of fields and code sets (the enumerations for legitimate values of fields) comes from fixlatest. The definition of messages comes from the respective FIX protocol version XML files.

You can customise the code sets and/or add new Fields by updating the FIX Orchestra file in the QuickFIX/J project.
This is probably the fastest way to get something up and running.
This is not the ideal way in my opinion. If you have time to try it out, I recommend that custom builds be achieved by leaving the QuickFIX/J project intact and building the custom packages independently. In this way one can use the quickfij-core, quickjfixj-base and quickfixj-messages-fixt11 packages from the QuickFIX/J project without even having to maintain a fork of the entire project. Well once there is a release version of QuickFIX/J that is. In the meanwhile one can use the quickfij-core, quickjfixj-base and quickfixj-messages-fixt11 snapshots built from a clone of the QuickFIX/J project. One can just update the cloine from upstream without having to manage any conflicts caused by local changes to the fork/clone.

From customising-quickfixj.md please note From QuickFIX/J 3.0.0 the code generation for quickfix.Field prefers the FIX Orchestra Standard. I think this doc describes some of the ways to accomplish customisation.

So what is the easiest and most familiar way to build the custom quickfixj-messages-fix50sp2 ? Please see ./quickfixj-messages/readme.md .
One way to start a custom build this is to copying this maven module, or only quickfixj-messages-all, into an independent build. ....

I would suggest generating code using a copy of the quickfixj-messages-all and quickfixj-messages-fix50sp2 modules. To make it straightforward, use these modules as they are in branch QuickFIX/J 2_3_x . You can thereby build quickfixj-messages-fix50sp2 using ONLY the FIX50SP2.modified.xml in the original way . You can use all the tools such as the quickfixj-codegenerator from the latest version of the project but generate code using the old way. You can use quickfij-core, quickjfixj-base and quickfixj-messages-fixt11 from the latest version of the project.

I can add an example of this approach in quickfixj-custom-orchestration-examples, See PR - https://github.com/quickfix-j/quickfixj-custom-orchestration-examples/pull/1 . It won't be able to do this for a couple of weeks though.

Update ! Example has been added, see further comments.

Hope this helps, kind regards

[david-gibbs-ig]

So basically this isn't a bug, to see a quick resolution, update the code set in the FIX Orchestra File.

[david-gibbs-ig]

Please find an example build from customizing the FIX50SP2 QFJ Dictionary here :

https://github.com/david-gibbs-ig/quickfixj-custom-orchestration-examples/tree/main/custom-legacy-codegen-application-messages

https://github.com/david-gibbs-ig/quickfixj-custom-orchestration-examples

QFJ 3.0.0 is not yet released so to see this working -

build and install QFJ locally
./mvnw install -Dmaven.javadoc.skip=true -DskipTests -PskipBundlePlugin,minimal-fix-latest

build the examples
mvn clean install -Dmaven.javadoc.skip=true

use the generated artifact as a dependency for your application alongside the QFJ base, core and FIXT1.1 artifacts, taking care to omit the FIX50SP2 artifact published from the QFJ project, as described in the readme,md .

i.e.
<groupId>org.quickfixj.custom.examples</groupId> <artifactId>legacy-codegen-application-messages</artifactId> <version>0.0.1-SNAPSHOT</version>
instead of
<groupId>org.quickfixj</groupId> <artifactId>quickfixj-messages-fix50sp2</artifactId> <version>3.0.0-SNAPSHOT</version>

[david-gibbs-ig]

Example of a custom build and using it here : quickfix-j/quickfixj-custom-orchestration-examples#1

The example is for FIXLatest. An example of using legacy codegen custom library will follow. The example of building a custom library is already n the project.

Update ! The axample of using the custom build from the legacy codegen is now committed.

@sushil023 @chrjohn See also comments above

[david-gibbs-ig]

@sushil023 @chrjohn quickfix-j/quickfixj-custom-orchestration-examples#1 includes examples of building custom messages packages and using them at runtime. You can clone from main on this fork https://github.com/david-gibbs-ig/quickfixj-custom-orchestration-examples to try them. You can build the SNAPSHOT version of the QuickFIX/J project independently as described above to install the QuickFIX/J SNAPSHOT artifacts locally so they can be used in the example custom projects. Hope this helps.