Schema conversion retains entity types unknown in target schema

[hlg]

From reading the code in org.bimserver.schemaconverter.AbstractSchemaConverter I was expecting that entities of types not contained in the target schema would be omitted from serialization. However, a sample conversion from IFC4 to IFC2x3 revealed that this is not the case. For example, the file IfcDoorType_4.ifc.txt contains an entity of type IfcDoorType which was unknown in IFC2x3. The conversion result, IfcDoorType_2x3.ifc.txt, still contains exactly the same number of entities, including the IfcDoorType entity. Consequently the subsequent checkin of the invalid IFC2x3 file into a new BIMserver project fails.

I could not yet trace how these elements end up in the model, but I suspect org.bimserver.ifc.step.serializer.IfcStepSerializer to follow some references outside of the filtered model it was given to serialize.

Proper schema conversion could help to deal with legacy files when transitioning from IFC2x3 to IFC4, but since in IFC4 some entity types were deleted, this issue would reduce the usefulness of BIMserver in the process.

[rubendel]

Good one, these schema converters were never finished. Also some of the code is in the wrong project. AbstractSchemaConverter and it's subclasses should be in a plugin project and only the SchemaConverter interface should be in the BIMserver project. That way other people can write better implementations without having to change BIMserver code.

[hlg]

I found that actually the non-streaming serializer works fine, because of the check whether model.contains(referencedObject) (IfcStepSerializer.java#L224). The issue only applies to the streaming serializer, which does not perform this check and thus includes referenced entities outside the model, which was indeed properly prepared by AbstractSchemaconverter.

Of course, still, converters would need to do some cleanup in consequence of omitting those entities. Having schema conversion as an extension point would be great.

[rubendel]

Ah oke, I agree the nicest fix would be in the schema converter.

Ideally the conversion would not be fully "automatic" as well, a programmed conversion would for some types make more sense.

[hlg]

I believe the streaming serializers don't currently use the schema converter. That's another reason why it can't be fixed in the serializer alone. If you factor out schema converters as plugins, then no serializer should bypass the conversion, of course.