pmml File is missing DataField

[siriJR]

Hello，My model is displayed in python environment 【model.active_fields】has 70features, But there are only 46 features in pmml。Some features are not displayed in the pmml file, which leads to errors when making predictions in the java environment.

---

print(sklearn.version) 0.20.0
print(sklearn.externals.joblib.version) 0.12.5
print(sklearn_pandas.version) 1.8.0
print(sklearn2pmml.version) 0.52.1

thx.

[siriJR]

sorry,It should be the exception caused by this, long type is converted to int：

java.lang.IllegalArgumentException: 100000005561

 at org.jpmml.evaluator.TypeUtil.parseInteger(TypeUtil.java:147)
 at org.jpmml.evaluator.TypeUtil.parseInteger(TypeUtil.java:118)
 at org.jpmml.evaluator.TypeUtil.parse(TypeUtil.java:76)
 at org.jpmml.evaluator.TypeUtil.parseOrCast(TypeUtil.java:60)
 at org.jpmml.evaluator.RichDataField.parseValues(RichDataField.java:95)
 at org.jpmml.evaluator.RichDataField.getMap(RichDataField.java:64)
 at org.jpmml.evaluator.MapHolder.get(MapHolder.java:33)
 at org.jpmml.evaluator.InputFieldUtil.getStatus(InputFieldUtil.java:295)
 at org.jpmml.evaluator.InputFieldUtil.prepareScalarInputValue(InputFieldUtil.java:140)
 at org.jpmml.evaluator.InputFieldUtil.prepareInputValue(InputFieldUtil.java:111)
 at org.jpmml.evaluator.InputField.prepare(InputField.java:70)
 at com.jd.ofc.orcs.sim.service.impl.MaliciousModelServiceImpl.evaluate(MaliciousModelServiceImpl.java:118)
 at com.jd.ofc.orcs.sim.service.impl.MaliciousModelServiceImpl.excute(MaliciousModelServiceImpl.java:38)
 at com.jd.ofc.orcs.sim.FullReductionModelTest.testExecute(FullReductionModelTest.java:43)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
 at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
 at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
 at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
 at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
 at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
 at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
 at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
 at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
 at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
 at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
 at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
 at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
 at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
 at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
 at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.lang.ArithmeticException: integer overflow
 at java.lang.Math.toIntExact(Math.java:1011)
 at org.jpmml.evaluator.TypeUtil.parseInteger(TypeUtil.java:145)
 ... 35 more

Process finished with exit code -1

[vruusmann]

My model is displayed in python environment 【model.active_fields】has 70 features, But there are only 46 features in pmml

Your model only needs 46 features for making a prediction. The remaining (70 - 46) = 24 features are not needed (simply wasting space and computation resources).

All JPMML family conversion libraries analyze model objects to identify needed/not needed features; the former are kept, the latter discarded. It's a feature, not a bug.

If that's a problem for you, you can always open the final PMML file (manually or programmatically), and insert whatever garbage you like into it.

[vruusmann]

sorry,It should be the exception caused by this, long type is converted to int：
java.lang.IllegalArgumentException: 100000005561

This is a known limitation: jpmml/jpmml-evaluator#141

If you need to represent numeric values that don't fit into 32-bit integers, consider changing the data type of the field from integer to double. Alternatively, if it's a pseudo-numeric value (looks like number, but actually isn't - ex. a phone number), consider changing from integer to string.