Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Builder property for strict input schema check #243

Open
PsychedelicO0o opened this issue Apr 6, 2022 · 4 comments
Open

Builder property for strict input schema check #243

PsychedelicO0o opened this issue Apr 6, 2022 · 4 comments

Comments

@PsychedelicO0o
Copy link

PsychedelicO0o commented Apr 6, 2022

Hi:
I've trained a pipeline model in SparkML and converted it to PMML. When I load it by using LoadingModelEvaluatorBuilder() in Java and pass a feature map into it to get result, it throws an Exception:

Exception in thread "main" org.jpmml.evaluator.MissingFieldException: Field "pmml(prediction)" is not defined
	at org.jpmml.evaluator.ModelEvaluationContext.resolve(ModelEvaluationContext.java:149)
	at org.jpmml.evaluator.EvaluationContext.evaluate(EvaluationContext.java:95)
	at org.jpmml.evaluator.ExpressionUtil.evaluateMapValues(ExpressionUtil.java:321)
	at org.jpmml.evaluator.ExpressionUtil.evaluateExpression(ExpressionUtil.java:186)
	at org.jpmml.evaluator.ExpressionUtil.evaluate(ExpressionUtil.java:156)
	at org.jpmml.evaluator.ExpressionUtil.evaluateExpressionContainer(ExpressionUtil.java:82)
	at org.jpmml.evaluator.OutputUtil.evaluate(OutputUtil.java:250)
	at org.jpmml.evaluator.ModelEvaluator.evaluateInternal(ModelEvaluator.java:470)
	at org.jpmml.evaluator.mining.MiningModelEvaluator.evaluateInternal(MiningModelEvaluator.java:237)
	at org.jpmml.evaluator.ModelEvaluator.evaluate(ModelEvaluator.java:302)
	at com.jd.campus.algorithm.pmml.NewTest.main(NewTest.java:35)

Could you please help me find how to solve it and what's the reason, cause some models throw this Exception but others are normal. Thanks
Here Is PMML File And Java Code
JMMPL-Evaluator Version : 1.5.16

@vruusmann vruusmann transferred this issue from jpmml/jpmml-evaluator Apr 6, 2022
@vruusmann vruusmann transferred this issue from jpmml/jpmml-sparkml Apr 6, 2022
@vruusmann vruusmann changed the title Field "pmml(prediction)" is not defined Builder property for strict input schema check Apr 6, 2022
@vruusmann
Copy link
Member

When I try to run the example project then I get a different exception:

Exception in thread "main" org.jpmml.evaluator.InvalidResultException: Field "quota" cannot accept user input value 100.0
        at org.jpmml.evaluator.InputFieldUtil.performInvalidValueTreatment(InputFieldUtil.java:252)
        at org.jpmml.evaluator.InputFieldUtil.prepareScalarInputValue(InputFieldUtil.java:151)
        at org.jpmml.evaluator.InputFieldUtil.prepareInputValue(InputFieldUtil.java:111)
        at org.jpmml.evaluator.ModelEvaluationContext.prepare(ModelEvaluationContext.java:76)

@vruusmann
Copy link
Member

Anyway, the root cause of the original exception (a MissingFieldException) is that some of the input fields (aka features) have missing values, the first stage of the model then predicts a missing value, and this first stage prediction cannot be post-processed by the Output/OutputField element.

This issue is a functional duplicate of #237

I did rename some relevant exception classes in the JPMML-Evalautor 1.6.X development branch to make the troubleshooting process easier.

@vruusmann
Copy link
Member

vruusmann commented Apr 6, 2022

In the current case, the missing input field is red_pack_face_value.

See for yourself:

HashMap<String, Object> featureMap = JSON.parseObject(featureStr, new TypeReference<HashMap<String, Object>>() {});
String pmmlPath = "model_jy.pmml";
Evaluator evaluator = new LoadingModelEvaluatorBuilder().load(new File(pmmlPath)).build();
evaluator.verify();
Set<String> inputFieldNames = evaluator.getInputFields().stream()
	.map(inputField -> inputField.getName().getValue())
	.collect(Collectors.toSet());
inputFieldNames.removeAll(featureMap.keySet());
System.out.println("Missing input fields: " + inputFieldNames);

@vruusmann
Copy link
Member

I renamed/re-purposed this issue in order to design a ModelEvaluatorBuilder property that would equip the returned Evaluator instance with extra input fields checking logic.

If this property is activated, then the Evaluator#evaluate(Map) method would check that all input fields are part of the arguments map. If not, the evaluate method would fail fast, without executing the real model business logic at all.

This problem is all too common with end users.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants