Regression on data binding in 2.11.0 with specific field naming

[geoffreywiseman]

Dependabot suggested an upgrade from Spring Boot 2.2.7 to 2.3.0, but the build failed; I checked it out and had the same failures. It looked like a Jackson issue, so I went back to Spring Boot 2.2.7 with Jackson 2.11.0, the version used in Spring Boot 2.3.0, and had the same problem.

I'm having this problem in a Kotlin project, but using Jackson-module-kotlin 2.11.0 alone doesn't seem to cause the problem.

I'm having the same problem in two places, the problem seems to be about the same in each case. It's roughly something like this: I have an API request object that can come in multiple forms, say, an owner that can be a person or organization. So OwnerRequest has an organization property, but it also has a private internal val isOrganization that returns a boolean if the organization field is set.

e.g.

data class OwnerRequest(val person: Person = null, val organization: Organization = null) {
    private val isOrganization = this.organization != null
}

I can see how there's a risk of confusion here, since Kotlin exposes getters and setters for Vals, and by java bean standards is<Property> could be a boolean getter for a a property. But in Jackson 2.10.4, all behaves as desired, and there hasn't been a problem.

However, if I replace 2.10.4 with 2.11.0, now there's a problem. Suddenly, the server is responding as if organization wasn't set (triggering a validation failure).

As a reproduction, this test:

class JacksonTest {

	private val jackson = ObjectMapper()
		.findAndRegisterModules()

	@Test
	fun testOwnerDeserialization() {
		val value: OwnerRequest = jackson.readValue(
				"""
					{
						"org": "Wayne Industries"
					}
				""".trimIndent(),
				OwnerRequest::class.java
		)
		assertThat(value.org).isEqualTo("Wayne Industries")
		assertThat(value.toString()).isEqualTo("OwnerRequest(org=Wayne Industries, isOrg=true)")
	}
}

class OwnerRequest(val org: String? = null) {
	private val isOrg = org != null
	override fun toString(): String {
		return "OwnerRequest(org=$org, isOrg=$isOrg)"
	}
}

passes in 2.10.4 and fails in 2.11.0.

Here's the failure:

com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "org" (class ca.cpp.api.submitapi.OwnerRequest), not marked as ignorable (one known property: "isOrg"])
 at [Source: (String)"{
	"org": "Wayne Industries"
}"; line: 3, column: 2] (through reference chain: ca.cpp.api.submitapi.OwnerRequest["org"])

	at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:61)
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnknownProperty(DeserializationContext.java:855)
	at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1206)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1592)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperties(BeanDeserializerBase.java:1542)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:511)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1310)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:331)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:164)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4482)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3434)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3402)
	at ca.cpp.api.submitapi.JacksonTest.testOwnerDeserialization(JacksonTest.kt:23)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)

[cowtowncoder]

Kotlin issue, needs to go under Kotlin module issue tracker; will transfer.

[Koriit]

Hello, I'am facing exactly the same issue. Though in my project the constructor value provided is skipped and default value is used.

I prepared sample project if it may help:
https://github.com/Koriit/issues-jackson-340

As already noted everything is fine in 2.10.4

I think it may have the same root cause as #337

[geoffreywiseman]

Just FWIW for anyone following up with this issue, Spring Boot 2.3.1.RELEASE still shows the same problem, but it's still using Jackson 2.11.0, so that's not surprising.

I tried bumping jackson-module-kotlin to 2.11.1, with the same results, but I haven't made an isolated Jackson/Kotlin test project to confirm that's true with a full Jackson 2.11.1 set of dependencies yet.

[geoffreywiseman]

Yeah, made a simple test that doesn't use Spring Boot, just Kotlin and Jackson, using the test above. Problem still visible. If anyone wants to have access to the project so you can validate, happy to put it up on GitHub.

[dinomite]

@geoffreywiseman Would you put that repro up & link it here? I'll take a look.

[geoffreywiseman]

Sure: https://github.com/geoffreywiseman/jackson-module-kotlin-340

[dinomite]

Branched to work on this, just a test right now, I'll update when there is progress.

https://github.com/FasterXML/jackson-module-kotlin/compare/github-340-is-field

[dinomite]

Notes:

• #337
• Commit changing KotlinNamesAnnotationIntrospector
• Related databind issue

[dinomite]

I've done a bit of work on this branch but haven't come to a solution. I pulled part of the noted commit that changed KotlinNamesAnnotationIntrospector, which fixes this issue but breaks tests Github80 and Github52. I made the latter test easier to see which specific parts of serialization are broken, but I don't know enough internals of how the name-deciding flow works in Jackson to figure out what's happening yet. Will gladly take suggestions.

[geoffreywiseman]

Hey, @dinomite -- assume this still hasn't changed? I've just tested Jackson versions up to v2.12.1, and still seeing the same behavior. I definitely don't know the internals, so I'm not sure I can help with your Sep 1 comment. :/

[dinomite]

@geoffreywiseman Correct, no movement on this :-/ More digging on the interaction of naming determinations between j-m-k & Jackson databind is required.

[k163377]

This issue will be fixed by #630 and therefore closed as a duplicate.

[k163377]

A fix for this issue will be released in 2.15.
We welcome trials and reports of problems.

[geoffreywiseman]

For what it's worth, I was cleaning out some old bug reproduction repositories and re-checked my reproduction against jackson-module-kotlin v2.15.0, and looks like the test passes for me as well. Thanks! I'll archive my repo now.