Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kotlin object deserialization with @JsonCreator fails in Jackson 2.18.0 #841

Closed
1 of 3 tasks
silmeth opened this issue Oct 15, 2024 · 10 comments
Closed
1 of 3 tasks
Labels
Milestone

Comments

@silmeth
Copy link

silmeth commented Oct 15, 2024

Search before asking

  • I searched in the issues and found nothing similar.
  • I searched in the issues of databind and other modules used and found nothing similar.
  • I have confirmed that the problem only occurs when using Kotlin.

Describe the bug

In Jackson 2.17 and earlier, object which implements a @JsonCreator @JvmStatic fun deserialize() function serializes and deserializes successfully. After upgrading to Jackson 2.18.0 it fails.

To Reproduce

package jackson.test

import com.fasterxml.jackson.annotation.JsonCreator
import com.fasterxml.jackson.annotation.JsonInclude
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.datatype.jdk8.Jdk8Module
import com.fasterxml.jackson.module.kotlin.readValue
import com.fasterxml.jackson.module.kotlin.registerKotlinModule
import org.junit.jupiter.api.Assertions.assertEquals
import org.junit.jupiter.api.Test

object Foo {
    override fun toString(): String = "Foo()"

    @JvmStatic
    @JsonCreator
    fun deserialize(): Foo {
        println("Foo.deserialize() called")
        return Foo
    }
}

class JacksonTest {
    private val mapper =
        ObjectMapper()
            .copy()
            .setSerializationInclusion(JsonInclude.Include.NON_ABSENT)
            .registerModule(Jdk8Module())
            .registerKotlinModule()

    @Test
    fun shouldDeserializeSimpleObject() {
        val value = Foo
        val serialized = mapper.writeValueAsString(value)
        println(serialized)
        val deserialized = mapper.readValue<Foo>(serialized)

        assertEquals(value, deserialized)
    }
}

Expected behavior

Test passes with stdout (that’s the behaviour with Jackson 2.17.0):

{}
Foo.deserialize() called

Instead what happens with 2.18.0 is the test fails with:

Test output
com.fasterxml.jackson.databind.exc.ValueInstantiationException: Cannot construct instance of `jackson.test.Foo`, problem: `java.lang.NullPointerException`
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 2]
	at app//com.fasterxml.jackson.databind.exc.ValueInstantiationException.from(ValueInstantiationException.java:47)
	at app//com.fasterxml.jackson.databind.DeserializationContext.instantiationException(DeserializationContext.java:2015)
	at app//com.fasterxml.jackson.databind.DeserializationContext.handleInstantiationProblem(DeserializationContext.java:1426)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializerBase.wrapInstantiationProblem(BeanDeserializerBase.java:2010)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:536)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1497)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:348)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185)
	at app//com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
	at app//com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4917)
	at app//com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3860)
	at app//com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3843)
	at app//jackson.test.JacksonTest.shouldDeserializeSimpleObject(JacksonTest.kt:45)
	at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at [email protected]/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at [email protected]/java.lang.reflect.Method.invoke(Method.java:569)
	at app//org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:766)
	at app//org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
	at app//org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
	at app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
	at app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
	at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
	at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
	at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
	at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
	at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$8(TestMethodTestDescriptor.java:217)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:213)
	at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:138)
	at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:68)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:156)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:146)
	at app//org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:144)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:143)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:100)
	at [email protected]/java.util.ArrayList.forEach(ArrayList.java:1511)
	at app//org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:160)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:146)
	at app//org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:144)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:143)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:100)
	at [email protected]/java.util.ArrayList.forEach(ArrayList.java:1511)
	at app//org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:160)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:146)
	at app//org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:144)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:143)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:100)
	at app//org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
	at app//org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at app//org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
	at app//org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
	at app//org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
	at app//org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
	at app//org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
	at app//org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
	at app//org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
	at app//org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
	at app//org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:124)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:99)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:94)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:63)
	at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at [email protected]/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at [email protected]/java.lang.reflect.Method.invoke(Method.java:569)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:92)
	at jdk.proxy1/jdk.proxy1.$Proxy20.stop(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:200)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:132)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:103)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:63)
	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:121)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71)
	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
Caused by: java.lang.NullPointerException
	at com.fasterxml.jackson.databind.util.internal.PrivateMaxEntriesMap.checkNotNull(PrivateMaxEntriesMap.java:226)
	at com.fasterxml.jackson.databind.util.internal.PrivateMaxEntriesMap.put(PrivateMaxEntriesMap.java:648)
	at com.fasterxml.jackson.databind.util.internal.PrivateMaxEntriesMap.putIfAbsent(PrivateMaxEntriesMap.java:633)
	at com.fasterxml.jackson.databind.util.LRUMap.putIfAbsent(LRUMap.java:64)
	at com.fasterxml.jackson.module.kotlin.ReflectionCache.valueCreatorFromJava(ReflectionCache.kt:96)
	at com.fasterxml.jackson.module.kotlin.KotlinValueInstantiator.createFromObjectWith(KotlinValueInstantiator.kt:52)
	at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:214)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:534)
	... 92 more

and stdout – showing that it got correctly serialized to empty object, but @JsonCreator never gets called:

{}

Versions

Kotlin: 2.0.21
Jackson-module-kotlin: 2.18.0
Jackson-databind: 2.18.0

Additional context

Perhaps related to FasterXML/jackson-databind#4742 (but here it’s a simple object in Kotlin instead).

@silmeth silmeth added the bug label Oct 15, 2024
lberrymage added a commit to accrescent/parcelo that referenced this issue Oct 24, 2024
Running Parcelo in a container via our current Dockerfile revealed that
"uses-sdk" fields were not being parsed from applications' Android
manifests, effectively preventing app uploads since the targetSdk
property of uses-sdk is required by Parcelo. This bug wasn't caught
until now because it only seems to manifest itself when running via the
Dockerfile; running locally as in our recommended development
environment does not have the issue. The Jackson 2.18.0 upgrade has not
yet been included in a production release, so it's uncertain whether the
issue would have surfaced in our production environment.

We tracked the issue down to a regression in Jackson 2.18.0. The exact
cause is unknown. However, a number of seemingly related issues were
reported for Jackson 2.18.0 [1], so we plan to closely monitor those
issues and test upcoming Jackson releases carefully to prevent breakage.

[1]: See below:

- FasterXML/jackson-module-kotlin#841
- FasterXML/jackson-module-kotlin#842
- FasterXML/jackson-module-kotlin#843
- FasterXML/jackson-module-kotlin#832
- FasterXML/jackson-databind#4508
- FasterXML/jackson-databind#4752
@k163377
Copy link
Contributor

k163377 commented Oct 26, 2024

@cowtowncoder
In 2.18, have you made any changes to the caller of StdValueInstantiator.configureFromObjectSettings, in particular withArgsCreator?
I have investigated the issue reported here, and it seems that for no-argument factories, StdValueInstantiator._withArgsCreator was null until 2.17, whereas in 2.18 a non-null value is passed. It seems to be 2.18.
I would appreciate any help as it has been difficult to determine the cause.

@cowtowncoder
Copy link
Member

@k163377 Although number of changes for 2.18 is big enough that I don't have full recollection, but I don't think I made any changes targeting that method.
It is possible something with POJO Property Introspection could have necessitated changes tho. Git blame would show who changed code last.

But @JooHyukKim did work on better support for any setters and that could have resulted in changes in this class

One more question -- is there any change between 2.18.0 and current 2.18 (2.18.0-SNAPSHOT)?

One last thing: this late-stage change for 2.18.0 could be relevant here:

FasterXML/jackson-databind#4688

(via PR FasterXML/jackson-databind#4689 )

lberrymage added a commit to accrescent/parcelo that referenced this issue Oct 27, 2024
Running Parcelo in a container via our current Dockerfile revealed that
"uses-sdk" fields were not being parsed from applications' Android
manifests, effectively preventing app uploads since the targetSdk
property of uses-sdk is required by Parcelo. This bug wasn't caught
until now because it only seems to manifest itself when running via the
Dockerfile; running locally as in our recommended development
environment does not have the issue. The Jackson 2.18.0 upgrade has not
yet been included in a production release, so it's uncertain whether the
issue would have surfaced in our production environment.

We tracked the issue down to a regression in Jackson 2.18.0. The exact
cause is unknown. However, a number of seemingly related issues were
reported for Jackson 2.18.0 [1], so we plan to closely monitor those
issues and test upcoming Jackson releases carefully to prevent breakage.

[1]: See below:

- FasterXML/jackson-module-kotlin#841
- FasterXML/jackson-module-kotlin#842
- FasterXML/jackson-module-kotlin#843
- FasterXML/jackson-module-kotlin#832
- FasterXML/jackson-databind#4508
- FasterXML/jackson-databind#4752
@JooHyukKim
Copy link
Member

FYI, the mentioned any-setter job is as follows

Also, I will also try to look into the solution, to figure out solution. Won't be so soon tho.

@cowtowncoder
Copy link
Member

For testing, what may help slightly is the release of 2.18.1 (happening now), can avoid use of SNAPSHOT versions (except for fixes that go in 2.18 branch from now on etc).

@JooHyukKim
Copy link
Member

JooHyukKim commented Oct 29, 2024

First Observation

First, this is where call paths diverges (between 2.17 and 2.18).
The crossroad is at _vanillaProcessing check in BeanDeserializer.deserialize().
In versions...

  • 2.17, _vanillaProcessing is true so invokes vanillaDeserialize(), where as
  • 2.18, _vanillaProcessing is false then proceeds to deserializeFromObject(p, ctxt)

Below is the method in question.

image

Second Observation

Another thing is that hitting ...

((KotlinValueInstantiator) BeanDeserializer._valueInstantiator)._defaultCreator

... on IntelliJ debugger returns in versions...

  • 2.17, the creator method is returns, where as
  • 2.18, just plain constructor is returned.

Assumptions

Looking at _vanillaProcessing changed value, seems like object class with creator is was not accounted for moving on to 2.18. Next step may be investigating _vanillaProcessing change behavior.

WDYT?

@k163377
Copy link
Contributor

k163377 commented Nov 2, 2024

Issue submitted to databind with reproduction code.
FasterXML/jackson-databind#4777

This issue needs to be fixed in databind, but I will leave this issue open as I intend to add tests in kotlin-module after the fix.

@cowtowncoder
Copy link
Member

Hmmmh. I am not yet fully convinced that 2.17 behavior is more correct than 2.18.
It comes down to semantics of what "default creator" is: should it

a. Represent either 0-argument Constructor or 0-argument Factory method (2.17)
b. Only represent 0-argument constructor; 0-argument Factory method handled as a special Properties-based Creator

If we can revert/change handling to (a) without breaking any tests, that would seem preferable.

Now: why does this change break Kotlin module? Is it calling 0-args constructor somehow, instead of factory method?
Original test is bit problematic as it uses mapper.writeValueAsString() so JSON used is not 100% clear.

I'll have a look at the new jackson-databind test PR, that may make more sense.

@k163377
Copy link
Contributor

k163377 commented Nov 3, 2024

Now: why does this change break Kotlin module?

This is because KotlinValueInstantiator.createFromObjectWith is called as a result of the processing path change from 2.17.
https://github.com/FasterXML/jackson-module-kotlin/blob/2.19/src/main/kotlin/com/fasterxml/jackson/module/kotlin/KotlinValueInstantiator.kt#L47

The location where this path change is occurring is explained in @JooHyukKim 's comment.
#841 (comment)

@cowtowncoder
Copy link
Member

@k163377 Ok; conceptually that call should be legal (as in, handling it like arguments-taking Creators should work the same way), but I agree it is at least sub-optimal: 0-args factory method cannot take any arguments so there is no need to go through the usual complications of buffering etc.

So I think ideally we'd figure out how to keep 0-args factory method as special case that does not go through regular properties-based-creator handling.

If that is not possible, Kotlin module should probably be changed to handle this case (whether it considers 0-args special or not).

k163377 added a commit to k163377/jackson-module-kotlin that referenced this issue Nov 16, 2024
k163377 added a commit that referenced this issue Nov 16, 2024
@k163377
Copy link
Contributor

k163377 commented Nov 16, 2024

As I commented to FasterXML/jackson-databind#4777, @cowtowncoder 's fix will resolve this issue in 2.18.2.
I have also added a test to detect regression based on the code submitted by @silmeth .

Therefore, this issue is closed.

@k163377 k163377 closed this as completed Nov 16, 2024
@cowtowncoder cowtowncoder modified the milestones: 2.18.0, 2.18.2 Nov 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants