Are the model inference performances diff a lot between differnet code languages(ig. python and java) ?

[EasonChow001]

Are the model inference performances diff a lot between differnet code languages(ig. python and java) ?
I ran a tinybert model use different code language, but it seems the performances diff a lot.
Python: almost 2ms per inference.
Java: almost 10ms per inference.

[Craigacp]

How are you creating the inputs for the Java code? Using arrays is much slower than using ByteBuffers, because Java requires that arrays are copied to get into native code and python doesn't require that for numpy arrays (or at least I don't think it does), similarly the outputs are copied when using an array and not when using a buffer.

There may also be slight differences in how the native libraries are compiled for the published Python & Java bindings, so did you compile them from source or use the published ones on PyPI and Maven?

One further consideration is that the Java code will take a little while to warm up, as it has a JIT compiler for best performance, whereas Python is always interpreted and so it has less of a warm up effect, so it depends on how you're benchmarking it too.

[EasonChow001]

I retried, the time cost is near. Thx a lot.