GH-41262: [Java][FlightSQL] Implement stateless prepared statements

[stevelorddremio]

Rationale for this change

Expand the number of implemented languages for stateless prepared statements to include Java.

What changes are included in this PR?

Update FlightSqlClient and include a stateless server implementation example with tests.

Are these changes tested?

Yes, tests are added to cover a stateless server implementation.

Are there any user-facing changes?

There is a modified FlightSqlClient that is required to enable use of stateless prepared statements.

• GitHub Issue: [Java][FlightSQL] Implement stateless prepared statements #41262
• GitHub Issue: [Java][FlightSQL] Implement stateless prepared statements #41262

[jduo]

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

• GitHub Issue: [FlightSQL] Stateless prepared statement with parameter support #37720

Let's create a separate Issue for this vs the existing one being used for Go.

[github-actions]

⚠️ GitHub issue #41262 has been automatically assigned in GitHub to PR creator.

[lidavidm]

It looks like all tests are failing?

[vibhatha]

@lidavidm not sure if the failures are related though. Would it be possible to run the CIs again and verify?

[lidavidm]

Still failing.

[vibhatha]

Looking into a common CI failure[1] in PR[2] and in this PR the failure [3], seems to be the same.

The error occurs initially for current PR at [4], and for PR[2] the same failure[5] occurs.

Testing C ArrowArray from file 'map_non_canonical'
... with record batch #0
Traceback (most recent call last):
  File "/arrow/dev/archery/archery/integration/runner.py", line 521, in _run_c_array_test_cases
    do_run()
  File "/arrow/dev/archery/archery/integration/runner.py", line 501, in do_run
    importer.import_batch_and_compare_to_json(json_path,
  File "/arrow/dev/archery/archery/integration/tester_csharp.py", line 135, in import_batch_and_compare_to_json
    ArrowReaderVerifier.CompareBatches(batch, imported_batch,
Xunit.Sdk.TrueException: Bit at index 0/2 is not equal

In other failing CIs the failure is also with a java.net.SocketException: Broken pipe (Write failed)

Although there is a related failure[6], which seems to be an issue with a test case in this PR.

Error:  Errors: 
Error:    TestFlightSqlStateless.setUp:50->setUpClientServer:63 � IllegalState Failed to reset Derby database!

Seems like there are related an unrelated reasons for failing the CIs.

[1] https://github.com/apache/arrow/actions/runs/8746316554/job/24003804303?pr=41297#step:8:19557
[2] #41297
[3] https://github.com/apache/arrow/actions/runs/8743037827/job/23992652573?pr=41237#step:8:19559
[4] https://github.com/apache/arrow/actions/runs/8743037827/job/23992652573?pr=41237#step:8:15258
[5] https://github.com/apache/arrow/actions/runs/8746316554/job/24003804303?pr=41297#step:8:18118
[6] https://github.com/apache/arrow/actions/runs/8743037859/job/24003647141?pr=41237#step:5:3160

[vibhatha]

It seems the following could be related

1. [C#][Integration] Integration tests failing due to C# failing to write binary arrays with empty offsets #41302
2. [C#][Integration] Integration CI job is still failing for C# consumers #41263

[vibhatha]

It would be better to wait for these changes and rebasing would probably solve the C# related issues, but the following needs to be fixed here.

Error:  Errors: 
Error:    TestFlightSqlStateless.setUp:50->setUpClientServer:63 � IllegalState Failed to reset Derby database!

I am not very fluent with this component. @lidavidm any thoughts?

[lidavidm]

I'm not sure what you're looking at. The main Java pipelines all time out here.

[vibhatha]

I'm not sure what you're looking at. The main Java pipelines all time out here.

I was referring to this error in AMD64 Windows Server 2022 Java JDK 11

[stevelorddremio]

I am investigating the test failure and thank you for the additional possible related CI failures.
When I run the tests locally as a complete suite, on Mac, this test hangs at https://github.com/apache/arrow/pull/41237/files#diff-e1eee7ea54762a289b8e5851de5f7d5d6be56d46b2abf02972c31a5673fa6f44R89.
If I run the tests individually they all pass. So there looks to be some contention between tests.
Grateful for any further suggestions.

[stevelorddremio]

I have rebased with, hopefully, some fixes to CI issues.
java.lang.IllegalStateException: Failed to reset Derby database! is still occurring in Windows tests.
In other environments the tests are being cancelled. I suspect this is the same as the hang I see running tests locally in a FlightStream.getRoot() call.

[vibhatha]

@lidavidm most CIs are failing but the error message is

Error: The operation was canceled.

Could we re-run?

[stevelorddremio]

Noted @lidavidm. I'll update to wrap results in Any.

[lidavidm]

Thanks!

[stevelorddremio]

This implementation is currently on hold pending outcome of discussions in apache/arrow-rs#5731 and apache/arrow-go#34. The current Java implementation does not wrap PutResult.applicationMetadata with Any.

[stevelorddremio]

As the consensus is that results' metadata is not wrapped in Any, this would welcome a review.
One outstanding issue is why Java JNI build is failing. I'm not sure how to resolve that.

[lidavidm]

@stevelorddremio do you mind rebasing again?

[stevelorddremio]

Rebased @lidavidm. Clean run this time. Nice. Thanks for the prompt.

[lidavidm]

Thanks for sticking with this.

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit 1598782.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 6 possible false positives for unstable benchmarks that are known to sometimes produce them.