You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
....
2023-06-26T22:30:19,634 [historical-data-migration-sink-worker-2-thread-1] INFO org.opensearch.dataprepper.pipeline.Pipeline - Pipeline [historical-data-migration] Sink is ready, starting source...
2023-06-26T22:30:19,641 [historical-data-migration-sink-worker-2-thread-1] INFO org.opensearch.dataprepper.plugins.source.opensearch.worker.client.OpenSearchClientFactory - Using username and password for auth for the OpenSearch source
2023-06-26T22:30:19,642 [historical-data-migration-sink-worker-2-thread-1] ERROR org.opensearch.dataprepper.pipeline.common.PipelineThreadPoolExecutor - Pipeline [historical-data-migration] process worker encountered a fatal exception, cannot proceed further
java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Username may not be null
at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:?]
at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:?]
at org.opensearch.dataprepper.pipeline.common.PipelineThreadPoolExecutor.afterExecute(PipelineThreadPoolExecutor.java:70) [data-prepper-core-2.4.0-SNAPSHOT.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1137) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: java.lang.IllegalArgumentException: Username may not be null
at org.apache.http.util.Args.notNull(Args.java:54) ~[httpcore-4.4.16.jar:4.4.16]
at org.apache.http.auth.UsernamePasswordCredentials.<init>(UsernamePasswordCredentials.java:81) ~[httpclient-4.5.14.jar:4.5.14]
at org.opensearch.dataprepper.plugins.source.opensearch.worker.client.OpenSearchClientFactory.attachUsernamePassword(OpenSearchClientFactory.java:175) ~[opensearch-source-2.4.0-SNAPSHOT.jar:?]
at org.opensearch.dataprepper.plugins.source.opensearch.worker.client.OpenSearchClientFactory.createOpenSearchRestClient(OpenSearchClientFactory.java:137) ~[opensearch-source-2.4.0-SNAPSHOT.jar:?]
at org.opensearch.dataprepper.plugins.source.opensearch.worker.client.OpenSearchClientFactory.provideOpenSearchClient(OpenSearchClientFactory.java:75) ~[opensearch-source-2.4.0-SNAPSHOT.jar:?]
at org.opensearch.dataprepper.plugins.source.opensearch.worker.client.SearchAccessorStrategy.getSearchAccessor(SearchAccessorStrategy.java:61) ~[opensearch-source-2.4.0-SNAPSHOT.jar:?]
at org.opensearch.dataprepper.plugins.source.opensearch.OpenSearchSource.startProcess(OpenSearchSource.java:49) ~[opensearch-source-2.4.0-SNAPSHOT.jar:?]
at org.opensearch.dataprepper.plugins.source.opensearch.OpenSearchSource.start(OpenSearchSource.java:41) ~[opensearch-source-2.4.0-SNAPSHOT.jar:?]
at org.opensearch.dataprepper.pipeline.Pipeline.startSourceAndProcessors(Pipeline.java:210) ~[data-prepper-core-2.4.0-SNAPSHOT.jar:?]
at org.opensearch.dataprepper.pipeline.Pipeline.lambda$execute$2(Pipeline.java:251) ~[data-prepper-core-2.4.0-SNAPSHOT.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
... 2 more
2023-06-26T22:30:19,654 [historical-data-migration-sink-worker-2-thread-1] INFO org.opensearch.dataprepper.pipeline.Pipeline - Pipeline [historical-data-migration] - Received shutdown signal with processor shutdown timeout PT30S and sink shutdown timeout PT30S. Initiating the shutdown process
...
It looks like the plugin is expecting username and password configurations in the pipeline even when the source cluster does not have security enabled (which mine does not since i'm using the OSS distro and security is an X-Pack plugin)
To Reproduce
Steps to reproduce the behavior:
Start/use an ElasticSearch OSS source cluster (i'm using ES OSS 7.10.2) without any auth
Configure a Data Prepper pipeline with the following source:
Describe the bug
Firstly, thanks for merging in support for the scroll API for the ElasticSearch accessor! When I attempted to use this plugin with a test ElasticSearch OSS source cluster, I ran into the following error:
It looks like the plugin is expecting
username
andpassword
configurations in the pipeline even when the source cluster does not have security enabled (which mine does not since i'm using the OSS distro and security is an X-Pack plugin)To Reproduce
Steps to reproduce the behavior:
Expected behavior
The above source configuration should work without errors
Screenshots
N/A, stack-trace provided above
Environment (please complete the following information):
Additional context
I am able to work around this problem by providing dummy values for username and password in the config:
The ES source cluster ignores/does not parse these values so the API call goes through.
The text was updated successfully, but these errors were encountered: