Contribute Elasticsearch Messagelog history

[MichaelRitzert]

I'd like to contribute the Elasticsearch based logging solution that we have been using for a while now. There are a few parts building on each other:
– a very simple interface to Elasticsearch (via JSON queries and HTTP)
– a model merging initial data retrieved from a database with live data coming in via JMS + an implementation using ES as the database. From the design of the code, it should also be possible to include PostgreSQL as another option.
– the corresponding GUI forked from org.csstudio.alarm.beast.msghist. This also includes an OPI widget.

Currently, I'm using org.json and com.sun.jersey, so this targets Java 8 and the CSS 4.5.x branch. For Java 11+ / CSS master, switching the the respective standard solutions (javax.json and java.net.http) is prepared.

If there is interest, I'd prepare a pull request to have it in the main release. Any proposal where such code could go?

Additionally, there's the C++ library that we use to perform logging via JMS from the IOCs. Most likely this will just go on our github page as its own project.

[shroffk]

Hello,
You can create a new module org.csstudio.logging.es under the core/utility module.

I am not sure about the C++ libraries location.

[MichaelRitzert]

OK. I will create a pull request for the actual code.
We will then have to see about how to integrate it in the build, i.e. features and repo.

[MichaelRitzert]

I'm depending on too many things from outside core, so I don't think it can go there. There are at least dependencies on org.csstudio.apputil for StartEndDialog, and additionally org.csstudio.opibuilder for the OPI widget.

[shroffk]

ok, you are right...if it depends on non core things then it might be best to move it to applications

[shroffk]

I think diag or appunorganized would both work

[MichaelRitzert]

I think I have the build working with the ElasticSearch client in utilities and the rest of the code in appunorganized. The remaining question is about the features. The ES client is tiny, I just added this to core.utility.feature. The ES client is a bit larger. Shall I create a new feature for it?

[berryma4]

Yes, please create a new feature.

[MichaelRitzert]

I updated the pull request accordingly. #2583

[MichaelRitzert]

So the pull request has been merged, but now the build again times out
https://travis-ci.com/ControlSystemStudio/cs-studio/builds/123782993 so that the new features are not yet available in the p2 repo.
How can we proceed?

[berryma4]

I'll ask Travis ci for more time

…

On Fri, Aug 23, 2019, 06:13 Michael Ritzert ***@***.***> wrote: So the pull request has been merged, but now the build again times out https://travis-ci.com/ControlSystemStudio/cs-studio/builds/123782993 so that the new features are not yet available in the p2 repo. How can we proceed? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2579?email_source=notifications&email_token=AAGQQBJXVWBIA3OHRHZS3VTQF6Z4LA5CNFSM4HHWHGWKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD47Y56Y#issuecomment-524259067>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGQQBO7RKWYDJDZH7YXNLLQF6Z4LANCNFSM4HHWHGWA> .

[berryma4]

more time has been added, and the build fails right at the end, out of memory. I put in a pull request to check the new memory addition.

[INFO] org.csstudio.desy.snl.feature ...................... SUCCESS [  0.210 s]
[INFO] snl-repository ..................................... FAILURE [01:54 min]
[INFO] applications-p2repo ................................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 59:26 min
[INFO] Finished at: 2019-08-23T17:11:23Z
[INFO] Final Memory: 1849M/1989M
[INFO] ------------------------------------------------------------------------
[ERROR] Java heap space -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/OutOfMemoryError

[berryma4]

#2611