Restarting web server requires re-index of all projects

[judkinr]

We recently stepped up to a newer version of both OpenGrok and Tomcat, and have noticed a not-so-desirable effect. We restart our tomcat instance every so often to combat server/process instability. With the newer versions, we've noticed that we are unable to do OpenGrok searches once tomcat is restarted. It reports 'You must select a project!' despite no project being available. The search works fine, and there are projects available prior to the restart. The only way we've discovered to get around this was to completely re-index very app we had. This was not a problem with the older versions we were using. This is problematic because we have about 15 or so projects we like to index, and some take several hours to index. If we need to regenerate all of these every time tomcat restarts, that's going to be an issue.

[judkinr]

I have found that manually invoking the REST API calls to mark each project as 'indexed' will refresh the GUI and allow us to search again - but I wouldn't expect that to be the correct answer here.

[vladak]

How do you invoke the indexer exactly ? When the indexer successfully indexes a project, it marks it as indexed (both in its local configuration and via API call to the webapp) and if the -W indexer option is used, the configuration which contains this information is written to disk. The web app deployment should contain path to this file in the CONFIGURATION parameter within web.xml. When the webapp starts, it loads the configuration from disk and uses it.

The scenario you are seeing might happen when e.g. per-project indexing is in use. There, the -W option is not used. Instead, the overall indexing process retrieves the configuration from the webapp at the end of indexing and stores it to disk so that any data that have changed inside the webapp are reflected on the next startup.

[judkinr]

Hi Vlad,

When we invoke the indexer to generate the config, we are using the -W option. However, nothing seems to update the webapp config file to point to the correct data configuration. The indexer invokes the REST API and updates the running webapp, but nothing updates the file on disk.

Currently we also run indexing without the -U option because we do not run the indexing operation on the tomcat server itself, and it is unable to notify the server remotely of the updated configs/index. Instead, we 'manually' invoke the REST API calls after indexing is complete to notify the server of the updates.

Regardless, whether we update the webapp config on disk, or use the -U/-W flags, nothing seems to persist on tomcat restart. And, yes we are using the 'project' indexing.

[vladak]

In per-project setup, you need to download the configuration from the webapp (using REST API call) once the indexing is complete and write it to disk to the location where the webapp expects it.