Solr instance is not running in SolrCloud mode #221

ultimike · 2023-02-08T22:03:15Z

I'm attempting to use Drupal 10.0.3 with Search API 4.2.10.

I followed the instructions on the README, but after I set the number of shards to 3 and submit (while trying to upload a new config set from /admin/config/search/search-api/server/ddev_server I get the following error:

Configset upload failed with error code 400: Solr HTTP error: OK (400) { "responseHeader":{ "status":400, "QTime":6}, "error":{ "metadata":[ "error-class","org.apache.solr.common.SolrException", "root-error-class","org.apache.solr.common.SolrException"], "msg":"Solr instance is not running in SolrCloud mode.", "code":400}}

I am able to access the solr server at http://moreddev.ddev.site:8983/solr/#/

On the Drupal server view page (/admin/config/search/search-api/server/ddev_server):

Server Connection: The Solr server could be reached.
Collection Connection: The Solr collection could not be accessed. Further data is therefore unavailable.
Configured Solr Version: 8.11.2
Additional information: An error occurred while trying to retrieve additional information from the Solr server: Solr endpoint http://ddev-moreddev-solr:8983/ not found (code: 404, body: <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 404 Not Found</title> </head> <body><h2>HTTP ERROR 404 Not Found</h2> <table> <tr><th>URI:</th><td>/solr/techproducts/admin/luke</td></tr> <tr><th>STATUS:</th><td>404</td></tr> <tr><th>MESSAGE:</th><td>Not Found</td></tr> <tr><th>SERVLET:</th><td>default</td></tr> </table> </body> </html> , message: Solr HTTP error: OK (404) <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 404 Not Found</title> </head> <body><h2>HTTP ERROR 404 Not Found</h2> <table> <tr><th>URI:</th><td>/solr/techproducts/admin/luke</td></tr> <tr><th>STATUS:</th><td>404</td></tr> <tr><th>MESSAGE:</th><td>Not Found</td></tr> <tr><th>SERVLET:</th><td>default</td></tr> </table> </body> </html> ).

What am I missing?

thanks,
-mike

The text was updated successfully, but these errors were encountered:

rfay · 2023-02-08T22:19:21Z

Hi @ultimike - Unfortunately I haven't ever been able to make this solr cloud version work, and #195 never got far enough. @mkalkbrenner may be able to help here, but he's pretty busy.

I'm afraid, much to @mkalkbrenner's disapproval, I have to recommend the maintained, tested and stable traditional solr installation technique, ddev get ddev/ddev-drupal9-solr, https://github.com/ddev/ddev-drupal9-solr

gitressa · 2023-05-29T12:59:25Z

I think you need to add "1" after solr, like this: ddev-<project>-solr1

If you describe the project in DDEV, you'll see that the Docker containers are named solr1, solr2, etc.

$ ddev describe
┌─────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Project: myproject ~/dev/myproject https://myproject.ddev.site                                  │
│ Docker provider: docker 24.0.2                                                                  │
│ Router: traditional                                                                             │
├────────────┬──────┬────────────────────────────────────────────────────────┬────────────────────┤
│ SERVICE    │ STAT │ URL/PORT                                               │ INFO               │
├────────────┼──────┼────────────────────────────────────────────────────────┼────────────────────┤
│ web        │ OK   │ https://myproject.ddev.site                            │ drupal10 PHP8.1    │
│            │      │ InDocker: web:443,80,8025                              │ nginx-fpm          │
│            │      │ Host: 127.0.0.1:32785,32786                            │ docroot:'web'      │
│            │      │                                                        │ NodeJS:16          │
├────────────┼──────┼────────────────────────────────────────────────────────┼────────────────────┤
│ db         │ OK   │ InDocker: db:3306                                      │ mariadb:10.4       │
│            │      │ Host: 127.0.0.1:32784                                  │ User/Pass: 'db/db' │
│            │      │                                                        │ or 'root/root'     │
├────────────┼──────┼────────────────────────────────────────────────────────┼────────────────────┤
│ PHPMyAdmin │ OK   │ https://myproject.ddev.site:8037                       │                    │
│            │      │ InDocker: dba:80,80                                    │                    │
│            │      │ `ddev launch -p`                                       │                    │
├────────────┼──────┼────────────────────────────────────────────────────────┼────────────────────┤
│ solr1      │ OK   │ http://myproject.ddev.site:8983                        │                    │
│            │      │ InDocker: solr1:8983,8983                              │                    │
├────────────┼──────┼────────────────────────────────────────────────────────┼────────────────────┤
│ solr2      │ OK   │ http://myproject.ddev.site:8984                        │                    │
│            │      │ InDocker: solr2:8983,8984                              │                    │
├────────────┼──────┼────────────────────────────────────────────────────────┼────────────────────┤
│ solr3      │ OK   │ http://myproject.ddev.site:8985                        │                    │
│            │      │ InDocker: solr3:8983,8985                              │                    │
├────────────┼──────┼────────────────────────────────────────────────────────┼────────────────────┤
│ zoo        │ OK   │ InDocker: zoo:2181,2181,2888,3888,8080                 │                    │
├────────────┼──────┼────────────────────────────────────────────────────────┼────────────────────┤
│ Mailhog    │      │ MailHog: https://myproject.ddev.site:8026              │                    │
│            │      │ `ddev launch -m`                                       │                    │
├────────────┼──────┼────────────────────────────────────────────────────────┼────────────────────┤
│ All URLs   │      │ https://myproject.ddev.site,                           │                    │
│            │      │ https://127.0.0.1:32785,                               │                    │
│            │      │ http://myproject.ddev.site, http://127.0.0.1:32786     │                    │
└────────────┴──────┴────────────────────────────────────────────────────────┴────────────────────┘

If this is true, shouldn't the documentation on Apache Solr (Cloud) Integration for DDEV-Local be updated, if it is for Solr Cloud?

From: ddev-<project>-solr:8983
To: ddev-<project>-solr1:8983

gitressa · 2023-05-29T13:41:58Z

I created #227.

rfay · 2023-06-08T23:10:51Z

I'd sure rather have you create a solr cloud add-on that works! And then maintain it. Would you consider that?

gitressa · 2023-06-09T09:06:11Z

Thanks for the confidence in me @rfay! But I am afraid adjusting existing configuration is as far as my very limited Solr/Zookeeper/Docker abilities go ...

I would much prefer to get you, @ultimike and the original author @mkalkbrenner to review the PR, if at all possible.

In the meantime, perhaps you can check if it works? On my local DDEV, both standalone Solr and Solr Cloud work well, using #227. It would be great to get this confirmed from multiple sources.

Also, I created a drupal.org issue: https://www.drupal.org/project/search_api_solr/issues/3365826.

rfay · 2023-06-09T14:36:14Z

Following your d.o issue, and maybe we can get @mkalkbrenner (maintainer of search_api_solr) to speak up. And of course if you haven't carefully read and tried #195 please take a look at that.

rfay · 2023-06-09T14:38:52Z

The bottom line for me is that the classic recipe is supported and lots of people use it. I've never been successful with the newer multi-instance setup, and don't know how it adds value in a local dev setup, but I'm very naive about solr. https://github.com/ddev/ddev-drupal9-solr is supported and recommended, and this multi-solr is not supported by anybody I know of, although I keep hoping to get @mkalkbrenner supporting it, but he's a busy person.

gitressa · 2023-06-11T18:58:47Z

Yes, I agree it would be great if the maintainer of Solr @mkalkbrenner at some point had time to add the steps required to get https://github.com/docker-solr/docker-solr-examples/blob/master/docker-compose/docker-compose.yml working, so that after setting everything up with the excellent documentation on Apache Solr (Cloud) Integration for DDEV-Local it is possible for Docker/Solr/Zookeeper non-experts to deploy a Drupal 10 Solr Cloud to production, and push it over the goal line.

I managed to get https://github.com/docker-solr/docker-solr-examples/blob/master/docker-compose/docker-compose.yml up and running. But I know too little about Docker/Solr/Zookeeper to add the missing pieces in the puzzle. It may just be adjusting a few lines in docker-solr-examples/blob/master/docker-compose/docker-compose.yml to integrate solr/security.json, which is missing?

Like you describe, I used the classic single core Solr instead, and it works perfectly, except for the known issue ddev/ddev-drupal-solr#15. I get great performance. Solr Cloud may be advantageous, if you have heavy loads, and hundreds of simultaneous users? But many web sites don't have that need, so Solr Cloud would be overkill.

I think that after installing Solr with ddev get ddev/ddev-drupal9-solr that the start up slows down, so I have created ddev/ddev-drupal-solr#26.

mkalkbrenner · 2023-06-13T20:07:04Z

Strange, I run multiple projects with ddev and Solr Cloud. The name is always ddev-<project>-solr:8983, even in these I setup recently.

rfay · 2023-06-13T20:15:35Z

If you're using the https://github.com/ddev/ddev-contrib/blob/master/docker-compose-services/solr/docker-compose.solr.yaml#L20-L23 then the container names will definitely be solr1, solr2, etc.

mkalkbrenner · 2023-06-13T20:25:18Z

If you're using the https://github.com/ddev/ddev-contrib/blob/master/docker-compose-services/solr/docker-compose.solr.yaml#L20-L23 then the container names will definitely be solr1, solr2, etc.

Ok, you're right. And I think I now understand what is confusing everyone. I recommend to use
https://github.com/ddev/ddev-contrib/blob/master/docker-compose-services/solr/docker-compose.solr-standalone.yaml

This is sufficient for local development. It isn't the classic standalone mode you might have in mind. It a Solr Cloud with just one node.

and don't know how it adds value in a local dev setup

Beside the fact that it is much easier to perform Solr development, there're some APIs and features that aren't available if you choose the classic standalone mode:

But they're available in the "standalone" the ddev contrib module provides.

The setup with 3 nodes is just required if you really need to simulate or test cluster situations where you have shards, replication, worker nodes etc.

So instead changing the README and rename solr to solr1 we should change emphasize the "standalone cloud mode".

rfay · 2023-06-13T21:04:08Z

OK. The "main" docker-compose.solr.yaml here in ddev-contrib one would assume to be the "normal" one. But we should just remove that one and make the standalone one the main one, right?

Hopefully we can get an add-on out of this yet :)

gitressa · 2023-06-14T17:05:03Z

Thanks for clarifying that @mkalkbrenner. Going over the README, I got the impression that Solr Cloud with 3 Solr cores is the recommended method, so that's what I tried to get working.

Making it clearer that Solr Cloud with one node is the recommended method, and will work well for many situations, would be great. Having the documentation outline the simplest set up possible, followed by more advanced examples works best, to get the novice users up and running, in my opinion. The experts will know what to do.

So we could consider separating the two set ups, by having the first part of the README documentation only be about single core Solr Cloud? After that, we add a separate section about the more advanced 3 node Solr Cloud set up, and explain how that is set up: "For 3 node Solr core setup do the same steps as setting up single core Solr Cloud, except [...]" and then outline the differences in set up.

A missing piece in the puzzle for me is the Basic Authentication when deploying to the server. I had a look in the documentation and found https://solr.apache.org/guide/solr/latest/deployment-guide/basic-authentication-plugin.html but am not sure what the steps are.

The basic steps required to move Solr from development to server, including setting up authentication would be a great help to many.

UPDATE: Actually, I would escalate this: The basic steps required to set it up on the server is a prerequisite. If you can't deploy your Drupal site and Solr config to the production server, there's no point in using Solr Cloud.

mkalkbrenner · 2023-06-16T16:58:18Z

I just released new versions of solarium and search_api_solr this week.
I'll read some documentation and hopefully find some time next week to convert this ddev-contrib into a plugin.
Afterwards we can see how the documentation should be adjusted.

rfay · 2023-06-16T17:08:28Z

Awesome!

gitressa · 2023-06-18T12:00:12Z

Sounds fantastic @mkalkbrenner, thanks! You do an amazing job maintaining the module. It's a lot of work, with many moving parts, and I very much appreciate your efforts.

mkalkbrenner · 2023-06-20T10:55:44Z

@rfay
https://github.com/mkalkbrenner/ddev-solr

mkalkbrenner · 2023-06-20T11:04:49Z

I'll change the documentation within search_api_solr to use that add-on.
But I'll wait a bit for the decision regarding ddev/ddev#5002

rfay · 2023-06-20T12:47:16Z

Awesome!

gitressa mentioned this issue May 29, 2023

Update README and Standalone to use solr1 #227

Closed

2 tasks

gitressa mentioned this issue Jun 26, 2023

How to delete the "dev" Solr core created on installation ddev/ddev-drupal-solr#28

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Solr instance is not running in SolrCloud mode #221

Solr instance is not running in SolrCloud mode #221

ultimike commented Feb 8, 2023

rfay commented Feb 8, 2023

gitressa commented May 29, 2023

gitressa commented May 29, 2023

rfay commented Jun 8, 2023

gitressa commented Jun 9, 2023 •

edited

Loading

rfay commented Jun 9, 2023

rfay commented Jun 9, 2023

gitressa commented Jun 11, 2023 •

edited

Loading

mkalkbrenner commented Jun 13, 2023 •

edited

Loading

rfay commented Jun 13, 2023

mkalkbrenner commented Jun 13, 2023

rfay commented Jun 13, 2023

gitressa commented Jun 14, 2023 •

edited

Loading

mkalkbrenner commented Jun 16, 2023

rfay commented Jun 16, 2023

gitressa commented Jun 18, 2023

mkalkbrenner commented Jun 20, 2023

mkalkbrenner commented Jun 20, 2023

rfay commented Jun 20, 2023

Solr instance is not running in SolrCloud mode #221

Solr instance is not running in SolrCloud mode #221

Comments

ultimike commented Feb 8, 2023

rfay commented Feb 8, 2023

gitressa commented May 29, 2023

gitressa commented May 29, 2023

rfay commented Jun 8, 2023

gitressa commented Jun 9, 2023 • edited Loading

rfay commented Jun 9, 2023

rfay commented Jun 9, 2023

gitressa commented Jun 11, 2023 • edited Loading

mkalkbrenner commented Jun 13, 2023 • edited Loading

rfay commented Jun 13, 2023

mkalkbrenner commented Jun 13, 2023

rfay commented Jun 13, 2023

gitressa commented Jun 14, 2023 • edited Loading

mkalkbrenner commented Jun 16, 2023

rfay commented Jun 16, 2023

gitressa commented Jun 18, 2023

mkalkbrenner commented Jun 20, 2023

mkalkbrenner commented Jun 20, 2023

rfay commented Jun 20, 2023

gitressa commented Jun 9, 2023 •

edited

Loading

gitressa commented Jun 11, 2023 •

edited

Loading

mkalkbrenner commented Jun 13, 2023 •

edited

Loading

gitressa commented Jun 14, 2023 •

edited

Loading