Multiple geoPoints per document

[gmourier]

Allowing to store several geo coords per document could allow finding a document in case its geo coords are in several shapes or one or several.

Please, tell us why this could be interesting for your use case!

Thank you!

[mmachatschek]

We have an index for companies that have multiple subsidiaries where we have the addresses and geopoints that we would like to search through.

ATM we need to add a workaround to search for all the geopoints by either

• indexing the companies multiple times with different primary keys and different subsidiaries (in conjunction with https://docs.meilisearch.com/reference/api/distinct_attribute.html#get-distinct-attribute)
• or by separating the company subsidiary locations into a separate index and search the geopoints there, get the company ids from the result and filter the companies by the ids in the main companies index

[gmourier]

Hey @jhf, sorry to be late 👋

I'm not sure I understand well. Could you share an example of the data and a mock-up/screen of the interface to see how you would interact with it?

Thanks!

[furmannodesol]

Any solution for your multiple subsidiaries? @mmachatschek

[mmachatschek]

I currently use my first suggested solution from above. (Indexing the model multiple times with a custom key and using a distinct attribute) current downside we have is that the estimatedHits count is not really accurate

[furmannodesol]

Thank you @mmachatschek for the reply.
How do you fetch the results with a custom key which actually doesn't exist in the database. Would be great if you could elaborate little bit more.

[jhf]

Hey @jhf, sorry to be late 👋

I'm not sure I understand well. Could you share an example of the data and a mock-up/screen of the interface to see how you would interact with it?

Thanks!

Hi!
So, we have a search function, where the user may search for something, say the name of a place, the name of a shop, the name of a brand, the name of a category for a shop. However these results are always limited by geoposition, so the search may be done nearby London, or som place with a geographical position.
As a simplified example, say there are shops in the database:

• Shop A - Sporting goods - Near London
• Shop B - Food - Near London
• Shop D - Food - Far away from London.
• Shop C - Shoes - Far away from London.

Now the user enters the free text search, and types "S".
We would like to make a suggestion of completions, and that list could look like:

• "Sporting Goods"
• "Shop A"
• "Shop B"

And in that list, there should not be any "Shoes" because there are no relevant search results for that nearby London.

However if the user starts typing "F" then the suggestions may be

• "Food"

So how could we organise the data for searching?

I've assumed that we can not use some kind of nested data structure

• One index for the shops, each with one geoposition.
• Add the category "Shoes"/"Food"/"Sporting Goods" to the shop.
  Could we search for completion using one index?
  Could one make another index for category inside shop?

Rather we create different indices, one for shop and another for category.

• Shop index with name and geoposition.
• Category index with name and list of geopositions.
  And then search over two indices and order according to say distance, and within some distance interval by relevance.

Does that make it clearer @gmourier ?
Oh, and thank you for taking the time to ask!

Now the

[alexphiev]

Great proposal! Ideally, something like Algolia's solution:

"_geoloc": [
  { "lat": 47.279430, "lng": 5.106450 },
  { "lat": 47.293228, "lng": 5.004570 },
  { "lat": 47.316669, "lng": 5.016670 }
]

Reference: https://www.algolia.com/doc/guides/managing-results/refine-results/geolocation/

Thanks a lot!

[mmachatschek]

Hey @gmourier, sorry for pinging you but are there any updates or further thoughts on this topic?

[mmachatschek]

thanks for sharing the current status 🙌 @gmourier

[mmachatschek]

Hi @gmourier I would be happy to see the issue on the meilisearch/meilisearch repo. This would probably be something that I would try to explore and implement myself or to do a POC.

[irevoire]

Hey @mmachatschek, today I spent some time trying to describe how we could implement it, but there is a flaw in our filter design.
Now, since we have the _geoRadius operator I don't think we can implement it without rewriting a considerable part of the engine either on the filtering part (that would solve a second issue we have with the filter around nested objects) or a big rewrite of the geodatabase (that would be the occasion to handle the geo shape).
Whatever happens, I was very wrong when I said it was a quick win. Sorry, it's actually very hard to implement, and I can't even estimate which solution we should choose and how long it would take sorry 😢
Next monday I'll talk about it with kero and come back to you if he has a better idea.

[mmachatschek]

@irevoire very appreciate the effort here 👍

[irevoire]

Ok so in conclusion it's as I said.
One “not too long fix” would be to introduce a new special database just to handle the geo bounding box, but I don't think we're going to do that gr
We're probably going to wait until we rewrite the whole geo pipeline 😖

[ramanjitsevo]

Multiple geoPoints for a single document is it still an issue or it has been fixed?
My data is something like:
"locations" => [
0 => array:[
"id" => 8894
"lat" => 33.97
"lng" => -84.36
],
1 => array:[
"id" => 579723
"lat" => 33.7382
"lng" => -84.4079
],
2 => array:[
"id" => 579723
"lat" => 33.7382
"lng" => -84.4079
]
]
A single documents contains multiple locations.
I need to find documents within 20 miles range from $lat and $lng.

[gmourier]

Hey @ramanjitsevo 👋

While we are committed to other priorities right now, a workaround may be worth evaluating on your side in the meantime even if not ideal.

It has previously been described by @mmachatschek here

You could split your different locations into multiple documents and denormalize the root data, such as brand/company info within its various documents.

[
     {
          "id": "7adff1a6-20cf-11ee-be56-0242ac120002", //the id of a location; could be auto-generated or hashed from the lat, lng if there is no location identifier at hand
          "_geo": {
              "lat": 33.97,
              "lng": -84.36
          },
          ...,
          "brand": "Burger King",
          "brand_id": 1
    },
    {
          "id": "eb197582-20cf-11ee-be56-0242ac120002",
          "_geo": {
              "lat": 33.7382,
              "lng": -84.4079
          },
          ...,
          "brand": "Burger King",
          "brand_id": 1
    },
    {
          "id": "f73b52b8-20cf-11ee-be56-0242ac120002",
          "_geo": {
              "lat": 32.7382
              "lng": -83.4079
          },
          ...,
          "brand": "Taco Bell"
          "brand_id": 2
    }
]

Have you considered it?

Thanks

[martin-landhi]

Sound good temporally solution. i will test it and come back in a few days with conclutions, thanks @gmourier

[mmachatschek]

@martin-landhi, we are using this feature in production for ~ 9months. It is a solution but (hopefully only temporarily, until multi geo point support has landed) its slow (if you use paginated results) and needs some extra implementation to keep all documents for each variant in sync. Especially empty queries (landingpages) are very slow if you don't cache search results

empty queries -> super slow (if you have many variants)
using a search query + filters -> gets faster if less documents are in the search result (meilisearch doesn't have to do too much computation for the subset)

[martin-landhi]

Perfect @mmachatschek i will test it with more or less 20k documents and see the performance, but caching the answer can be good solution if it is slow, because most of them will be empty queries

[martin-landhi]

i could apply your solution @mmachatschek, thanks. About the performance right now in not an issue but when the data growth maybe i have to apply cache for this.

[martin-landhi]

So, we should not expect any progress here soon?

I have also many location per document to be searcheables

[ramanjitsevo]

Waste of time. I had to move to another platform.

[macraig]

Hi @martin-landhi @ramanjitsevo 👋 Unfortunately, we haven't made progress on this front. We truly understand how important this feature is to you and recognize its potential value for our users. While we'd love to address all user requests promptly, there are times when we face challenges that require us to reprioritize our efforts. Currently, our team is focused on addressing critical issues and working on features that have a broader impact across our user base.

Please know that we take your feedback and requests seriously, and we deeply appreciate your patience and understanding in this matter. We want to assure you that your requests have been duly noted, and we have not abandoned the idea. While we don't have a concrete timeline to share for the implementation at this moment, we assure you that we will revisit it as soon as our current priorities allow. Your feedback and input are valuable to us, and we are committed to continually improving our product to better meet your needs. We appreciate your ongoing support and look forward to serving you with a more comprehensive product experience in the future.

[macraig]

Adding feedback from @martin-landhi in #681

Is meilisearch Geolocation feature allow have several locations per document?
For example:
I have a companies that have projects in several locations. I want to search for companies that are working on some Location.
Meilisearch allow to create the document for those comapanies with all the locations where they are working and then search?

[Kerollmops]

Hey @furmannodesol 👋

Could you explain what you were doing and that no more works in the v1.0+ versions of Meilisearch by showing an example document update you are sending, please?

I'm not sure to understand. Were you creating a document for each geo-point with the original document id (set as distinct) and doing another search to retrieve the original documents?

Have a nice day 🍡

[furmannodesol]

Hello @Kerollmops custom
I have created indexes with custom primary key like
php artisan scout:index products --key custom_primary_key
WHERE
custom_primary_key is not a database table field.
and it works fine until v0.30 as I can add multiple documents to the index having same ID but with different custom_primary_key
HOWEVER
when I upgraded meilisearch to v1.2, it stops adding documents. I can see X records imported in the command line message but actually there are 0 product added. And only works if I replace custom_primary_key with any other unique field from database like ID or SLUG.

[Kerollmops]

Hey @furmannodesol,

Your issue seems more related to the primary key inference than the multi-geo point feature. We revamped the inference of the primary keys in v1.0.0. You can search for Revamp primary key inference on the release page to understand the difference, but basically, I advise you to define the primary key when creating your index.

If you have questions on how to do it with Laravel Scout, please, ask your question on our Discord to get help.

[mmachatschek]

@furmannodesol please also check out if your laravel/scout version is compatible with your meilisearch version there was an major version bumpt on scout side that added support for meilisearch v1. if you didn't upgrade the laravel/scout package you may experience issues because of incompatibility

[furmannodesol]

Hello @mmachatschek
Thanks for responding here!
In fact I have
Scout: v10.2.3
Meilisearch PHP package: v1.2.1
Meilisearch: v1.2.1

[martin-landhi]

I could made this appling several documents per item with multiple address and applying district id as described above.

Thanks

[garygreen]

Also sharing our interest in this feature. Our use case: we have members for hire on our site who set multiple areas they operate in, and we need a way for those to be discovered by others around a given point.

We tried to move to MeiliSearch from ElasticSearch, but found this key missing feature. The workarounds of having multiple indexes/copies of the data is not workable for us, which is unfortunate. ES, Algolia, RediSearch all support multiple location search, so MeiliSearch is lagging behind in this aspect.

I would love to see this rated higher in priority.

[mhdismail]

Any updates or a possible solution?

It's actually crucial to have support for multiple geo locations. We have a nested array of objects, and we want to find all items if at least of the objects (nested) is nearby by a certain threshold.

[mmachatschek]

For everyone interested in this topic.
Please upvote this discussion with the (upvote) button (if not already done) on the initial message of gmourier. This will help bumping the priority of this feature

[electronick86]

Hello, Is there any plan to develop such a feature in Meilisearch?

At the moment I use the technique of duplicating the documents in the index and using a distinct to only get one document in the result. The downside is that my facets counts are wrong as they don't take the distinct into account.

It would be great to avoid this work of creating several documents.

[macraig]

Hello @electronick86 , we do plan to add this feature in the future as we recognize its value, but at the moment we don't have a specific timeline for its release. In the meantime, your input helps us prioritize it, so we really appreciate you bringing this up.

[soulshake]

Adding our use case: we have a database of historical postcards, each of which can be associated with an arbitrary number of places. Our users want to search for an address and find all postcards related to that place.

Really hoping to see this feature added!

[macraig]

Hi @soulshake, thanks for sharing your use case! As I mentioned in a comment above, we haven't prioritized this feature just yet, but your vote and comment help us prioritize it.