🔎 Advanced search

[jancborchardt]

How to use GitHub

• Please use the 👍 reaction to show that you are interested into the same feature.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.

Is your feature request related to a problem? Please describe.
The current unified search dialog in the top right is quite small and compact.

Describe the solution you'd like
For the powerful things it supports, it would make more sense to have a bit more ways of filtering. E.g. having a left sidebar with filtering which app to focus on, or some detailed selections on owner of file, or date range, or size.

Edit by @sorbaugh (Last check: 04-09-2023): Since the requirements have shifted somewhat over the course of the discussion (and will likely continue to do so), here is a breakdown of the current state in tasks to minimize confusion. I will make an effort to keep this up to date:

Mockup (original comment -> #39162 (comment))

Frontend Task breakdown:

• 🔎 AS - Move search into a modal dialog and add Layout + Vue Components for filtering #40238
• 🔎 AS - Leverage existing filter logic to limit search in specific apps (e.g. in:files) #40239
• 🔎 AS - "Places" dropdown should be converted to show 3 apps as chips, then a 3-dot menu, otherwise it is not as discoverable #40240
• 🔎 AS - Implement results listing. The "Load more results" should continue to behave like now as loading the next few results in place. #40237

Backend Task breakdown:

• 🔎 AS - For files: Check for existing DAV Search functionality (date search, people/owner) #40246
• 🔎 AS - Implement new, extended search provider API within files to support the new filter criteria #40249
• 🔎 AS - Backward compatibility to be implemented if not give by design #40250

---

Date
For things like date range, we have to make sure it works across different apps like Files, Calendar, Contacts, Talk, Mail etc with different types of dates, so we need to define which date it is for the relevant type:

• Files: modification date (probably more relevant than creation date)
• Calendar: event date (probably more relevant than creation date) 🔎 Allow event date and attendee search in advanced search #41345
• Deck card: due date (probably more relevant than creation date, if it exists)
• Talk: message date
• Mail: message date 🔎 Allow message date and recipient search in advanced search mail#8864
• Contacts: ? birthday
• General fallback would be "Modified date"
• etc. …

People
The person dropdown could pick from what the "People menu" in the top right also lists, which is both Nextcloud accounts and Contacts. This would then e.g. for Mail also cover the "Recently contacted" address book which is nice.

• Files: owners and sharees
• Calendar: participants 🔎 Allow event date and attendee search in advanced search #41345
• Deck card: assignee
• Talk: message poster and/or person replied to
• Mail: sender and recipients 🔎 Allow message date and recipient search in advanced search mail#8864
• Contacts: the contact itself, possibly manager also
• General fallback would be sharing

We need mockups for this @nextcloud/designers :)

[schiessle]

Just as a reference, this is how the search view of the fulltext search looks:

It allows you already to filter between Deck boards, files and file extensions. I think it make sense to have one advanced search in the future which does also full text search if enabled. Maybe even as a option/checkbox to decide if the user wants to search for filenames/folders only, for content only or for both.

[marcoambrosini]

I did some research and interviews on this and I think that while there's room for UX improvements in the global search, we should not make it more complex than it already is. A few reasons:

• Different searchable items require different types of filters, so it would be better to have a tailor-made solution for each app instead of trying to make something that works for every searchable item in the same UI.
  For example, if I'm looking for a contact I might be interested in knowing whether it belongs to a group, while if I'm looking for a pdf I would want to filter by file type. I think that trying to reconcile the complexity that will emerge from all use-cases in the same UI will end up being confusing.

• Users generally report that if they know what they're looking for, they search it from within the app.

• This also seems to be the trend in our competitors' solutions: global searches are simple and provide quick and relevant results, in-app searches allow fine-tuning and more complex filtering.

---

While conducting interviews I found out that most people felt frustrated with global search and never or rarely use it. They resort to syncing their files and emails to their OS and search there instead.

A few recurring themes:

• Users don't trust the search: the lack of full-text search and fuzzy search renders the functionality unreliable and frustrating for most users because it doesn't return the expected amount of results.
  One user compared their local OS search with the global search with the same keyword. The local OS search yielded 40 results while the global search only 2;

• Users are unhappy with the performance. It takes a long time to get the results.

• Users think that the search results are not relevant to them, and wish the search was "smarter".

• The interface is "messy".

---

So it seems that the feature is there but it's not very useful to people. To change this I think we need to tackle the issue from 2 angles.

1. Layout. there are a lot of little improvements that would make the search experience much better. A few ideas:

• Better display of the search filters that are already there (not in a menu);
• Different components for different types of results. Now all the search result components are 70px tall and not many of them fit in the view;
• Modal or fullscreen view to accomodate more results;
• Lazy loading while scrolling instead of having to click on "More results" if a filter is applied;
• Relevance: Think about a set of rules on what to prioritise for each user? For example one user complained that a lot of results coming from shares took priority over their own files.
• Previews if technically possible? (this emerged from a user that values the previews in their OS search and I think it'd be useful to have);
  On top of global search improvements, we should also take a step back and open a meta issue on search in which we track what's going on in each app and bring it all up to par. What do you think @nextcloud/designers?

2. Reliability and performance.
   For the full-text and fuzziness side of things which is a major issue, is that something we tackle @AndyScherzinger? Looping in @juliushaertl too as he had some technical advice about how indexing could be implemented. From a UX perspective I think that this is something that most users would expect and it'd be super useful.

[AndyScherzinger]

For the full-text and fuzziness side of things which is a major issue, is that something we tackle @AndyScherzinger?

No. Full-text search is something we have already given the FTS app is actually installed and set up (usually not the case since it requires further components, i.e. elastic search). Fuzzy search when don't have at all and currently isn't on the roadmap.

[marcoambrosini]

Here's a proposal on the layout changes mentioned above and incorporating some feedback from the design team. You can also open the view on penpot by clicking on this

---

@nextcloud/designers I've been thinking about shortcuts and It's just too confusing that depending on where you are in Nextcloud, the global search appears only sometimes upon pressing ctrl + f. This kind of unpredictability just makes users feel more insecure about the software and adds to the cognitive load.

These shortcuts are for advanced users anyway, so we only do a favour to advanced users if we separate the two and have consistent behaviour upon pressing ctrl+f. Less proficient users will click on the search button either way.
I would reserve ctrl + f for in-app-search and suggest ctrl + k for the global-search. ctrl + k is becoming a widespread pattern and many applications use it already for broader searches. Slack, Telegram, Gnome, Github, Discord to name a few.

[juliusknorr]

Looks very nice, one small comment it might be good to have a reset filter to quickly get back to all results.

Has there been any thought around how this would impact the search icon in the to right? Shall it just open the modal then or is the modal in addition to the existing popover?

[marcoambrosini]

one small comment it might be good to have a reset filter to quickly get back to all results.

I thought this could be in the menu. What do you think @juliushaertl?

Shall it just open the modal

Yes :)

[juliusknorr]

I thought this could be in the menu. What do you think @juliushaertl?

I think a quick reset option outside might be nice, at least when more than one filter option is selected (to reduce the amount of clicks needed, but either way would be fine I guess.

[gennaios]

I think full screen results, perhaps from some link in the box that says show all, is ideal. Some have thousands of files or many contacts.

As for search performance, all supported databases have built-in FTS. Unlike the current FTS plugins, one specifies specific columns with which to create an FTS index. All large databases use such or something external like Sphinx. Such would offer fuzzy search, allow terms to be specified in any order, handle diacritics, etc. Performance would then be solved and there would be no need for lazy loading.

So for example with contacts, one can search any combination of terms in any desired fields such as name address city etc.; with files name path tag etc.

Large databases do not use like queries. They all use some form of FTS. As NC supports multiple databases, it merely takes more work to support them all.

[AndyScherzinger]

Mockup looks nice and provides a first set of options to narrow down the search by limiting the apps to be respected when searching for relevant items.

I discussed the feature with Frank and the initial idea is to basically improve the UI side of search but still base search on what we have offered by the backend as of today. Afaik we already have ways to limit the search scope yet it is hard to discover and the goal would be to raise such scope-limiting capabilities to the UI in a discoverable way.

cc @jancborchardt @marcoambrosini

@karlitschek what do you think about #39162 (comment) as a first step? It would limit the search scope where the first limiting step would then always be the app. This could be a trade off that is okay.

[jancborchardt]

@marcoambrosini we talked about this again and since in-app search will likely still be further out, we should ideally have some more filtering in the global search already.

For reference here a screenshot from Google Drive, so we could do the bar of selections like:

• Apps (instead of type)
• People
• Date (cause it’s not always modified)
• Title only (for fulltext yes/no)
• Maybe something else special that we can think about?

Type	People	Modified	Location	To do / state

[florom]

Hi Devs,

I would like to add the downsides of the current Global search here, since I got a link in the forum thanks to jtr.
In priority order.

1. The search Window is to small - I barely see the results even on a big screen
   • Small Window in the side
   • Symbols too big
   • For Example in Files: Filename in next line path (uses too much space) the screen is usually has more width than height.
2. It does not load all results - I always have to load more results this is very very uncomfortable
3. There is no way to sort anything like date, size, format
4. It searches everywhere - usually I want to search the section I am in at the moment (It should first search in the section you are in and then if you like everywhere else.
5. If you search in Files there is no recursive Search (only the level you are in which is usually not where you find the file anyway)

If you place more buttons in this tiny window it will even get worse IMHO. The presentation here is 250% bigger then it will be implemented. Even in the mockup only 4! results (totall 8) are visible. Usually it is not so clear what filename it was so there should be at least 20 results. @jancborchardt @AndyScherzinger

In my opinion this should be a a whole new app window (Global Search) and the Search in the menu only considers the current open app.

Another thing I do not understand is why contacts have an additionally search button in the menu bar.
The contacts quick search is amazing and and I see no benefits in having it one-click available when it uses up so much space all the time.

Thanks for reading.

[marcoambrosini]

After design call it has been decided that we're going to use multiple selects, some of which with tags. Much like google drive search posted above. Here's the updated mockup.

Shared components used for the filtering:

• NcSelect with multi option and wrapped for "places"
• NcSelect single for "modified" options being (today, this week, this month, this year and custom). Clicking custom should open a small dialog (mockup on the right) that allows us to set the after and before fields.
• NcSelect for one single user at a time for "People", there's an example of this in the Vue documentation
• Tertiary NcButton for the "Title only" option. This is a stateful button and the pressed state can be toggled via prop.
• To the right we have another tertiary NcButton that allows us to clear all filters at once.

cc @nextcloud/designers for any further comments

[gennaios]

An old thread that was closed in favor of this one, as well as several others, has many comments about improvement to search itself. As it seems no one or few reply to such, perhaps somehow the topic is being avoided. Yes I think FTS is the only method. It perhaps merely would take a few weeks per database to figure it out. If someone just said yes we need to figure it out, perhaps others including myself would help.

Any kind of search results showing some limited number out of total, isn't it really best and perhaps only truly useful in such a case to show the most relevant, and then show others with options to sort by for example name, city, date created/modified, etc.?

[jancborchardt]

@Altahrim @marcoambrosini any help needed at this stage, or what is the status? I can’t see a linked pull request

[AndyScherzinger]

@jancborchardt there is none, also had to be dropped for 27.1

Research state is: that i.e. in:files is supported i.e. to limit it to just search in files but currently there is no other limiting/filtering criteria present for unified search to limit it's search scope for files. so anything we want would need to be build (hence v28).

Si for example for files the statement is that anything we can do in dav search should be relatively easy to be added to search (for files)

[nimishavijay]

Notes form design review with @jancborchardt @fenn-cs @sorbaugh @szaimen

Overall

• Heading: "Global search"
• Close button missing for the modal
• "Global search" --> "Search term..."

Filter chips (Files, Talk, etc)

• style it like NcUserBubble component
• Border radius pill
• Background: --color-primary-light or grey
• Smaller close icon

Search results

• should be like NcListItem component
• remove shadow
• Show "Search in [appname]" only for the apps that have an in-app search. Files doesn't have an in-app search yet, so don't show "Search in Files"
• App name in "Search in [appname]" should have capital first letter as it is a proper noun ("Search in Talk", "Search in Mail" instead of "Search in talk")

Places

• Wording: "Apps and settings"
• Style should be NcButton
• Clicking on button opens an NcActionMenu with a list of the available apps, similar to app list in mobile view header bar

People filter

• Needs label "People"
• Use component NcSelect with userSelect prop

Date and time filter

• NcButton component
• Button name "Modified"
• On click, open action menu with predefined date options and a date range picker
• Use native date picker for date range

[marcoambrosini]

@jancborchardt @nimishavijay @szaimen can we drop this pattern of selects within small tooltips? It's very clumsy. Everyone uses a dialog here.