Slow search on rosindex

[mortenfyhn]

I hope this is the right spot for this kind of feedback.

When I search on index.ros.org, it takes around ten seconds to get the search results. The old index simply listed everything on one page, so I could use my browser's search function, which is immediate. How about adding a "show all" button on the main index page to get that possibility back? The default can still be 50 elements per page.

[hidmic]

@mortenfyhn does it always take ~10s? Or just the first time it loads? Most of the time used to and should be spent pulling the lunr.js-based index, not searching. Ten seconds is way too much even for that anyways.

[mortenfyhn]

Seems to be first time only, after that it takes roughly 2.5 s. I'm using a Latitude 7490 laptop on a 100 Mbps line, so that should be okay.

[hidmic]

Yeah, it's the time it takes to pull the lunr.js index. We can further split it in chunks, but I don't how much that'd improve things. @tfoote any ideas?

[tfoote]

If anything I think more chunking will slow things down. It looks like we're actually spending more time waiting for chunks than downloading them, and they're pretty small at the moment. Can we parallelize the downloads? Or can we background load the chunks while the searchbox is on the screen or at least as soon as the user starts typing in the search box?

Here's the network timeline I see getting the index files. That's only about half the index files. The rest are even smaller and load sequentially quicker.

[tfoote]

We could also try out requestIdleCallback to background load it.

[ros-discourse]

This issue has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/growing-issue-with-ros-documentation/36075/44

[mortenfyhn]

I'll update:

First of all: This is faster now. I'm not sure if it's faster devices and internet connection or the website is faster or both.

Within a browser tab, it seems to cache perfectly so subsequent searches are very fast. If I open a new tab, I get the loading again. But the loading is much more reasonable than what I experienced back in 2019.

Might be worth checking on a more modest computer, and with a browser that's guaranteed to not already have stuff cached.

rosindex-speed.mp4