Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement text search #123

Merged
merged 9 commits into from
Nov 16, 2021
Merged

Implement text search #123

merged 9 commits into from
Nov 16, 2021

Conversation

marvin-roesch
Copy link
Collaborator

@marvin-roesch marvin-roesch commented Oct 19, 2021

This adds a new button to the top right which turns into a search field when clicked. Search is powered by lunr.js and indices for all languages are created at build time so they are readily available in production.

I've tried to make foreign language search work as well as possible already, but input from our translators would be appreciated. @lianzhao and @botanicaxu, I'd especially like your feedback on the Chinese search!

Changes can be viewed at this preview.

Closes #99

@lianzhao
Copy link
Contributor

Great work @PaleoCrafter !!

I have tried your online preview in Chinese. It works in some cases, but failed in others😂

It's hard to tokenize Chinese. I don't have much experience in both Chinese tokenization and lunr.js. It seems that lunr.js has official Chinese support? I'll try https://github.com/MihaiValentin/lunr-languages/blob/master/lunr.zh.js in my local dev env .

@marvin-roesch
Copy link
Collaborator Author

FYI @lianzhao I'll merge the branch for now and we can work on improving the Chinese tokenization as we go, since I expect that to be an ongoing effort.

@marvin-roesch marvin-roesch merged commit 5c2b218 into row-ds-update Nov 16, 2021
@lianzhao
Copy link
Contributor

Hello @PaleoCrafter, I did some research about tokenize Chinese with lunr today, and found a bug in lunr.zh.js which prevent me from tokenizing Chinese correctly with custom dictionary.

I've send a PR to lunr team. With that fix and custom dict, I can make the search work on my local dev env.

@marvin-roesch marvin-roesch deleted the feature/text-search branch November 23, 2021 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants