Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

should url decode the search index pattern #4291

Closed
tuziben opened this issue Dec 18, 2023 · 3 comments
Closed

should url decode the search index pattern #4291

tuziben opened this issue Dec 18, 2023 · 3 comments
Assignees
Labels
bug Something isn't working

Comments

@tuziben
Copy link
Collaborator

tuziben commented Dec 18, 2023

Describe the bug
When we use the Elastic or OpenSearch Golang clients to search some indexes in this pattern: gh-archive*, will get a error message: "message":"index ID pattern gh-archive%2Ais invalid. patterns must match the following regular expression:^[a-zA-Z\][a-zA-Z0-9-_\.\]{0,254}$"

Steps to reproduce (if applicable)
Steps to reproduce the behavior:

  1. quickwit built with this hash 8a40a4d143937aacdbc5206d57f3ef7ea26ed9bb
  2. config in quickwit
rest:
  listen_port: 7280
  cors_allow_origins:
    - "*"
  extra_headers:
    x-elastic-product: Elasticsearch

use the Elastic and Opensearch client to query Quickwit
Golang code

       // elastic client 7.x, 8.x
       // opensearch 2.*

 	client, err := elasticsearch.NewClient(elasticsearch.Config{
		Addresses: []string{"http://127.0.0.1:7280/api/v1/_elastic"},
	})

	if err != nil {
		fmt.Println(err)
	}

	index := "gh-archive*"
	queryBody := "{\n    \"query\": {\n        \"bool\": {\n            \"filter\": [\n                {\n                    \"match_phrase\": {\n                        \"id\": {\n                            \"query\": \"32230578641\"\n                        }\n                    }\n                }\n            ]\n        }\n    }\n}"
	res, err := client.Search(
		client.Search.WithIndex(index),
		client.Search.WithBody(strings.NewReader(queryBody)),
	)

	if err != nil {
		fmt.Println(err.Error())
		return
	}

	fmt.Println(res)

        {
          "message": "index ID pattern `gh-archive%2A` is invalid. patterns must match the following regular expression: `^[a-zA-Z\\*][a-zA-Z0-9-_\\.\\*]{0,254}$`"
        }

This issue seems unique to the Golang client; clients in other programming languages like Python and Java don't have this problem. I've looked through the Golang client's code and couldn't find any part where it's using URL encoding. But, based on the output we're seeing, it looks like the Golang client is URL encoding the asterisk (*).

So, to fix this, I think Quickwit should decode the URLs in all its search indices to get them back to their original format.

pub fn validate_index_id_pattern(pattern: &str) -> anyhow::Result<()> {

@tuziben tuziben added the bug Something isn't working label Dec 18, 2023
@fmassot
Copy link
Contributor

fmassot commented Dec 18, 2023

@tuziben can you give the quickwit version?

@tuziben
Copy link
Collaborator Author

tuziben commented Dec 18, 2023

  1. quickwit built with this hash 8a40a4d143937aacdbc5206d57f3ef7ea26ed9bb

The main branch or any other version can reproduce this issue

@fmassot
Copy link
Contributor

fmassot commented Dec 21, 2023

Fixed by #4292

@fmassot fmassot closed this as completed Dec 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
No open projects
Status: Done
Development

No branches or pull requests

2 participants