Skip to content

Commit

Permalink
[raz][s3] Fix encoding for special chars in directory name listing 10…
Browse files Browse the repository at this point in the history
…00+ keys (#3677)

- If a directory has 1000+ keys, then when listing the objects using the GET request, it also required a marker header to fetch more keys.
- This marker headers needs to be unquoted when sent to RAZ but left unchanged when sent to S3 to have proper access on the path from RAZ side.
  • Loading branch information
Harshg999 authored Apr 1, 2024
1 parent 47db78a commit 5f67c2c
Showing 1 changed file with 5 additions and 4 deletions.
9 changes: 5 additions & 4 deletions desktop/core/src/desktop/lib/raz/raz_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -256,11 +256,12 @@ def _make_s3_request(self, request_data, request_headers, method, params, header

# In GET operations with non-ascii chars, only the non-ascii part is URL encoded.
# We need to unquote the path fully before making a signed request for RAZ.
if method == 'GET' and 'prefix' in url_params and '%' in url_params['prefix']:
if sys.version_info[0] < 3 and isinstance(url_params['prefix'], unicode):
url_params['prefix'] = url_params['prefix'].encode()
if method == 'GET':
if url_params.get('prefix') and '%' in url_params['prefix']:
url_params['prefix'] = lib_urlunquote(url_params['prefix'])

url_params['prefix'] = lib_urlunquote(url_params['prefix'])
if url_params.get('marker') and '%' in url_params['marker']:
url_params['marker'] = lib_urlunquote(url_params['marker'])

allparams = [raz_signer.StringListStringMapProto(key=key, value=[val]) for key, val in url_params.items()]
allparams.extend([raz_signer.StringListStringMapProto(key=key, value=[val]) for key, val in params.items()])
Expand Down

0 comments on commit 5f67c2c

Please sign in to comment.