Skip to content

Commit

Permalink
Keep all uploaded zip content accessible
Browse files Browse the repository at this point in the history
iterdir() is platform dependent, that is the order of the returned items
may be different on different platforms. In cases where a zip file
contains multiple base_file candidates it will be overridden by the last
one found (which varies on different platforms).

Also, different files with the same extension (file1.csv, file2.csv) will not
be accessible from file_paths as they get overridden, too.

The fix enumerates all files to make them accessible from file_paths.
  • Loading branch information
ridoo committed Oct 13, 2023
1 parent 43188af commit 7869104
Showing 1 changed file with 22 additions and 6 deletions.
28 changes: 22 additions & 6 deletions geonode/storage/data_retriever.py
Original file line number Diff line number Diff line change
Expand Up @@ -205,17 +205,33 @@ def _unzip(self, zip_name: str) -> Mapping:
at the end the zip is deleted
"""
zip_file = self.file_paths["base_file"]
the_zip = zipfile.ZipFile(zip_file, allowZip64=True)
the_zip.extractall(self.temporary_folder)
with zipfile.ZipFile(zip_file, allowZip64=True) as the_zip:
the_zip.extractall(self.temporary_folder)

available_choices = get_allowed_extensions()
not_main_files = ["xml", "sld", "zip", "kmz"]
base_file_choices = [x for x in available_choices if x not in not_main_files]
for _file in Path(self.temporary_folder).iterdir():
if any([_file.name.endswith(_ext) for _ext in base_file_choices]):
self.file_paths["base_file"] = Path(str(_file))
elif not zipfile.is_zipfile(str(_file)):
if not zipfile.is_zipfile(str(_file)):
if any([_file.name.endswith(_ext) for _ext in base_file_choices]):
self.file_paths["base_file"] = Path(str(_file))
ext = _file.name.split(".")[-1]
self.file_paths[f"{ext}_file"] = Path(str(_file))
if f"{ext}_file" in self.file_paths:
existing = self.file_paths[f"{ext}_file"]
self.file_paths[f"{ext}_file"] = [
Path(str(_file)),
*(existing if type(existing) == list else [existing]),
]
else:
self.file_paths[f"{ext}_file"] = Path(str(_file))

tmp = self.file_paths.copy()
for key, value in self.file_paths.items():
if type(value) == list:
for index, file_path in enumerate(value):
n = f"{key}_{index}" if index > 0 else key
tmp[n] = file_path
self.file_paths = tmp

# remiving the zip file
os.remove(zip_name)
Expand Down

0 comments on commit 7869104

Please sign in to comment.