Enhancing download_single_file to Handle Non-Existent Files

[eggry]

THU-Cloud-Downloader/thu_cloud_download.py

Lines 89 to 95 in 8b610c4

	def download_single_file(url: str, fname: str, pbar: tqdm):
	global sess
	resp = sess.get(url, stream=True)
	with open(fname, 'wb') as file:
	for data in resp.iter_content(chunk_size=1024):
	size = file.write(data)
	pbar.update(size)

When the download_single_file function encounters a URL that points to a non-existent file, it currently does not raise an error and proceeds as if downloading a regular file. As a result, the downloaded file is merely a webpage saying “文件不存在”.

A notable challenge is that the server still responds with a status code of 200 instead of 404 in such case. An ugly-but-effective way solution involves checking for redirection in resp: an exist file will undergo a redirection from files/?dl=1 to seafhttp/files/. In contrast, a Not Found response is served directly from files/?dl=1 without any redirection.

def download_single_file(url: str, fname: str, pbar: tqdm):
    global sess
    resp = sess.get(url, stream=True)
    if not resp.history:
        raise ValueError("File may non-exist!")
    with open(fname, 'wb') as file:
        for data in resp.iter_content(chunk_size=1024):
            size = file.write(data)
            pbar.update(size)

Wonder if there are more elegant solutions...