S3 as source for GeoTIFF is slow.

[meteoDaniel]

I have uploaded a 1MB GeoTIFF into a S3 bucket.
I am wondering now because it is much slower compared to classic server hosting.

A few questions come to my mind:
Do you know why S3 is slow?
What s the best way of serving tiles? Is it possible to run titiler and use local GeoTIFF files?

Thanks a lot :)

[vincentsarago]

@meteoDaniel I'm not sure I understand the questions 🤷‍♂️

Do you know why S3 is slow?

S3 is pretty fast, if creating tiles using TiTiler is slow, it might due to your GDAL configuration . @kylebarron started a doc over https://github.com/developmentseed/titiler/blob/0a5288de30845865256a0124ed233f918812aa78/docs/concepts/performance_tuning.md to explain what GDAL env can impact TiTiler

What s the best way of serving tiles?

specifics ?

Is it possible to run titiler and use local GeoTIFF files?

yes, titiler uses rasterio which use GDAL. if gdal have access to your GeoTIFF, TiTiler will work fine.

[meteoDaniel]

Thanks for introducing the performance tuning guide. Nevertheless it did not solved the problem.S3 is still significantly slower. I have adapted the titiler server but nothing changed:

docker run --name titiler -p 8000:8000 -e PORT=8000 -e WORKERS_PER_CORE=1 -e GDAL_HTTP_MERGE_CONSECUTIVE_RANGES=YES -e GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR -e  --rm developmentseed/titiler

I have called the following tile layers:

http://0.0.0.0:8000/cog/tiles/{z}/{x}/{y}.png?url=https://meteointelligence.com/bilder/202104151650.tif
http://0.0.0.0:8000/cog/tiles/{z}/{x}/{y}.png?url=https://meteointelligence-radar-images.s3.eu-central-1.amazonaws.com/202104151655.tif

Please note that the bucket contains at least 1 file and the directory on meteointelligence.com contains much more files.

And what I was asking for is: What kind of setup do you recommend for a tile server?

[meteoDaniel]

PS: Using https://api.cogeo.xyz S3 is faster. So it might be that there are some wrong configurations.

Is https://api.cogeo.xyz hosted on AWS?

[vincentsarago]

yes api.cogeo.xyz is hosted on AWS (us-east-1).

Not sure why it's slow on your laptop 🤷‍♂️