Loading COG from private S3 repository

[glemoine62]

I have been struggling a bit trying to load COGs from a private S3 repository. Public ones are straightforward (you can use leafmap.add_cog_layer). But I did not find a working example for a private one. I think the problem is somewhat similar to #733, i.e. the S3 credentials and other parameters do not propagate to .add_cog_layer (and titiler?) as was the case for the s3* functions (now resolved).

I managed to work out the following, though, based on this example:

import leafmap
import json
import boto3

import rasterio as rio
from rasterio.session import AWSSession

with open('data/s3_config.json', 'r') as f:
      config = json.load(f)

# Set up a session with the credentials (NB other parameters are used later on)
session = boto3.Session(aws_access_key_id=config['s3']['access_key'], aws_secret_access_key=config['s3']['secret_key'])

rio_env = rio.Env(AWSSession(session), 
                  AWS_S3_ENDPOINT=config['s3']['host'],
                  bucket=config['s3']['bucket'],
                  AWS_HTTPS='YES', 
                  AWS_VIRTUAL_HOSTING=False)
rio_env.__enter__()

myPrivateCog = f"/vsis3/{config['s3']['bucket']}/subdir/myPrivateCog.tif"

m = leafmap.Map()
m.add_raster(myPrivateCog)

m

# Run in a separate cell
rio_env.__exit__()

data/s3_config.json stores the relevant parameters (my use case is for a Swift S3 repository).

This works, because add_raster uses get_local_tile_layer which ultimately calls rasterio, that will run in the rio.Env
I somehow phantom this should be working for add_cog_layer as well, but did not get it to work.

Hopefully useful to others as well.

[giswqs]

The add_cog_layer() only works for public S3 bucket. For private bucket, use add_raster. See this example.

import os
os.environ["AWS_ACCESS_KEY_ID"] = "YOUR AWS ACCESS ID HERE"
os.environ["AWS_SECRET_ACCESS_KEY"] = "YOUR AWS ACCESS KEY HERE"
os.environ["AWS_REQUEST_PAYER"] = "requester"

m = leafmap.Map(center=[34.979166, -84.920496], zoom=14)
m.add_basemap("Satellite")
src = "s3://naip-analytic/tn/2021/60cm/rgbir_cog/34084/m_3408401_ne_16_060_20210404.tif"
m.add_raster(src, layer_name="NAIP")
m