Problem in signing items for collection modis-13A1-061

[OldLipe]

Hi,

We are experiencing an issue generating tokens for the modis-13A1-061 collection.

The rstac package uses the https://planetarycomputer.microsoft.com/api/sas/v1/token endpoint to sign the items in a collection.

library(rstac)
library(httr)
library(magrittr)

item <- stac("https://planetarycomputer.microsoft.com/api/stac/v1") %>%
  stac_search(collections = "modis-13A1-061", limit = 1) %>%
  post_request()

item_signed <- items_sign(item, sign_fn = sign_planetary_computer())
httr::HEAD(item_signed$features[[1]]$assets$`500m_16_days_EVI`$href)
# status code: 403

As shown in the example below, only the https://planetarycomputer.microsoft.com/api/sas/v1/sign endpoint is working:

asset_url <- item$features[[1]]$assets$`500m_16_days_EVI`$href
# https://modiseuwest.blob.core.windows.net/modis-061-cogs/MYD13A1/08/05/2022297/MYD13A1.A2022297.h08v05.061.2022315120509_500m_16_days_EVI.tif

# Sign by collection (same as rstac)
collection_token_url <- "https://planetarycomputer.microsoft.com/api/sas/v1/token/modis-13A1-061"
token <- httr::content(httr::GET(collection_token_url))[["token"]]
httr::HEAD(url =  paste0(asset_url, "?", token))
# status code: 403

# Sign by asset
asset_token_url <- paste0("https://planetarycomputer.microsoft.com/api/sas/v1/sign/?href=", asset_url)
httr::HEAD(url = httr::content(httr::GET(asset_token_url))[["href"]])
# status code: 200

Any idea what is going on?

[TomAugspurger]

My R is a bit rusty, but can you confirm that this

collection_token_url <- "https://planetarycomputer.microsoft.com/api/sas/v1/token/modis-13A1-061"
token <- httr::content(httr::GET(collection_token_url))[["token"]]

is roughly equivalent to 

```bash
curl "https://planetarycomputer.microsoft.com/api/sas/v1/token/modis-13A1-061"

and you were seeing a 403 error? At the moment, I'm getting an OK 200 response at that endpoint, so maybe try again?

[cboettig]

I think they should be equivalent. Do you expect them to have the same signature though? I get back slightly different tokens:

collection_token_url <- "https://planetarycomputer.microsoft.com/api/sas/v1/token/modis-13A1-061"
token <- httr::content(httr::GET(collection_token_url))[["token"]]
out <- processx::run("curl", "https://planetarycomputer.microsoft.com/api/sas/v1/token/modis-13A1-061")
token2 <- jsonlite::fromJSON(out$stdout)$token
testthat::expect_equal(token, token2)
#> Error: `token` not equal to `token2`.
#> 1/1 mismatches
#> x[1]: "st=2022-11-14T17%3A03%3A11Z&se=2022-11-15T17%3A48%3A11Z&sp=rl&sv=2021-06-
#> x[1]: 08&sr=c&skoid=c85c15d6-d1ae-42d4-af60-e2ca0f81359b&sktid=72f988bf-86f1-41a
#> x[1]: f-91ab-2d7cd011db47&skt=2022-11-15T10%3A32%3A16Z&ske=2022-11-22T10%3A32%3A
#> x[1]: 16Z&sks=b&skv=2021-06-08&sig=y9KgIskYPRMGMzFGbHh258glxyQ14wceQ9A1UN2WFUE%3
#> x[1]: D"
#> y[1]: "st=2022-11-14T17%3A03%3A12Z&se=2022-11-15T17%3A48%3A12Z&sp=rl&sv=2021-06-
#> y[1]: 08&sr=c&skoid=c85c15d6-d1ae-42d4-af60-e2ca0f81359b&sktid=72f988bf-86f1-41a
#> y[1]: f-91ab-2d7cd011db47&skt=2022-11-14T16%3A47%3A00Z&ske=2022-11-21T16%3A47%3A
#> y[1]: 00Z&sks=b&skv=2021-06-08&sig=tK%2B9Z42c5u5Xq/sKlm5yB1X/ixiOpbIvYzVE%2BG4Ub
#> y[1]: Mo%3D"

^{Created on 2022-11-15 with reprex v2.0.2}

Note the ske=2022-11-22T10 part is different in each at least.

[OldLipe]

I think the issue is in the signature of the modis-13A1-061 collection.
It seems this collection holds assets from other collections, e.g:

curl "https://planetarycomputer.microsoft.com/api/stac/v1/search?collections=modis-13A1-061&limit=1" | grep "href" 

The "href" asset is from the modis-061 collection

"href":"https://modiseuwest.blob.core.windows.net/modis-061/MYD13A1/35/10/2022297/MYD13A1.A2022297.h35v10.061.2022315162451.hdf"

The asset: "500m_16_days_red_reflectance" is from the modis-061-cogs collection

"href":"https://modiseuwest.blob.core.windows.net/modis-061-cogs/MYD13A1/35/10/2022297/MYD13A1.A2022297.h35v10.061.2022315162451_500m_16_days_red_reflectance.tif"

Therefore, the subscription for the modis-061 collection only supports the assets belonging to this collection:

# get token for collection modis-13A1-061
curl "https://planetarycomputer.microsoft.com/api/sas/v1/token/modis-13A1-061"

# status 200
curl -I "https://modiseuwest.blob.core.windows.net/modis-061/MYD13A1/35/10/2022297/MYD13A1.A2022297.h35v10.061.2022315162451.hdf?st=..."

# status 403
curl -I "https://modiseuwest.blob.core.windows.net/modis-061-cogs/MYD13A1/35/10/2022297/MYD13A1.A2022297.h35v10.061.2022315162451_500m_16_days_red_reflectance.tif?st=..."

[TomAugspurger]

Just a few notes on terminology:

• STAC collection: modis-13A1-061
• Azure storage account: modiseuwest
• Azure Blob Storage containers: modis-061-cogs, modis-061. Storage containers are nested under storage accounts.

Some STAC collections, like modis-13A1-061, contain assets from multiple storage containers. SAS tokens, however, are tied to a specific storage account. When you make a request to https://planetarycomputer.microsoft.com/api/sas/v1/token/<collection>, it uses the msft:storage_account and msft:container fields from the STAC collection to get the storage account / container to generate a SAS token for. For https://planetarycomputer.microsoft.com/api/stac/v1/collections/modis-13A1-061, that's modiseuwest/modis-061, which explains why the request for the HDF5 file (in modis-061) succeeds, while the request for the COG (in modis-061-cogs) fails.

That's why in the planetary-computer Python package, the signing functions all end up working with the individual asset HREF, parsing the storage account / container, and using the /api/sas/v1/token/<account>/<container> endpoint: https://github.com/microsoft/planetary-computer-sdk-for-python/blob/7970d3bda89f4dbd052516cd1df52b00b658d855/planetary_computer/sas.py#L155-L157.

[OldLipe]

Thanks for the clarification, Tom! I will follow the same idea in the rstac signature function.