CoCoKeyprovider | Support plug-ins of KMS

[Xynnn007]

Background

Now CoCo KeyProvider has the following functions:

• Generateing KEK, its KID and registering the KEK into KBS
• Encrypting the image (concretely, encrypting LEK) with the KEK

It's important that the KEK is either imported from a local file, or randomly generated inside the CoCoKeyprovider.

What we want

Different KMSes have already given a good practical implementation for managing cryptographic keys. As mentioned in confidential-containers/trustee#37, we should think about bringing in KMS plug-in for the whole lifetime of KEKs.

Initial Design Plan

Image Encryption

1. Interface definition

For CoCo Keyprovider, there are two basic functions:

1. A symmetric key (KEK) to encrypt the LEK sent by skopeo.
2. If needed, register the KEK into KBS.

1 requires keyprovider to generate a new (or fetch an old) datakey from KMS.
2 requires keyprovider to deliver "something" that presents the key into KBS. The "something" might not be the KEK itself, but with this "something", KBS can easily recover the original plaintext of the KEK.

Based on the two observations, we propose a unified trait Encryptor that refers to a specific key entity, such as a DataKey in a KMS.

#[async_trait::async_trait]
pub trait Encryptor {
    /// Use this key to encrypt the given data slice, and res
    async fn encrypt(&mut self, data: &[u8]) -> Result<Vec<u8>>;
    
    /// Export the blob that can be used to fetch the plaintext of the key
    /// on the KBS side
    async fn export_keyblob(&mut self) -> Result<Vec<u8>>;
}

Image Decryption

The decryption operation needs more discussion.

Now, we implement the decryption inside attestation-agent with manual crypto calculations. With KMS, there might be two different design ways

Way 1

The same as current implementation, s.t. decryption operation occurs inside attestation-agent. The data-flow will as the diagram

Pro: We do not need to change any APIs of KBS which CC-KBC calls. Concretely, now KBS just provides an API for CC-KBC to "fetch the resource" rather than "do decryption".

Con: (probably a con?) encryption and decryption are in different componants -- encryption in keyprovider but decryption in KBS.

Way 2

This way, we integrate both encryption and decryption inside KBS, and make coco-keyprovider an proxy for KBS.

Pro: encryption and decryption are in same componants -- KBS

Con: we need to open another HTTP API from KBS, and probably not related to the "confidential resource storage" functionality of KBS.

To avoid make KBS more complicated, I prefer way 1. And to make future work easy, I suggest make KMS plug-ins a single library like what this PR is doing. In this way we can import the library both in CoCo Keyprovider and KBS side.

In this way the Encryptor should be extended with the following functions

#[async_trait::async_trait]
pub trait Encryptor {
     // ... defined before

     /// Create self from a given blob
     async fn from_blob(blob: &[u8]) -> Box<Self>;

     /// Export the plaintext of the Key
     async fn plaintext(&self) -> Result<Vec<u8>>
}

Related Issue

confidential-containers/trustee#37

[fitzthum]

One thing to think about is whether we ever want to support "remote" attestation, meaning that the key never leaves the KMS. Option 2 fits better with this but it would be more complicated to implement.

[Xynnn007]

One thing to think about is whether we ever want to support "remote" attestation, meaning that the key never leaves the KMS. Option 2 fits better with this but it would be more complicated to implement.

You're right. There is a trade off between the thought "Never-Leave-KMS" and the simplicity of the design.

My idea is that the key can leave KMS.

Now commercial KMSes always give two kinds of abstrations for keys: Customer Master Key (CMK) and Data Encryption Key (DEK or Datakey).

1. CMK: Never leaves KMS, and could be either asymmetric or symmetric. The KMS exposes encrypt and decrypt API. Usually for little piece of data.
2. DEK: Can leave KMS, usually symmetric. The KMS always exposes generate_data_key that returns a plaintext and a ciphertext of a newly generated DEK, where the ciphertext is encrypted by a corresponding CMK. Usually to encrypt large pieces of data.

The DEK can work as the KEK, in this way "Never-Leave-KMS" is not ensured. The reasons are:

1. Although plaintexts of the KEK will be exposed outside the KMS, they can be safe:
   a. Encryption phase: the componants that get in touch with the plaintext are all on the tenant side, by which the attackers like CSP cannot steal. We can use programming best practics to flush the memory of the plaintext keys after using them.
   b. Decryption phase: although the plaintext could be delivered to CSP, the confidentiality is ensured by the remote attestation. As the entire channel where the plaintext appears are encrypted and not readable to CSP (the Secure Channel between KBS and AA and AA itself is inside a attested TEE).
2. As mentioned, if CMK works as the KEK, it is more complicated to implement. Concretely, the encrypted LEK blob (PLBCO) should be delivered from cc-kbc (attestation-agent) to KBS, and then KBS delivers the blob to KMS to decrypt. This require a new API for KBS, s.t. decrypt data by given key id.
3. DEK allows us to do the decryption outside KMS, while its confidentiality is protected by that the plaintext is only shown in trusted realm and is disposable. In this way, we encrypt the image using the plaintext of DEK and then eliminate the plaintext, and store the ciphertext, corresponding CMK id and other information inside KBS (all of these are named a keyblob). Whenever a request for this key resource, KBS will use the keyblob to get the plaintext of the key and send back to AA.
4. This might not be a reason, because it is for signature verification key. Now we uses a plaintext of public key to verify image signature inside image-rs, which means that the public key is delivered from KBS->AA->Kata-Agent. If we brings KMS in, the plaintext of the public key will be delivered from KMS->KBS->AA->Kata-Agent. However, if we want to keep the attribute of "never-leave-KMS", then we need:
   a. Attestation-Agent should work as a side car as this comment and expose a new API for "Verifying this data blob using the given key id". image-rs should call this new API.
   b. Also, KBS should open a new API for "Verifying this data blob using the given key id" which cc-kbc calls.

We might need more views upon this : ).

cc @sameo @jiazhang0 @peterzcst @arronwy @jialez0

[jepio]

Answer to 4.: public key is not sensitive, so there is no reason to enforce "never-leave-kms" on it.

[jepio]

I have my doubts about "CMK" and "Datakey". You say "commercial KMS" support this, but I don't see direct equivalents with the same semantics in Azure Key Vault or Azure managed HSM. So I don't think we should specify the interface in those terms.

[Xynnn007]

Hi @jepio Thanks for your comments.

Answer to 4.: public key is not sensitive, so there is no reason to enforce "never-leave-kms" on it.

Agreed. What we can do is to export the public key after signing and store the public key inside the KBS before using it.

I have my doubts about "CMK" and "Datakey". You say "commercial KMS" support this, but I don't see direct equivalents with the same semantics in Azure Key Vault or Azure managed HSM. So I don't think we should specify the interface in those terms.

I must have ignored something important, for I can only learn from the open documents. In my mind Key Vault of Azure has enough power to follow the abstract as the following:

• Use the key created by Create Key as "CMK" (Customer Master Key, the core idea of which key is that it will never leave kms as plaintext)
• Use a key generated randomly inside "Key Vault"'s plugin to work as "Datakey". It could be wrappered by "CMK" mentioned before as this document says

I did a rough research on some other KMSes like aliyun KMS, aws KMS, gcp KMS and Azure Key Vault. I found that they can all implement this abstraction.

My biggest worry is that this abstraction is not good enough for us to leverage KMS to do image encryption and signing. However up to now I did not even get a better design idea.

[jepio]

Maybe we should focus on the use cases and then figure out the common interface. I also thought KBS should be stateless if it uses a KMS as a backend, in that case all secret keys should be stored in the KMS.

For encrypted images: we need a symmetric encryption key (LEK/Data Encryption Key/Content Encryption Key) that is used to perform layer decryption outside KMS. I say outside KMS because I don't see how it is feasible to do the cryptographic operations in an HSM for every single pod with the image sizes we deal with. So the encryption key needs to be protected (wrapped) with a KEK, which should also be symmetric.

Ideally we could store the wrapped symmetric key inside container image metadata, so that it is transported together with the encrypted image, otherwise we could store the wrapped key in object storage. If the wrapped key is part of the container image, then KBS interface needs to change to support unwrapping. If the wrapped key is in object storage then KBS can retrieve it. The key wrapping might rely on vendor-specific wrapped key representation ("datakey"), or we could rely on a standard representation like JSON Web Encryption (https://datatracker.ietf.org/doc/html/rfc7516).

For image signing the situation is different (more complicated) because signing really should happen inside a KMS. But I actually don't know if we should even try to address this, as there are various established ways to sign/verify images: sigstore, notary v2, gpg (does anyone have a good example of gpg integration with a KMS). So I'm not sure we should even try to involve KBS in this process.

[fitzthum]

@jepio Currently the LEK is part of the container image. It is stored as part of the layer annotation packet. Hence the unwrap_key API of the AA. Currently every KBC decrypts the LEK locally after requesting the KEK from a KBS, but in theory you could implement a KBC that forwards the packet to the KBS (and then potentially a KMS) for decryption. This is currently not supported in the generic KBS API (although we could probably shoe-horn in the functionality using some of the additional fields). Maybe I'm just restating what you were saying, but basically I think it isn't an enormous gap on the decryption side (not that we should necessarily prioritize). This would change the story a bit regarding encryption, though.

[jepio]

Currently the LEK is part of the container image. It is stored as part of the layer annotation packet. Hence the unwrap_key API of the AA. Currently every KBC decrypts the LEK locally after requesting the KEK from a KBS, but in theory you could implement a KBC that forwards the packet to the KBS (and then potentially a KMS) for decryption.

Good to know that the annotation packet already works the way I outlined above. If we agree that KMS support should be using fundamental KMS primitives (wrap key/unwrap key), then we can keep the annotation packet KMS-agnostic.

This is currently not supported in the generic KBS API (although we could probably shoe-horn in the functionality using some of the additional fields). Maybe I'm just restating what you were saying, but basically I think it isn't an enormous gap on the decryption side (not that we should necessarily prioritize).

We could live with the gap, but if we want proper KMS support then we will want to support non-extractable KEKs, which means we'll need to expand KBS to support some KMS like operations (unwrap/wrap key at least?)

This would change the story a bit regarding encryption, though.

I'm not super familiar with the encryption flow, but my intuition tells me we could reuse the same KMS interface in the keyprovider, communicate directly to KMS and bypass KBS?

[Xynnn007]

I'm not super familiar with the encryption flow, but my intuition tells me we could reuse the same KMS interface in the keyprovider, communicate directly to KMS and bypass KBS?

This is Way 1 in my initial proposal. It sounds good for me to keep the KMS operations, s.t. unwrap/wrap key.

An idea to work with the current keyid field in AnnotationPacket:

1. Define the basic semantics of KMS something like

#[async_trait::async_trait]
pub trait KMSAgent {
    /// Generate a key inside the KMS, and return the `keyid` of this key. The `keyid`
    /// can then be used to encrypt or decrypt data slice using this key.
    async fn generate_key(&mut self) -> Result<String>;
  
    /// Use the key of `keyid` to encrypt the `data` slice inside KMS, and then
    /// return the ciphertext of the `data`. The encryption operation should occur
    /// inside KMS. This function only works as a wrapper for different KMS APIs
    async fn encrypt(&mut self, data: &[u8], keyid: &str) -> Result<Vec<u8>>;

    /// Use the key of `keyid` to decrypt the `ciphertext` slice inside KMS, and then
    /// return the plaintext of the `data`. The decryption operation should occur
    /// inside KMS. This function only works as a wrapper for different KMS APIs
    async fn decrypt(&mut self, ciphertext: &[u8], keyid: &str) -> Result<Vec<u8>>;
}

Here the keyid field of both functions are specified by the concrete KMS. Some KMSes use only an UUID + region id to specify a concrete identity of a key, some uses other fields. The keyid here implies that the code writer should think of a way to reorganize the identity of the key into a single string without '/'.

2. The mapping rule (bijection) from a KBS Resource URI to a keyid

To avoid ambiguity, it should be clear that a keyid inside an AnnotationPacket is a KBS Resource URI. Let's use KBS Resource URI to refer to the id in an AnnotationPacket, and use keyid to refer to the parameter used in the interface defined before.

Then the mapping from a KBS Resource URI to a keyid:

kbs://<uri-of-the-kbs>/<repository>/kms-<name-of-kms>/<tag> (KBS Resource URI inside AnnotationPacket)
will be mapped to
KMS plugin <name-of-kms> and the keyid <tag>

3. basic encryption & decryption stream

When encrypting the image, cocokeyprovider would do the following pseudocode

let keyid = KMSAgent::generate_key();
let encrypted_lek = KMSAgent::encrypt(lek, keyid);
Serialize encrypted_lek and keyid in the AnnotationPacket

When decrypt the image

Deserialize encrypted_lek and keyid from the AnnotationPacket
let lek = KMSAgent::decrypt(encrypted_lek, keyid);

Either KBS has a unwrap/wrap API or not, the basic logic should exist (either inside KBS or a separate keyprovider).

What about this?

[Xynnn007]

Now different commercial KMS have different API. Some like Azure gives detailed parameters when en/decrypt like aad , iv , etc. Some does not, and they only expose an abstract key context, where the iv, aad and tag for AEAD are transparent to the users.

An naive idea from my side is to encapsulate the iv, aad, etc inside the KMS implementation. The interface only receives and kid to specify the key (context) to en/decrypt. Concrete KMS implementation decides how to translate this kid into a context to call the corresponding KMS API.

[Xynnn007]

duplication of #353