Author	王丰土
Date	2020-05-28
Email	wangfengtu@huawei.com

1. Program Objectives

The registry module is located as follows:

In addition to accepting calls from the Manager module, the Registry module also calls the store module to store the downloaded images and layers.

In the process of pulling the image, the libcurl library is used to realize the interaction with the registry. The processing of some certificates and TLS algorithms that need to be used in the interaction process has been implemented in libcurl, and the path can be configured and passed in when using.

Authentication in the process of interacting with the registry only needs to support basic authentication. The Bear during the interaction with the registry is generated by the registry, and the client only needs to save it and carry it in subsequent operations.

It is necessary to implement the pull part of the protocol Docker Registry HTTP API V2, and the pull part of the OCI distribution spec. This article mainly describes the download of docker images. For the download of OCI images, please refer to the protocol. There are two formats for downloading the manifest of a docker image:

Image Manifest Version 2, Schema 2
Image Manifest Version 2, Schema 1

2. Overall Design

The internal structure of the Registry is as follows:

Registry apiv2 module: Implement the interaction protocol with the registry, including Schema1 and Schema2. Mainly to implement the specific protocols for downloading manifest, downloading config files, and downloading layers files, including the ping operation that needs to be performed when downloading.
Registry module: Call the registry apiv2 module to download the mirror-related files, and after decompression/validity check, the interface of the store is registered as the mirror, and the interface of the Manager module is provided.
Auth/certs module: Manage local username/password, certificate, shared private key and other data.
http/https request module: use the libcurl library to encapsulate the http/https interaction process that interacts with the registry, including calling the auth/certs module to obtain/set related password certificates and other operations, as well as auth interacting with the registry Implementation of protocols such as authentication login operations.

3. Interface Description

typedef struct {
    /* Username to use when logging in to the repository */
    char *username;
    /* Password used to log in to the repository */
    char *password;
}registry_auth;

typedef struct {
    char *image_name;
    char *dest_image_name;
    registry_auth auth;
    bool skip_tls_verify;
    bool insecure_registry;
} registry_pull_options;

typedef struct {
    /* Mirror repository address */
    char *host;
    registry_auth auth;
    bool skip_tls_verify;
    bool insecure_registry;
} registry_login_options;

int registry_init();
int registry_pull(registry_pull_options *options);
int registry_login(registry_login_options *options);
int registry_logout(char *auth_file_path, char *host);

void free_registry_pull_options(registry_pull_options *options);
void free_registry_login_options(registry_login_options *options);

4. Detailed Design

registry module

The Registry module calls the registry apiv2 module to download the mirror-related files, decompresses/checks the validity, and registers the interface of the store as the mirror, and provides a calling interface to the Manager module.

Login operation: directly call the interface implementation provided by the registry apiv2 module.

Logout operation; call the interface implementation provided by the auth/certs module.

The following describes the process of pulling images. The protocol implementation in the interaction process is implemented by the registry apiv2 module:

According to the incoming image name, assemble the address to obtain the manifest, and request the manifest from the mirror repository.

The returned manifests format (assuming schema1 is returned):

200 OK
Docker-Content-Digest: <digest>
Content-Type: <media type of manifest>

{
   "name": <name>,
   "tag": <tag>,
   "fsLayers": [
      {
         "blobSum": "<digest>"
      },
      ...
    ]
   ],
   "history": <v1 images>,c
   "signature": <JWS>
}

For the detailed meaning of the format, please refer to the link: https://docs.docker.com/registry/spec/manifest-v2-1/

If it is schema2, the returned json format sample is as follows:

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
  "manifests": [
    {
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "size": 7143,
      "digest": "sha256:e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7fc331f",
      "platform": {
        "architecture": "ppc64le",
        "os": "linux",
      }
    },
    {
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "size": 7682,
      "digest": "sha256:5b0bcabd1ed22e9fb1310cf6c2dec7cdef19f0ad69efa1f392e94a4333501270",
      "platform": {
        "architecture": "amd64",
        "os": "linux",
        "features": [
          "sse4"
        ]
      }
    }
  ]
}

For the detailed meaning of the format, please refer to the link: https://docs.docker.com/registry/spec/manifest-v2-2/

We only support the following MediaTypes of Manifests:

application/vnd.docker.distribution.manifest.v2+json
application/vnd.docker.distribution.manifest.v1+prettyjws. it needs to be able to download and use, signature will not be parsed for now.
application/vnd.docker.distribution.manifest.v1+json
application/vnd.docker.distribution.manifest.list.v2+json
application/vnd.oci.image.manifest.v1+json supports OCI image

After obtaining the manifest, parse the manifest to obtain the configuration of the image and the digest information of all layers.
According to the digest of the mirror's configuration and the digest information of all layers, splicing out the url addresses for downloading all these data and downloading them (this can be downloaded concurrently).
After the download is complete, you need to decompress the image layer data, decompress it into tar format data, and calculate the sha256 value. Then, it is necessary to parse the image configuration information, obtain the DiffID of the layer saved in the configuration, and compare it with the downloaded layer data for sha256 to verify its correctness.

When verifying, take the rootfs.diff_ids[$i] value in the configuration (that is, the sha256 value of the $i-th layer), and take the downloaded data of the $i-th layer decompressed into tar format as the sha256 value. The two values need to be completely consistent. The values in the configuration are as follows:

"RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:e7ebc6e16708285bee3917ae12bf8d172ee0d7684a7830751ab9a1c070e7a125",
                "sha256:f934e33a54a60630267df295a5c232ceb15b2938ebb0476364192b1537449093",
                "sha256:bf6751561805be7d07d66f6acb2a33e99cf0cc0a20f5fd5d94a3c7f8ae55c2a1",
                "sha256:943edb549a8300092a714190dfe633341c0ffb483784c4fdfe884b9019f6a0b4",
                "sha256:c1bd37d01c89de343d68867518b1155cb297d8e03942066ecb44ae8f46b608a3",
                "sha256:cf612f747e0fbcc1674f88712b7bc1cd8b91cf0be8f9e9771235169f139d507c",
                "sha256:14dd68f4c7e23d6a2363c2320747ab88986dfd43ba0489d139eeac3ac75323b2"
            ]
        }

Call the store module to register the downloaded layer data and configuration to generate an image, and add the corresponding name to the image.

registry apiv2 module

This module implements the protocol for interacting with the image repository (Docker Registry HTTP API V2) and the part of the protocol related to pulling images. The manifest forms include Image Manifest Version 2, Schema 1 and Schema 2.

Here is a brief description of the protocol. For details, please refer to the detailed description in the link below:

https://docs.docker.com/registry/spec/api/ https://docs.docker.com/registry/spec/manifest-v2-1/ https://docs.docker.com/registry/spec/manifest-v2-2/

And the OCI distribution spec based on the above protocol:

https://github.com/opencontainers/distribution-spec/blob/master/spec.md

The interaction process with the mirror repository will first try to use the https protocol, and if it fails, it will continue to try to use the http protocol to interact (this behavior can be configured).

There are some general configuration/interaction procedures for the interaction process, as follows:

Ping the registry. In the process of login/download, you only need to ping the registry once. The main purpose of Ping is to obtain relevant information returned by the registry. ping request format:

GET /v2/
Host: <registry host>
Authorization: <scheme> <token>

success: 200 OK V2 protocol is not supported: 404 Not Found not certified:

401 Unauthorized
WWW-Authenticate: <scheme> realm="<realm>", ..."
Content-Length: <length>
Content-Type: application/json; charset=utf-8

{
"errors:" [
{
            "code": <error code>,
            "message": "<error message>",
            "detail": ...
        },
        ...
    ]
}

The header information returned by Ping must contain the fields: Docker-Distribution-API-Version:registry/2.0

If it returns 401 Unauthenticated, the field WWW-Authenticate must be included to indicate where we should go for authentication.

Perform authentication. First obtain the relevant authentication method information from the returned 401 unauthenticated http header field WWW-Authenticate. Authentication methods only support Basic and Bearer methods. Note that multiple WWW-Authenticate fields may be carried, indicating that multiple authentication methods are supported at the same time, and each field needs to be parsed and processed.

In Basic mode, all subsequent requests need to carry the following header information:

Authorization: Basic QWxhZGRpbjpPcGVuU2VzYW1l

Where QWxhZGRpbjpPcGVuU2VzYW1l is the base64 encoded string of the username and password in the format username:passord.

Bearer token method, all subsequent requests need to carry the following header information:

Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJFUzI1NiIsImtpZCI6IkJWM0Q6MkFWWjpVQjVaOktJQVA6SU5QTDo1RU42Ok40SjQ6Nk1XTzpEUktFOkJWUUs6M0ZKTDpQT1RMIn0.eyJpc3MiOiJhdXRoLmRvY2tlci5jb20iLCJzdWIiOiJCQ0NZOk9VNlo6UUVKNTpXTjJDOjJBVkM6WTdZRDpBM0xZOjQ1VVc6NE9HRDpLQUxMOkNOSjU6NUlVTCIsImF1ZCI6InJlZ2lzdHJ5LmRvY2tlci5jb20iLCJleHAiOjE0MTUzODczMTUsIm5iZiI6MTQxNTM4NzAxNSwiaWF0IjoxNDE1Mzg3MDE1LCJqdGkiOiJ0WUpDTzFjNmNueXk3a0FuMGM3cktQZ2JWMUgxYkZ3cyIsInNjb3BlIjoiamxoYXduOnJlcG9zaXRvcnk6c2FtYWxiYS9teS1hcHA6cHVzaCxwdWxsIGpsaGF3bjpuYW1lc3BhY2U6c2FtYWxiYTpwdWxsIn0.Y3zZSwaZPqy4y9oRBVRImZyv3m_S9XDHF1tWwN7mL52C_IiA73SJkWVNsvNqpJIn5h7A2F8biv_S2ppQ1lgkbw

One of the long list of tokens is obtained from the authentication server,

For the detailed implementation of the protocol, see the link: https://docs.docker.com/registry/spec/auth/token/

The login verification process is as shown above. The meaning of each step is as follows:

The client tries to pull the image from the registry, that is, sends a request to pull the image, or other operations.
If the registry needs to log in, it will return 401 Unauthorized not authenticated, and the returned message carries the WWW-Authenticate field, as shown below:

www-Authenticate: Bearer realm="https://auth.docker.io/token",service="registry.docker.io",scope="repository:samalba/my-app:pull,push"

The meaning of each field is as follows:

Bearer realm: Authentication server address.
service: mirror repository address.
scope: The scope of the operation, that is, which permissions are required.

According to the previously returned information, the client assembles a URL request to request a bear token from the authentication server for subsequent interaction. The assembled URL looks like this:

https://auth.docker.io/token?service=registry.docker.io&scope=repository:samalba/my-app:pull,push

The authentication server returns information such as token and expiration time

HTTP/1.1 200 OK
Content-Type: application/json

{"token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJFUzI1NiIsImtpZCI6IlBZWU86VEVXVTpWN0pIOjI2SlY6QVFUWjpMSkMzOlNYVko6WEdIQTozNEYyOjJMQVE6WlJNSzpaN1E2In0.eyJpc3MiOiJhdXRoLmRvY2tlci5jb20iLCJzdWIiOiJqbGhhd24iLCJhdWQiOiJyZWdpc3RyeS5kb2NrZXIuY29tIiwiZXhwIjoxNDE1Mzg3MzE1LCJuYmYiOjE0MTUzODcwMTUsImlhdCI6MTQxNTM4NzAxNSwianRpIjoidFlKQ08xYzZjbnl5N2tBbjBjN3JLUGdiVjFIMWJGd3MiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6InNhbWFsYmEvbXktYXBwIiwiYWN0aW9ucyI6WyJwdXNoIl19XX0.QhflHPfbd6eVF4lM9bwYpFZIV0PfikbyXuLx959ykRTBpe3CYnzs6YBK8FToVb5R47920PVLrh8zuLzdCr9t3w", "expires_in": 3600,"issued_at": "2009-11-10T23:00:00Z"}

Re-request the pull operation, this time carrying the Bearer token in the Authorization field as the identification of successful authentication:

Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJFUzI1NiIsImtpZCI6IkJWM0Q6MkFWWjpVQjVaOktJQVA6SU5QTDo1RU42Ok40SjQ6Nk1XTzpEUktFOkJWUUs6M0ZKTDpQT1RMIn0.eyJpc3MiOiJhdXRoLmRvY2tlci5jb20iLCJzdWIiOiJCQ0NZOk9VNlo6UUVKNTpXTjJDOjJBVkM6WTdZRDpBM0xZOjQ1VVc6NE9HRDpLQUxMOkNOSjU6NUlVTCIsImF1ZCI6InJlZ2lzdHJ5LmRvY2tlci5jb20iLCJleHAiOjE0MTUzODczMTUsIm5iZiI6MTQxNTM4NzAxNSwiaWF0IjoxNDE1Mzg3MDE1LCJqdGkiOiJ0WUpDTzFjNmNueXk3a0FuMGM3cktQZ2JWMUgxYkZ3cyIsInNjb3BlIjoiamxoYXduOnJlcG9zaXRvcnk6c2FtYWxiYS9teS1hcHA6cHVzaCxwdWxsIGpsaGF3bjpuYW1lc3BhY2U6c2FtYWxiYTpwdWxsIn0.Y3zZSwaZPqy4y9oRBVRImZyv3m_S9XDHF1tWwN7mL52C_IiA73SJkWVNsvNqpJIn5h7A2F8biv_S2ppQ1lgkbw

The server verifies the Bearer token and allows the pull operation.

The following describes the process of downloading manifest/config/layers data:

According to the incoming image name, assemble the address to obtain the manifest, and request the manifest from the mirror repository.

Request manifests:

GET /v2/<name>/manifests/<reference>
Host: <registry host>
Authorization: <scheme> <token>

The name here is the image name, excluding the tag, and the reference refers to the tag.

For example, to pull the image docker.io/library/node:latest, the above format is:

GET /v2/library/node/manifests/latest

The Content-Type field in the returned header information will carry the specific manifest type (see the description in Section 5.1.2 above). The body content is the corresponding json string.

The configuration and layer of the image are blobs for the registry. As long as the digest value is parsed in the manifest, the blob data can be obtained. The value of digest is the sha256 value of the configuration/layer data (before decompression), and is also part of the url to download these blob data:

Get the request for layer/digest (Requests can be concurrent):

GET /v2/<name>/blobs/<digest>
Host: <registry host>
Authorization: <scheme> <token>

Obtain success (for the format of failure return, please refer to the protocol):

200 OK
Content-Length: <length>
Docker-Content-Digest: <digest>
Content-Type: application/octet-stream

<blob binary data>

auth/certs module

This module is divided into two parts, auth is responsible for managing the username and password of login, and provides an interface for reading and setting. certs is responsible for managing the certificates and private keys used in https requests when interacting with the registry, and provides a read interface.

The certificate used to log in to the registry is stored in the /root/.isulad/auths.json file, as follows:

# pwd
/root/.isulad
# ls
aeskey auths.json auths.json.lock
# cat auths.json
{
"auths": {
"dockerhub.test.com": {
"auth": "nS6GX1wnf4WGe6+O+nS6Py6CVzPPIQJBOksaFSfFAy9LRUijubMIgZhtfrA="
}
}
}

The auths in auths.json save registry information and the corresponding username and password. The user name and password encryption rule is to combine the user name and password into a $USERNAME:$PASSWORD, then use AES encryption. After that, use base64 to encode the encrypted data and store it as the auth field of the json file.

The certificate used for HTTPS requests is placed in the /etc/isulad/certs.d/$registry directory:

# pwd
/etc/isulad/certs.d/dockerhub.test.com
# ls
ca.crt tls.cert tls.key

http/https request module

Interacting with image repositories requires calling the libcurl library to implement the client protocol of registry API V2.

The processing of the protocol has been described above, and the encapsulation of http/https requests is mainly described here.

libcurl provides atomic commands to implement requests. This module needs to encapsulate the http_request interface based on the atomic interface provided by libcurl. Multiple interfaces need to be encapsulated so that various requests can be handled easily. It mainly encapsulates three functions:

Interact with the registry by returning data in memory, which is used for operations with a small amount of data such as ping.
Interact with the authentication server by returning data through the memory to obtain the token. This function is used inside the module and does not provide an external interface.
Return data through files. It is used for fetching relatively large amount of data requests, such as fetching blob data and fetching manifests data.

In addition to the URL, the following parameters need to be supported:

Authentication information such as username and password
Whether to return the message header or the body or both
TLS related information
Customize message header information

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

registry_degisn.md

registry_degisn.md

1. Program Objectives

2. Overall Design

3. Interface Description

4. Detailed Design

registry module

registry apiv2 module

auth/certs module

http/https request module

Files

registry_degisn.md

Latest commit

History

registry_degisn.md

File metadata and controls

1. Program Objectives

2. Overall Design

3. Interface Description

4. Detailed Design

registry module

registry apiv2 module

auth/certs module

http/https request module