Skip to content
This repository has been archived by the owner on May 6, 2020. It is now read-only.

Add docker config to dev guide #893

Merged
merged 11 commits into from
Jan 4, 2018
Merged

Add docker config to dev guide #893

Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
fed961f
docs: Add missing distros to dev guide
jodh-intel Jan 4, 2018
fbfb327
docs: Add brackets and periods to dev guide
jodh-intel Jan 4, 2018
ed867e8
docs: Render variables correctly in dev guide
jodh-intel Jan 4, 2018
47dd041
docs: Explain install will overwrite binaries in dev guide
jodh-intel Jan 4, 2018
22d4bb1
docs: Move note to correct section of dev guide
jodh-intel Jan 4, 2018
7ed1a84
docs: Explain how to reconfigure Docker in dev guide
jodh-intel Jan 4, 2018
a582502
docs: Added packaging repo ref to dev guide
jodh-intel Jan 4, 2018
b165e89
docs: Add TOC to dev guide
jodh-intel Jan 4, 2018
1a1b750
docs: Create extra sections in dev guide
jodh-intel Jan 4, 2018
7a6e9af
docs: Removed blank line in dev guide
jodh-intel Jan 4, 2018
dbd4f6a
docs: Improving wording in dev guide
jodh-intel Jan 4, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
195 changes: 115 additions & 80 deletions docs/developers-clear-containers-install.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,17 @@
# Developers Clear Containers 3.0 Install

* [Initial setup](#initial-setup)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of interest - did you generate this by hand, or use a tool? If a tool, might be an idea to note the tool in the commit so others can follow suit later (n.b. I hand-added a toc entry earlier this week, and now realise maybe I should have seeked a tool)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually used a hacky script that requires manual post-processing to work with both github (and pandoc for local testing) but I believe @sameo found a website that does it when you feed it an .md file. Wouldn't it be awesome if github supported auto-generating a TOC, like you get for free with rST ;(

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would be nice if github handled it automatically (they do have their own md flavour - so should be able to add it ...) but, there are some tools around:
https://github.com/ekalinin/github-markdown-toc
https://github.com/jonschlinkert/markdown-toc

We could test and pick one for sure :-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep - we should prolly discuss this for kata too at some point...

* [Requirements to build individual components](#requirements-to-build-individual-components)
* [Clear Containers 3.0 components](#clear-containers-3.0-components)
* [Setup the environment](#setup-the-environment)
* [Build and install components](#build-and-install-components)
* [Proxy](#proxy)
* [Shim](#shim)
* [Runtime](#runtime)
* [Agent](#agent)
* [Kernel](#kernel)
* [See Also](#see-also)

This guide is not intended for end-users. Instead, this guide provides
instructions for any developers eager to try Clear Containers 3.0 and who
want to build Clear Containers from the source code and are familiar with the
Expand All @@ -11,19 +23,22 @@ The recommended way to create a development environment is to first install the
packaged versions of the Clear Containers components to create a working
system:

* [CentOS*](centos-installation-guide.md)
* [Clear Linux*](clearlinux-installation-guide.md)
* [Fedora*](fedora-installation-guide.md)
* [SLES*](sles-installation-guide.md)
* [Ubuntu*](ubuntu-installation-guide.md)

The installation guide instructions will install all packaged
components, plus docker, the hypervisor and the Clear Containers image
and kernel.
The installation guide instructions will install all required Clear Containers
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest using a series comma here (I believe I saw this in the style guide but correct me if I'm wrong) - I'm not certain whether "Clear Containers image" and "kernel" should be paired together or separately. I'm assuming together for this re-write: The installation guide instructions will install all required Clear Containers components, plus Docker*, the hypervisor, and the Clear Containers image and kernel.

components, plus Docker*, the hypervisor, and the Clear Containers image and
kernel.

## Requirements to build individual components

* [go 1.8.3](https://golang.org/)
* [go 1.8.3](https://golang.org/).
* [gcc](https://gcc.gnu.org/) and associated C language build tooling
such as `make`, `autoconf` and `libtool` which are required
to build `cc-shim`
(e.g. `make`, `autoconf` and `libtool`) are required
to build `cc-shim`.

## Clear Containers 3.0 components

Expand All @@ -42,13 +57,13 @@ See [the upgrading document](upgrading.md) for further details.

### Setup the environment

1. Define GOPATH
1. Define `GOPATH`

```bash
$ export GOPATH=$HOME/go
```

2. Create GOPATH Directory
2. Create `GOPATH` Directory

```bash
$ mkdir -p $GOPATH
Expand All @@ -67,106 +82,126 @@ See [the upgrading document](upgrading.md) for further details.

### Build and install components

1. Proxy
#### Proxy

```bash
$ cd $GOPATH/src/github.com/clearcontainers/proxy
$ make
$ sudo make install
```

2. Shim
```bash
$ cd $GOPATH/src/github.com/clearcontainers/proxy
$ make
$ sudo make install
```

```bash
$ cd $GOPATH/src/github.com/clearcontainers/shim
$ ./autogen.sh
$ make
$ sudo make install
```
Note: the previous install step will overwrite any proxy binary installed from
the `cc-proxy` package.

3. Runtime
#### Shim

```bash
$ cd $GOPATH/src/github.com/clearcontainers/runtime
$ make build-cc-system
$ sudo -E PATH=$PATH make install-cc-system
```
```bash
$ cd $GOPATH/src/github.com/clearcontainers/shim
$ ./autogen.sh
$ make
$ sudo make install
```

4. Agent
Note: the previous install step will overwrite any shim binary installed from
the `cc-shim` package.

```bash
$ cd $GOPATH/src/github.com/clearcontainers/agent
$ make
```
#### Runtime

The agent is installed inside the root filesystem image
used by the hypervisor, hence to test a new agent version it is
necessary to create a custom rootfs image. The example below
demonstrates how to do this using the
[osbuilder](https://github.com/clearcontainers/osbuilder) tooling.
```bash
$ cd $GOPATH/src/github.com/clearcontainers/runtime
$ make build-cc-system
$ sudo -E PATH=$PATH make install-cc-system
```

```bash
$ cd $GOPATH/src/github.com/clearcontainers/osbuilder
The previous install step will create `/usr/local/bin/cc-runtime`. This
ensures the packaged version of the runtime from the `cc-runtime` package is
not overwritten. However, since the installation guide configured Docker to
call the runtime as `/usr/bin/cc-runtime`, it is necessary to modify the
docker configuration to make use of your newly-installed development runtime
binary:

$ # Create a rootfs image
$ sudo -E make rootfs USE_DOCKER=true
```bash
$ sudo sed -i 's!cc-runtime=/usr/bin/cc-runtime!cc-runtime=/usr/local/bin/cc-runtime!g' /etc/systemd/system/docker.service.d/clear-containers.conf
```

$ # Overwrite the default cc-agent binary with a custom built version
$ sudo cp $GOPATH/src/github.com/clearcontainers/agent/cc-agent ./workdir/rootfs/usr/bin/cc-agent
For more details on the runtime's build system, run:

$ # Generate a container.img file
$ sudo -E make image USE_DOCKER=true
```bash
$ make help
```

$ # Install the custom image
$ sudo install --owner root --group root --mode 0755 workdir/container.img /usr/share/clear-containers/
#### Agent

$ # Update the runtime configuration
$ # (note that this is only an example using default paths).
$ sudo sed -i.bak -e 's!^\(image = ".*"\)!# \1 \
image = "/usr/share/clear-containers/container.img"!g' \
/usr/share/defaults/clear-containers/configuration.toml
```
```bash
$ cd $GOPATH/src/github.com/clearcontainers/agent
$ make
```

For more details on the runtime's build system, run:
The agent is installed inside the root filesystem image
used by the hypervisor, hence to test a new agent version it is
necessary to create a custom rootfs image. The following example
demonstrates how to do this using the
[osbuilder](https://github.com/clearcontainers/osbuilder) tooling.

```bash
$ make help
$ cd $GOPATH/src/github.com/clearcontainers/osbuilder

$ # Create a rootfs image
$ sudo -E make rootfs USE_DOCKER=true

$ # Overwrite the default cc-agent binary with a custom built version
$ sudo cp $GOPATH/src/github.com/clearcontainers/agent/cc-agent ./workdir/rootfs/usr/bin/cc-agent

$ # Generate a container.img file
$ sudo -E make image USE_DOCKER=true

$ # Install the custom image
$ sudo install --owner root --group root --mode 0755 workdir/container.img /usr/share/clear-containers/

$ # Update the runtime configuration
$ # (note that this is only an example using default paths).
$ sudo sed -i.bak -e 's!^\(image = ".*"\)!# \1 \
image = "/usr/share/clear-containers/container.img"!g' \
/usr/share/defaults/clear-containers/configuration.toml
```

5. Kernel
#### Kernel

The latest kernel for Clear Linux can be found on
[the releases page](https://github.com/clearcontainers/linux/releases).
The Clear Linux kernel can be used to rebuild and modify a custom VM
container kernel as needed. The example below demonstrates how to do this
using the [osbuilder](https://github.com/clearcontainers/osbuilder) tooling.
The latest kernel for Clear Linux can be found on
[the releases page](https://github.com/clearcontainers/linux/releases).
The Clear Linux kernel can be used to rebuild and modify a custom VM
container kernel as needed. The following example demonstrates how to do this
using the [osbuilder](https://github.com/clearcontainers/osbuilder) tooling.

```bash
$ cd $GOPATH/src/github.com/clearcontainers/osbuilder
```bash
$ cd $GOPATH/src/github.com/clearcontainers/osbuilder

$ # Fetch latest kernel sources from github.com/clearcontainers/linux
$ make kernel-src
$ # Fetch latest kernel sources from github.com/clearcontainers/linux
$ make kernel-src

$ # Optionally modify kernel sources or config at $PWD/workdir/linux
$ # Optionally modify kernel sources or config at $PWD/workdir/linux

$ # build kernel
$ make kernel
$ # build kernel
$ make kernel

$ # Install the custom image
$ sudo install --owner root --group root --mode 0755 workdir/vmlinuz.container /usr/share/clear-containers/custom-vmlinuz
$ sudo install --owner root --group root --mode 0755 workdir/vmlinux.container /usr/share/clear-containers/custom-vmlinux
$ # Install the custom image
$ sudo install --owner root --group root --mode 0755 workdir/vmlinuz.container /usr/share/clear-containers/custom-vmlinuz
$ sudo install --owner root --group root --mode 0755 workdir/vmlinux.container /usr/share/clear-containers/custom-vmlinux

$ # Update the runtime configuration
$ # (note that this is only an example using default paths).
$ # Note: vmlinuz is used for pc platform type.
$ # vmlinux is used for pc-lite and q35-lite platform type.
$ sudo sed -i.bak -e 's!^\(kernel = ".*"\)!# \1\nkernel = "/usr/share/clear-containers/custom-vmlinuz"!g' \
/usr/share/defaults/clear-containers/configuration.toml
```
$ # Update the runtime configuration
$ # (note that this is only an example using default paths).
$ # Note: vmlinuz is used for pc platform type.
$ # vmlinux is used for pc-lite and q35-lite platform type.
$ sudo sed -i.bak -e 's!^\(kernel = ".*"\)!# \1\nkernel = "/usr/share/clear-containers/custom-vmlinuz"!g' \
/usr/share/defaults/clear-containers/configuration.toml
```

## See Also

* [General Debugging](../README.md#debugging)
* [Debugging the agent inside the hypervisor](debug-agent.md)
* [Debugging the kernel inside the hypervisor](https://github.com/clearcontainers/runtime/blob/master/docs/debug-kernel.md)

* Packaging
The repository used to create the packaged versions of Clear Containers
components for various Linux distributions is:
* https://github.com/clearcontainers/packaging