From 54cf6922f00d3f124cc27a4de50fc2b39485ef56 Mon Sep 17 00:00:00 2001 From: PhilippMDoerner Date: Mon, 22 Jul 2024 02:57:18 +0200 Subject: [PATCH] Improve deployment docs (#245) --- docs/deployment.md | 157 ++++++++++++++++++++++++--------------------- 1 file changed, 85 insertions(+), 72 deletions(-) diff --git a/docs/deployment.md b/docs/deployment.md index 7bb433524..b7258c6d8 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -1,6 +1,7 @@ # Deploying Prologue with nginx and docker + This is an example of how you can deploy a prologue web-application that was compiled under Linux. -. + ## Base Images We will look at 2 base images as starting points: @@ -18,6 +19,7 @@ This guide assumes that: 4. You have set up your server with a domain name ## Compile your binary + Compiling your binary differs between alpine and other linux images. This is because unlike most other linux distributions, alpine does not use the `gnu c library` (glibc) to link its binaries against. Alpine uses `musl`, which is much more minimalistic. @@ -25,28 +27,26 @@ Alpine uses `musl`, which is much more minimalistic. Because `musl` and `glibc` are different, compiling your application to link with them also differs. ### On bitnami/minideb + bitnami/minideb is basically a Debian image, reduced to the minimum. -Since it is based on debian, it uses `gnu c library` (glibc), which is the main advantage of this image. +Since it is based on debian, it uses `gnu c library` (glibc), which is the main advantage of this image. Since the majority of Linux systems use `glibc`, compiling on any Linux distro will give you a binary that is dynamically linked against your `glibc` version and thus is likely to work in the image. If you have to ask whether your distro is using `glibc`, you are using `glibc`. You can compile your project like this: + ```sh nim c \ ---forceBuild:on \ ---opt:speed \ --define:release \ ---threads:on \ --mm:orc \ --deepcopy:on \ --define:lto \ --define:ssl \ ---hints:off \ ---outdir:"." \ .nim ``` + **For Clang users (e.g. MacOs)**: Clang doesn't work properly with `--define:lto`. Replace it with `--passC:"-flto" --passL:"-flto"` which achieves the same thing. If you want to avoid re-typing all the flags, you can define them in a [nimble task](https://github.com/nim-lang/nimble#nimble-tasks) that you can trigger instead! @@ -54,25 +54,20 @@ Just add the following to your `.nimble` file (run [`nimble init`] ```txt task release, "Build a production release": - --verbose - --forceBuild:on - --opt:speed --define:release - --threads:on --mm:orc --deepcopy:on --define:lto --define:ssl # If you use smtp clients - --hints:off - --outdir:"." setCommand "c", "src/.nim" ``` This allows you to run `nimble release` in your project to compile it with the specified flags! ### On alpine -Alpine is among the smallest image out there. -At barely 5.53MB, any image you base on it is unlikely to get large. + +Alpine is among the smallest image out there. +At barely 5.53MB, any image you base on it is unlikely to get large. This doesn't have effects on your application's performance, but does speed deployment as the image uploads faster to your server. Since Alpine uses `musl`, compiling has some extra steps. @@ -82,51 +77,45 @@ For simplicities' sake, we will be linking to musl dynamically. 1. [download](https://www.musl-libc.org/download.html) the tar file 2. Unpack the tar file somewhere -3. Run `bash configure` in the unpacked directory. WARNING: Make sure that you do NOT install musl with `--prefix` being `/usr/local`, as that may adversely affect your system. Use somewhere else where it is unlikely to override files, e.g. `/usr/local/musl`. This path will further be referred to as `` +3. Run `bash configure` in the unpacked directory with the --prefix flag. For example: `bash configure --prefix /usr/local/musl`. WARNING: Make sure that you do NOT install musl with `--prefix` being `/usr/local`, as that may adversely affect your system. Use somewhere else where it is unlikely to override files, e.g. `/usr/local/musl`. This path will further be referred to as ``. 4. Run `make && make install` in the unpacked directory 5. Add `` to your PATH environment variable -6. Validate whether you set everything up correctly by opening a new terminal and seeing whether you have access to the `musl-gcc` binary +6. Validate whether you set everything up correctly by opening a new terminal and running `musl-gcc`, you should be getting an output. #### Compiling your dynamically linked musl binary -Like before, you can write a compile command. + +Like before, you can write a compile command. This time though, we need to tell the compiler to use `musl-gcc` instead of `gcc` to dynamically link with `musl`. We similarly want to replace the linker with `musl-gcc`. -We can use the flags `--gcc.exe:"musl-gcc"` and `--gcc.linkerexe:"musl-gcc"` to get a compile command: +We can use the flags `--gcc.exe:"musl-gcc"` and `--gcc.linkerexe:"musl-gcc"` to get a functional compile command. +If you're using musl with a GCC version later than 14.1, you may need to turn off GCC's new type-checks on pointers using `--passc:"-fpermissive"` and `--passl:"-fpermissive"`. This should more generally not pose a problem, as nim itself has a proper type-system. ```sh nim c \ --gcc.exe:"musl-gcc" \ --gcc.linkerexe:"musl-gcc" \ ---forceBuild:on \ ---opt:speed \ +--passc:"-fpermissive" +--passl:"-fpermissive" --define:release \ ---threads:on \ --mm:orc \ --deepcopy:on \ --define:lto \ --define:ssl \ ---hints:off \ ---outdir:"." \ .nim ``` Instead of a bash script we can also just set up a nimble task instead: + ```txt task alpine, "Build an alpine release": - --verbose --gcc.exe:"musl-gcc" --gcc.linkerexe:"musl-gcc" - --forceBuild:on - --opt:speed --define:release - --threads:on --mm:orc --deepcopy:on --define:lto - --define:ssl - --hints:off - --outdir:"." - setCommand "c", "src/nimstoryfont.nim" + --define:ssl # If you use smtp clients + setCommand "c", "src/.nim" ``` ## Prepare buildFiles @@ -139,8 +128,9 @@ task alpine, "Build an alpine release": We will store these files in a directory called `buildFiles` and include them in our docker image(s) later. ### Set up your server to have SSL certificates with certbot + There are many great resources on how you can get free SSL certificates. -We recommend using [certbot](https://certbot.eff.org/instructions). +We recommend using [certbot](https://certbot.eff.org/instructions). Follow the instructions on their website to set up your certificates. After the initial setup, you should automate the renewal of those certificates. @@ -153,14 +143,16 @@ certbot renew --pre-hook "docker container stop " --post-ho ``` ### Provide an `nginx.conf` config file + Nginx requires a config file, `nginx.conf` to define what files to serve and how to forward requests to the prologue application. Here we'll set nginx up to use SSL with the SSL certificates received from the previous step. Further, to make nginx forward requests to our prologue backend, we use its "[proxy_pass](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass)" directive. -Note that all directories in the config file are directories within the *docker* container, not your actual server. +Note that all directories in the config file are directories within the _docker_ container, not your actual server. See below an example of a small `nginx.conf` file: + ```txt #user http; worker_processes 1; @@ -194,14 +186,14 @@ http { autoindex off; listen 443 ssl http2; - listen [::]:443 ssl http2; - + listen [::]:443 ssl http2; + root /media; # Passes requests on to the prologue application server location /server { rewrite ^/server/(.*) /$1 break; #Removes "/server" from the url for the application-server - + proxy_pass http://localhost:8080; #Hands request over to localhost:8080 where the application server is listening proxy_set_header Host $host; proxy_send_timeout 600; @@ -219,12 +211,13 @@ http { } ``` -Note that the file assumes there will be a `fullchain.pem` and `privkey.pem`, which you currently have on your server (e.g. `/etc/letsencrypt/live`). +Note that the file assumes there will be a `fullchain.pem` and `privkey.pem`, which you currently have on your server (e.g. `/etc/letsencrypt/live`). We will make these certificates accessible by creating the `/cert/live` directories inside the docker image and mounting the certificate directory to that folder (e.g. when creating a container via [docker volumes](https://docs.docker.com/storage/volumes/)). For now store your `nginx.conf` file in your `./buildFiles` directory. ### Provide a `settings.json` config file for prologue + In order for prologue to function correctly, we will provide a simple settings file. ```json @@ -242,14 +235,15 @@ In order for prologue to function correctly, we will provide a simple settings f Note that the port you specify here must be the same port used in the `proxy_pass` directive of `./buildFiles/nginx.conf`. -Put this `settings.json` in your `./buildFiles` directory. +Put this `settings.json` in your `./buildFiles` directory. ## Setting up docker + After completing the prior steps, we can now think how to deploy our application how we want to deploy our application and server. We have 2 different applications to manage here: nginx and our prologue application. We can deploy these in 2 different ways: -- Prologue and Nginx in the same container (simpler) -- Prologue and Nginx in separate containers via docker-compose (Enables hosting multiple web-applications from the same server) +- Prologue and Nginx in the same container (simpler) +- Prologue and Nginx in separate containers via docker-compose (Enables hosting multiple web-applications from the same server) ## A) Prologue and Nginx in the same container @@ -268,20 +262,23 @@ rc-service nginx start # Starts up nginx Store your script file locally in `./buildFiles/startupScript.sh`. ### Write your dockerfile + After completing the prior steps, we can write a `dockerfile`. It will contain the instructions to create a docker image, from which containers are made. -This process differs between `bitnami/minideb` and `alpine`, since `alpine` uses `apk` to install packages from the [alpine repositories](https://alpine.pkgs.org/?) while `bitnami/minideb` uses apt to install [debian packages](https://www.debian.org/distrib/packages). +This process differs between `bitnami/minideb` and `alpine`, since `alpine` uses `apk` to install packages from the [alpine repositories](https://alpine.pkgs.org/?) while `bitnami/minideb` uses apt to install [debian packages](https://www.debian.org/distrib/packages). This means the installation commands and package names differ. -We will also set up various folders, in order to later use them as mounting points for [docker volumes](https://docs.docker.com/storage/volumes/) when creating containers from our images. +We will also set up various folders, in order to later use them as mounting points for [docker volumes](https://docs.docker.com/storage/volumes/) when creating containers from our images. This way you can access files inside the container (e.g. media files so that nginx can serve them, or an sqlite database) without loosing them when the container shuts down. Store the dockerfile on your applications root directory. #### On `bitnami/minideb` + To install packages without `apt`'s commandline prompts, this image ships a `install_packages` command. Here an example `dockerfile`: + ```txt FROM bitnami/minideb @@ -309,7 +306,9 @@ CMD ["/dockerStartScript.sh"] ``` #### On `alpine` + Here an example dockerfile: + ```txt FROM alpine @@ -336,6 +335,7 @@ CMD ["/dockerStartScript.sh"] ``` ### Build the docker image + With the binary, buildFiles and dockerfile ready, you can create your image: ```sh @@ -348,6 +348,7 @@ sudo docker save -o image.tar Once created, move that image file to your server, e.g. through `scp`. ### Run the docker image on your server + After copying your `image.tar` file to the server, you can load it there with docker and run a container from it. Besides starting the container, the commands needs to: @@ -357,13 +358,14 @@ Besides starting the container, the commands needs to: 4. (Optional) Open up a port to talk to your database (e.g. 5432 for Postgres) Opening up the ports is done using `-p`, mounting the volumes with `-v`. -Note that in `-v`, the first path is the one *outside* your container. -It specifies which folder on your server to mount. -The second path is the one *inside* your container. +Note that in `-v`, the first path is the one _outside_ your container. +It specifies which folder on your server to mount. +The second path is the one _inside_ your container. It specifies which folder in your container the server folder gets mounted to. -You may want to write yourself a script that loads the file, stops and removes any previously running container and then creates a new one from the new image. +You may want to write yourself a script that loads the file, stops and removes any previously running container and then creates a new one from the new image. Here an example: + ```sh #!/bin/sh sudo docker load -i image.tar @@ -382,26 +384,28 @@ sudo docker run -p 80:80 -p 443:443 \ Now you can run the command (or script), and your container will start up and be accessible via HTTP and HTTPS! ## B) Prologue and Nginx in separate containers via docker-compose - + You can have your proxy (nginx) and your webserver (prologue) also in different images and thus different containers, that can communicate. -For this you need to set it up so they get started together, with the ports they need opened and volumes they need mounted etc. +For this you need to set it up so they get started together, with the ports they need opened and volumes they need mounted etc. That's where you use [docker-compose](https://docs.docker.com/get-started/08_using_compose/). First, add 2 directories to `./buildFiles` to separate the config files and dockerfiles of your proxy and your webserver: -- `./buildFiles/nginx` - - Move the `nginx.conf` here -- `./buildFiles/prologue` - - Move the `settings.json` here - - Move your application binary here +- `./buildFiles/nginx` + - Move the `nginx.conf` here +- `./buildFiles/prologue` + - Move the `settings.json` here + - Move your application binary here Also, change your commands or nimble tasks for compiling to output into the `./buildFiles/prologue` directory. ### Write your `docker-compose.yml` + A docker compose file contains the same things you would write in a `docker run` command: name of the container, the image to use, which volumes to mount, which ports to expose etc. Here an example of such a `docker-compose.yml` file that you can keep in your projects main folder: + ```txt version: "3.4" services: @@ -417,11 +421,11 @@ services: - /media:/media - :/var/log/nginx container_name: - + #Prologue webserver that receives requests on port 8080 prologue: image: - expose: + expose: - "8080" # Annotation for readability to make it clear that this container should be talked to on port 8080 volumes: - /media:/media @@ -430,10 +434,12 @@ services: To run this docker compose file with `docker-compose up`, you will first need to build images with the names you specify up there. -This means you now need 2 dockerfiles that can build these images. +This means you now need 2 dockerfiles that can build these images. ### Write your nginx dockerfile + An example for a dockerfile of an alpine image with nginx: + ```txt FROM alpine @@ -454,7 +460,7 @@ RUN mkdir /media CMD ["nginx", "-g", "daemon off;"] ``` -Since the container only contains nginx, we can use `CMD ["nginx", "-g", "daemon off;"]` instead of a bash script to start it. +Since the container only contains nginx, we can use `CMD ["nginx", "-g", "daemon off;"]` instead of a bash script to start it. Put the dockerfile in `./buildFiles/nginx` @@ -463,7 +469,9 @@ Main reason being that the image appears to be binary-incompatible with the ngin Other than that, the official nginx image can be used just as well. ### Write your prologue dockerfile + An example for a dockerfile of an alpine image with our prologue application: + ```txt FROM alpine @@ -482,11 +490,13 @@ RUN mkdir /media #Startup command CMD ["/"] ``` + Since the container only contains our application, we can just start it via `CMD ["/"]`. Put the dockerfile in `./buildFiles/prologue`. ### Build the docker images + With the dockerfiles ready, we can create the images via the following commands from the applications root directory: ```sh @@ -496,19 +506,21 @@ sudo docker build --file ./buildFiles/nginx/dockerfile --tag sudo docker build --file ./buildFiles/nimstoryfont/dockerfile --tag ./buildFiles/nimstoryfont # Stores your images in your current working directory -sudo docker save -o nginx-image.tar +sudo docker save -o nginx-image.tar sudo docker save -o prologue-image.tar ``` Once created, move that image file to your server, e.g. via `scp`. ### Run the docker image on your server + After copying your `docker-compose.yml`, the `nginx-image.tar` and the `prologue-image.tar` to your server, you can deploy your application. Simply load the images into docker and run `docker-compose up` (or `docker-compose restart`). There is no need to specify volumes or ports, as that is already in the `docker-compose.yml`. You may want to write yourself a small script that loads the images and restarts: + ```sh #!/bin/sh sudo docker load -i .tar @@ -520,32 +532,35 @@ sudo docker-compose restart Run the command and your containers will start up and be accessible via HTTP and HTTPS! # Known issues + ## Compile your binary (under `bitnami/minideb` - `error "/lib/x86_64-linux-gnu/libc.so.6: version 'GLIBC_' not found"` + You will run into this issue if your local `glibc` version is more up-to-date than the one `bitnami/minideb` has access to. This is the case because during compilation your binary is dynamically linked with your local `glibc` version. -That means in order to run, it expects the environment that executes it to have *at least* that same glibc version. +That means in order to run, it expects the environment that executes it to have _at least_ that same glibc version. To fix this, you need to link your binary to an older glibc version when compiling, even though your own version is newer. [Doing so is not straightforward.](https://stackoverflow.com/questions/2856438/how-can-i-link-to-a-specific-glibc-version) ### Solution 1: Using zig + The simplest way is installing the [compiler og the zig programming language](https://ziglang.org/download/), as it contains a Clang compiler, which you can tell which glibc version to use. The steps go as follows: -- Install zig -- Write a bashscript called `zigcc` +- Install zig +- Write a bashscript called `zigcc` ```sh #!/bin/sh zig cc $@ ``` -- Move `zigcc` to somewhere on your path, e.g. `/usr/local/bin`. -This is required since the nim compiler does not tolerate spaces in commands that call compilers. -- Write yourself a bashscript with a command to compile your project. -This can't be done via nimble tasks since the syntax is not allowed within nimble tasks. -Replace the "X.XX" with the glibc version that you want as well as the other placeholders. +- Move `zigcc` to somewhere on your path, e.g. `/usr/local/bin`. + This is required since the nim compiler does not tolerate spaces in commands that call compilers. +- Write yourself a bashscript with a command to compile your project. + This can't be done via nimble tasks since the syntax is not allowed within nimble tasks. + Replace the "X.XX" with the glibc version that you want as well as the other placeholders. ```sh #!/bin/sh @@ -556,20 +571,18 @@ nim c \ --clang.linkerexe="zigcc" \ --passC:"-target x86_64-linux-gnu.X.XX -fno-sanitize=undefined" \ --passL:"-target x86_64-linux-gnu.X.XX -fno-sanitize=undefined" \ ---forceBuild:on \ ---opt:speed \ --deepcopy:on \ --mm:orc \ --define:release \ --define:lto \ --define:ssl \ ---outdir:"." \ src/.nim ``` -- Run projectCompile.sh +- Run projectCompile.sh ### Solution 2: Create a compilation environment -Instead of using zig, you can set up a second docker container that contains the glibc version you want, gcc, nim, nimble and the C-libs you require. + +Instead of using zig, you can set up a second docker container that contains the glibc version you want, gcc, nim, nimble and the C-libs you require. You can then mount your project-folder via [docker volume](https://docs.docker.com/storage/volumes/) in the container and compile as normal. Then, you can just compile your binary within the container as usual, your normal compilation command.