update documentation

docs/_data/docs.yml

@@ -20,6 +20,7 @@
 
 - title: Getting Started
   docs:
+    - ansible-installing
     - installing
     - quick-start
     - uninstalling

docs/_docs/ansible_installing.md

@@ -0,0 +1,102 @@
+---
+title: Installing CSLE
+permalink: /docs/ansible-installing/
+---
+
+## Installing CSLE
+
+This section contains instructions for installing CSLE. A video that walks through the installation process is
+available at https://www.youtube.com/watch?v=l_g3sRJwwhc.
+
+### Prerequisites
+
+To install and run CSLE you need at least one server or virtual machine that meets the following criteria:
+
+- Ubuntu 18.04+;
+- at least 16GB RAM (the exact amount of RAM necessary depends on the emulations to deploy; 16GB is sufficient for the smallest emulations, e.g., in the size of 5-10 containers);
+- at least 2 CPUs;
+- 50 GB of free hard-disk space;
+- outside Internet access;
+- and a UNIX user account with `sudo` privileges.
+
+Below are a list of dependencies that will be installed with CSLE (unless they are already installed):
+
+- Docker 20.10.14+;
+- Python 3.9+;
+- Python libraries: `torch`, `numpy`, `gym`, `docker`, `paramiko`, `stable-baselines3`, `psycopg`, `pyglet`, `flask`, `click`, `waitress`, `scp`, `psutil`, `grpcio`, `grpcio-tools`, `scipy`, `confluent-kafka`, `requests`, `pyopenssl`, `sphinx`, `mypy`, `mypy-extensions`, `mypy-protobuf`, `types-PyYAML`, `types-protobuf`, `types-paramiko`, `types-requests`, `types-urllib3`, `flake8`, `pytest`, `gevent`, `eventlet`, `dnspython`, `csle-ryu-fork`, `gpytorch`, `pulp`, `Bayesian optimization`, `emukit`, `cma`, `pycryptodome`.
+- PostgreSQL 12+;
+- `make`, `git`, `bzip2`, `build-essential`;
+- Prometheus 2.23+;
+- Node exporter 1.0.1+;
+- cAdvisor (any version);
+- Grafana (any version);
+- Node v16.13.1+;
+- and `npm` v6.14.8+.
+
+
+### Running the Installation
+
+The network setup for a minimal installation includes (i) a leader server; (ii)
+zero or more worker servers; and (iii) a user host, e.g., a laptop or workstation
+(see Fig. 26.)
+
+<p align="center">
+<img src="./../../img/ansible_installation_setup.png" width="70%">
+<p class="captionFig">
+Figure 26: Network topology of a typical CSLE installation; the cluster consists of a 
+leader and zero or more workers; the installation commands are executed from an 
+external user host that runs Ansible; the servers are connected through a private 
+network as well as to the Internet; the user host has access to the private network.
+</p>
+</p>
+
+Start by setting up SSH keys so that a) all servers (leader and workers) have 
+SSH access to each other without requiring a password; and b) the user host from which 
+the installation will be executed has SSH access to all servers. To do this, start by 
+generating an SSH key pair on each host using the command ssh-keygen. 
+After generating the keys, copy the public key of each host (e.g., `id_rsa.pub`) 
+to the file `.ssh/authorized_keys` on the other hosts.
+
+Next, clone the CSLE git repository on the user host by running the following command.
+
+```bash
+git clone https://github.com/Limmen/csle
+```
+
+<p class="captionFig">
+Listing 5: Commands to clone the CSLE git-repository.
+</p>
+
+Now configure the installation by setting the parameters in the files:
+
+- `csle/ansible/inventory`
+- `csle/ansible/groups_vars/all/variables.yml`
+
+In most cases the default parameters can be used. However, you must always configure the 
+IP addresses of the servers in your cluster in `csle/ansible/inventory`
+as well as configure the variables `leader_ip`, `leader_public_ip`, `metastore_ip`,
+and cluster_nodes in the file `csle/ansible/groups_vars/all/variables.yml`.
+
+After configuring the installation, install Ansible on the user host by running the command:
+
+```bash
+pip install ansible
+```
+
+<p class="captionFig">
+Listing 6:  Command to install Ansible.
+</p>
+
+Finally, start the automated installation by running the following commands on the user host:
+
+```bash
+cd csle/ansible; ansible-playbook --ask-become-pass install.yml
+```
+
+<p class="captionFig">
+Listing 7:   Command to run the automated installation of CSLE with Ansible.
+</p>
+
+The above command will ask for a sudo password to the servers. 
+After entering the password, Ansible will install all components of CSLE on all
+servers in your cluster. This process usually takes around 40 minutes.

docs/_docs/background.md

@@ -7,7 +7,9 @@ permalink: /docs/background/
 
 The development of CSLE started as a research project in 2020 at KTH Royal Institute of Technology, 
 led by Prof. Rolf Stadler and Prof. Pontus Johnson. 
-This research has been supported in part by the Swedish armed forces and is 
-conducted at the division of Network and Systems Engineering (NSE) and 
-the KTH Center for Cyber Defense and Information Security (CDIS). 
+This research has been supported in part by the Swedish armed forces 
+as well as the Defense Advanced Research Project Agency (DARPA) 
+through the CASTLE program under Contract No. w912cg23c0029. 
+The research is conducted at the division of Network and Systems Engineering (NSE) 
+and the KTH Center for Cyber Defense and Information Security (CDIS).  
 The project is currently maintained by Kim Hammar (*[email protected]*).

docs/_docs/code_analysis.md

@@ -17,7 +17,7 @@ To run the Python linter on the CSLE code base, execute the command:
 ```
 
 <p class="captionFig">
-Listing 138: Command to run the Python linter.
+Listing 137: Command to run the Python linter.
 </p>
 
 Alternatively, the following commands can be executed for the same effect as the command above:
@@ -30,7 +30,7 @@ flake8 examples/
 ```
 
 <p class="captionFig">
-Listing 139: Commands to run the Python linter.
+Listing 138: Commands to run the Python linter.
 </p>
 
 To run the Python static type checker, execute the command:
@@ -40,7 +40,7 @@ simulation-system/libs/type_checker.sh
 ```
 
 <p class="captionFig">
-Listing 140: Command to run the Python static type checker.
+Listing 139: Command to run the Python static type checker.
 </p>
 
 To run the JavaScript linter, run the commands:
@@ -51,5 +51,5 @@ cd management-system/csle-mgmt-webapp/; npm run lint:fix
 ```
 
 <p class="captionFig">
-Listing 141: Commands to run the JavaScript linter.
+Listing 140: Commands to run the JavaScript linter.
 </p>

docs/_docs/development_conventions.md

@@ -68,7 +68,7 @@ csle/.flake8
 ```
 
 <p class="captionFig">
-Listing 119: Configuration file for the `flake8` Python linter.
+Listing 118: Configuration file for the `flake8` Python linter.
 </p>
 
 Configuration file for the `flake8` Python linter.
@@ -78,7 +78,7 @@ csle/management-system/csle-mgmt-webapp/.eslintrc.json
 ```
 
 <p class="captionFig">
-Listing 120: Configuration file for the `eslint` JavaScript linter.
+Listing 119: Configuration file for the `eslint` JavaScript linter.
 </p>
 
 
@@ -119,7 +119,7 @@ def stop_all_executions() -> None:
 ```
 
 <p class="captionFig">
-Listing 121: Example of a Python function with a comment.`
+Listing 120: Example of a Python function with a comment.`
 </p>
 
 Example of a comment to a JavaScript function:
@@ -146,7 +146,7 @@ const convertListToCommaSeparatedString = (listToConvert) => {
 ```
 
 <p class="captionFig">
-Listing 122: Example of a JavaScript function with a comment.
+Listing 121: Example of a JavaScript function with a comment.
 </p>
 
 ### Unit and Integration Testing
@@ -221,7 +221,7 @@ to code servers (i.e., DockerHub and PyPi).
 <p align="center">
 <img src="./../../img/ci_pipeline.png" width="75%">
 <p class="captionFig">
-Figure 29: The continuous integration (CI) pipeline of CSLE.
+Figure 30: The continuous integration (CI) pipeline of CSLE.
 </p>
 </p>
 

docs/_docs/examples.md

@@ -50,7 +50,7 @@ traces = MetastoreFacade.list_emulation_traces()
 ```
 
 <p class="captionFig">
-Listing 111: Example of collecting attack and system traces with CSLE.
+Listing 114: Example of collecting attack and system traces with CSLE.
 </p>
 
 ### System Identification
@@ -94,7 +94,7 @@ gaussian_mixture_system_model=system_model)
 ```
 
 <p class="captionFig">
-Listing 112: Example of system identification through expectation maximization with CSLE.
+Listing 115: Example of system identification through expectation maximization with CSLE.
 </p>
 
 ### Strategy Training
@@ -142,7 +142,7 @@ for policy in experiment_execution.result.policies.values():
 ```
 
 <p class="captionFig">
-Listing 113: Example of strategy optimization through the Proximal Policy Optimization (PPO) reinforcement learning algorithm in CSLE.
+Listing 116: Example of strategy optimization through the Proximal Policy Optimization (PPO) reinforcement learning algorithm in CSLE.
 </p>
 
 ### Strategy Evaluation
@@ -181,6 +181,6 @@ StoppingGamePomdpDefenderEnv.emulation_evaluation(
 ```
 
 <p class="captionFig">
-Listing 114: Example of evaluating learned security strategies in the emulation system of CSLE.
+Listing 117: Example of evaluating learned security strategies in the emulation system of CSLE.
 </p>
 

docs/_docs/how_to_base_container.md

@@ -11,34 +11,34 @@ To add a base container with the name "`my_container`", do the following steps:
      csle/emulation-system/base_images/docker_files
    ```
    <p class="captionFig">
-   Listing 153: Directory with base Docker images.
+   Listing 152: Directory with base Docker images.
    </p>
 2. Add a Dockerfile to the following directory:
     ```bash
      csle/emulation-system/base_images/docker_files/my_container
     ```
    <p class="captionFig">
-   Listing 154: Directory with the Dockerfile of a container with the name `my_container`.
+   Listing 153: Directory with the Dockerfile of a container with the name `my_container`.
    </p>
 3. Add build and push instructions to the following Makefile:
     ```bash
       csle/emulation-system/base_images/Makefile
     ```
    <p class="captionFig">
-   Listing 155: Makefile for base Docker images.
+   Listing 154: Makefile for base Docker images.
    </p>
 4. Update the following README file:
      ```bash
      csle/emulation-system/base_images/README.md
      ```
    <p class="captionFig">
-   Listing 156: README file for base Docker images.
+   Listing 155: README file for base Docker images.
    </p>
 5. Build and push the image to DockerHub by running the commands:
     ```bash
      cd csle/emulation-system/base_images; make my_container
      cd csle/emulation-system/base_images; make push_my_container
     ```
    <p class="captionFig">
-   Listing 157: Commands to build and push the base Docker image `my_container` to DockerHub.
+   Listing 156: Commands to build and push the base Docker image `my_container` to DockerHub.
    </p>

docs/_docs/how_to_config_param.md

@@ -12,12 +12,12 @@ To add a new configuration parameter, perform the following steps:
      csle/simulation-system/libs/csle-common/src/csle_common/dao/emulation_config/config.py
     ```
    <p class="captionFig">
-   Listing 180: Python file that manages the CSLE configuration.
+   Listing 179: Python file that manages the CSLE configuration.
    </p>
 3. Add the new parameter to an appropriate class in the following file (add a new class if needed):
     ```bash
       csle/simulation-system/libs/csle-common/src/csle_common/constants/constants.py
     ```
    <p class="captionFig">
-   Listing 181: Python file with constants in `csle-common`.
+   Listing 180: Python file with constants in `csle-common`.
    </p>

docs/_docs/how_to_derived_container.md

@@ -11,34 +11,34 @@ To add a derived container with the name "`my_container`", perform the following
      csle/emulation-system/derived_images/
     ```
    <p class="captionFig">
-   Listing 158: Directory with derived Docker images.
+   Listing 157: Directory with derived Docker images.
    </p>
 2. Add a Dockerfile inside the directory:
     ```bash
      csle/emulation-system/derived_images/my_container/
     ```
    <p class="captionFig">
-   Listing 159: Directory with the Dockerfile for a derived Docker image with the name `my_container`.
+   Listing 158: Directory with the Dockerfile for a derived Docker image with the name `my_container`.
    </p>
 3. Add build and push instructions to the following Makefile:
     ```bash
      csle/emulation-system/derived_images/Makefile
     ```
    <p class="captionFig">
-   Listing 160: Makefile for derived Docker images.
+   Listing 159: Makefile for derived Docker images.
    </p>
 4. Update the following README file:
     ```bash
      csle/emulation-system/derived_images/README.md
     ```
    <p class="captionFig">
-   Listing 161: README file for derived Docker images.
+   Listing 160: README file for derived Docker images.
    </p>
 5. Build and push the image to DockerHub by running the commands:
     ```bash
      cd csle/emulation-system/derived_images; make my_container
      cd csle/emulation-system/derived_images; make push_my_container
     ```
    <p class="captionFig">
-   Listing 162: Commands to build and push the derived Docker image `my_container` to DockerHub.
+   Listing 161: Commands to build and push the derived Docker image `my_container` to DockerHub.
    </p>

docs/_docs/how_to_emulation_config.md

@@ -11,33 +11,33 @@ To add a new emulation configuration with the name "`level_13`" and version 0.1.
       csle/emulation-system/envs/010/
     ```
    <p class="captionFig">
-   Listing 163: Directory with emulation configurations with version 0.1.0.
+   Listing 162: Directory with emulation configurations with version 0.1.0.
    </p>
 2. Add the emulation configuration file `config.py` to the directory:
     ```bash
       csle/emulation-system/envs/010/level_13/
     ```
    <p class="captionFig">
-   Listing 164: Directory with emulation configuration file for the emulation with the name `level_13` and version 0.1.0.
+   Listing 163: Directory with emulation configuration file for the emulation with the name `level_13` and version 0.1.0.
    </p>
 3. Update the following README file:
     ```bash
       csle/emulation-system/envs/010/README.md
     ```
    <p class="captionFig">
-   Listing 165: README file for emulation configurations with version 0.1.0.
+   Listing 164: README file for emulation configurations with version 0.1.0.
    </p>
 4. Update the following Makefile with installation instructions:
     ```bash
       csle/emulation-system/envs/010/README.md
     ```
    <p class="captionFig">
-   Listing 166: Makefile for emulation configurations with version 0.1.0.
+   Listing 165: Makefile for emulation configurations with version 0.1.0.
    </p>
 5. Insert the emulation configuration into the metastore by running the command:
      ```bash
       csle/emulation-system/envs; make install
      ```
    <p class="captionFig">
-   Listing 167: Command to insert emulation configurations into the metastore.
+   Listing 166: Command to insert emulation configurations into the metastore.
    </p>

docs/_docs/how_to_monitoring_agent.md

@@ -11,14 +11,14 @@ To update the monitoring agents that run on the emulated devices, perform the fo
       csle/simulation-system/libs/csle-collector/src/
     ```
    <p class="captionFig">
-   Listing 173: Directory with source code of the monitoring agents in CSLE.
+   Listing 172: Directory with source code of the monitoring agents in CSLE.
    </p>
 2. Update the gRPC API by editing the protocol-buffer files in the directory:
     ```bash
       csle/simulation-system/libs/csle-collector/protos/
     ```
    <p class="captionFig">
-   Listing 174: Directory with protocol buffers for the monitoring agents in CSLE.
+   Listing 173: Directory with protocol buffers for the monitoring agents in CSLE.
    </p>
 3. Regenerate the gRPC Python files by running the commands:
     ```bash
@@ -33,5 +33,5 @@ To update the monitoring agents that run on the emulated devices, perform the fo
      python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/ryu_manager/. ./protos/ryu_manager.proto
     ```
    <p class="captionFig">
-   Listing 175: Commands to generate gRPC Python files from protobuf files.
+   Listing 174: Commands to generate gRPC Python files from protobuf files.
    </p>