Intelligent Control Systems: Practical Assignment

The assignment can be completed either locally on your own machine (natively on macOS / Ubuntu or inside Docker container) or in the cloud leveraging GitHub Codespaces. Accordingly, either follow the instructions in Section 1, Section 2, or Section 3.

1. Native installation on macOS / Ubuntu

1.1 Requirements

As our codebase relies on JAX, we natively only support Linux x86_64 or macOS hosts (both x86_64 / Intel & arm64 / Apple Silicon). We therefore strongly recommend to use either one of the two supported host environment, or complete the assignment on GitHub Codespaces / a Dev container instead.

Furthermore, you can either use our provided scripts to install the required dependencies in a Conda environment (Option 1), or install all packages manually (Option 2). We strongly recommend to use Conda, as this will allow you to easily select the desired Python version and prevent any version conflicts.

1.1.1 Important note for Windows users

Windows is not officially supported by JAX, which we depend on in this codebase. We quote from the JAX README:

Windows users can use JAX on CPU and GPU via the Windows Subsystem for Linux. In addition, there is some initial community-driven native Windows support, but since it is still somewhat immature, there are no official binary releases and it must be built from source for Windows. For an unofficial discussion of native Windows builds, see also the Issue #5795 thread.

While it is possible to install Linux Subsystem for Windows or alternatively configure a dual boot setup with Windows & Ubuntu, we are not able to offer any assistance with this.

1.2 Option 1: Installation using Conda

We primarily support the installation of the required dependencies using Conda. Please first install the latest version of Conda or Miniconda using the instructions on the Conda website.

1.2.1 Create a new Conda environment and install dependencies

Then, run our bash script to create the new Conda environment ics with all required dependencies:

./00-conda-setup.sh

If you encounter permission issues, please run chmod +x /00-conda-setup.sh to give executable permissions to the bash script.

1.2.2 Activate the Conda environment and add assignment folder to PYTHONPATH

Subsequently, you can activate the Conda environment ics by running:

conda activate ics && ./02-add-to-pythonpath.sh

1.2.3 Install PyTorch with GPU support (optional)

If you want to leverage a NVIDIA GPU for increasing the neural network training speed in Problem 1, you need to install PyTorch with GPU support. Please note that we will not offer any support for installing PyTorch with GPU support and it is totally up to your discretion to follow this installation step as the assignment can also be completed solely using the CPU.

As documented in the PyTorch Get Started guide, please run in the ics Conda environment:

conda install pytorch torchvision pytorch-cuda=11.7 -c pytorch -c nvidia

1.2.4 Install JAX with GPU support (optional)

If you want to leverage a NVIDIA GPU for increasing the LNN training speed in Problem 2c, you need to install JAX with GPU support. Please note that we will not offer any support for installing JAX with GPU support and it is totally up to your discretion to follow this installation step as the assignment can also be completed solely using the CPU.

Please run in the ics Conda environment:

conda install jax cuda-nvcc -c conda-forge -c nvidia

In case you are encountering issues with the installation or if JAX does not find your GPU, please refer to the JAX README.

1.3 Option 2: Manual installation

This framework requires Python 3.10. Please note that some required dependencies might not be updated yet to work with Python 3.11.

1.3.1 Install ffmpeg

FFmpeg is required to create Matplotlib animations and save them as .mp4 video files. Please follow installation instructions online such as this one. On Ubuntu, the package can be easily installed via:

sudo apt update && apt install -y ffmpeg 

1.3.2 Install Python dependencies

You can install the jax_double_pendulum package and all necessary dependencies by running the following command in the top level directory of the repository:

pip install .

1.3.3 Add assignment folder to PYTHONPATH

As we import some Python modules from folders outside a package, we need to add the assignment folder to the PYTHONPATH environment variable.

export PYTHONPATH=$PYTHONPATH:$(pwd)

If you encounter any issues with the installation of JAX, it is recommended to follow the installation instructions in the JAX repository.

1.4 Usage

Please don't forget, if applicable, to activate the Conda environment before running any scripts.

Generally, all Python scripts can be executed from the top level directory of the repository. For example:

python examples/main.py

For Jupyter notebooks, you can start use our script to start a Jupyter notebook server in the terminal:

./10-start-notebook-as-student.sh

2. GitHub Codespaces

A codespace is a development environment that's hosted in the cloud. Different instance types ranging from 2 CPU cores, 4 GB of RAM, and 32 GB of storage and up to 16 CPU cores, 32 GB of RAM and 128 GB of storage are available.

Important: 180 core-hours of GitHub Codespaces usage per month are included for free in GitHub Pro, which is offered as part of the GitHub Student Developer Pack. If you haven't already, please register for the pack using your TU Delft email address to get access to this free usage quota. This will be sufficient for 90 hours per month of continuous usage with the smallest 2-core instance type.

2.1 Accessing the code for students

As you are studying this README, you probably know that the code template is available on GitHub in the tud-cor-sr/ics-pa-sv repository. Please click on Use this template and then Create new repository to create a new repository for the assignment solution in your own personal GitHub account. Please make sure to make the repository private.

2.2 Open in GitHub Codespaces

Then, please open the new repository in a GitHub Codespaces instance by clicking on Code -> Open with Codespaces.

2.3 Installation

No worries, all dependencies are automatically installed in the GitHub Codespaces environment. You should be able to start working right away. The python executable is symlinked to /usr/local/bin/python.

If you are encountering issues with missing Python dependencies or you observe a jupyter: command not found error when executing the ./10-start-notebook-as-student.sh bash script, please run the following command in the GitHub Codespaces terminal:

./01-pip-install.sh

Afterwards, the issues should be resolved.

2.4 Usage

You can open the Jupyter notebooks in the editor and then use the integrated Jupyter notebook extension to execute them. If you are prompted to select a kernel, please choose the kernel Python 3.10.x /usr/local/bin/python.

Alternatively, you can also start a Jupyter notebook server in the VS Code terminal, for which port-forwarding should be configured automatically:

./10-start-notebook-as-student.sh

2.5 Commiting & pushing with git

Complimentary to saving your code on the Codespaces instance, you will also want to push the code changes to your GitHub repository, so that your code is not lost when the instance is deleted. We refer to the internet for comprehensive guides on git. In the following, we will only point out the basic usage of pushing code from Codespaces to the main branch of the GitHub repository:

1. Click on Source Control in the left sidebar.
2. Here, you can see all files you have modified / added / deleted. You can add / stage changes by clicking on the + symbol right of the filename.
3. You can commit the Staged Changes by writing a concise message descriping the changes into the text box and then clicking on Commit.
4. Click on Sync Changes or Push to mirror the local commits to the remote GitHub repository.

2.6 Managing the memory usage

A standard 2-core instance of GitHub Codespaces has a memory limit of 4 GB. This memory limit can be exhausted quite quickly, in particular when running neural network trainings. You can monitor the current memory usage of your instance by running htop in the terminal. When the instance runs out of memory, this usually results in a Jupyter notebook kernel crash. You would for example see the following or a very similar error message:

The Kernel crashed while executing code in the the current cell or a previous cell. Please review the code in the cell(s) to identify a possible cause of the failure.

When you are running low on memory, please stop and close any Jupyter notebooks you might not be using at the moment. This will free some memory.

If you keep encountering kernel crashes, we recommend upgrading to a larger GitHub Codespaces instance. One option is the 4-core instance type with 8 GB of memory. The various instance options are documented here. Please note that in the GitHub Student Developer Pack, you have 180 core-hours of Codespaces per month included for free. For a 4-core instance, this evaluates to 45 hours of continuous usage of the instance per month. To change your instance to an 4-core instance, please click on Code -> Codespaces -> On current branch -> ... -> Change machine type. Then, select the 4-core instance type.

2.7 Working with Matplotlib

There are two ways to work with Matplotlib plots within standard Python scripts (i.e. not Jupyter notebooks):

1. Add the following line to the top of your Python script, which allows you to run the script interactively in VS Code.

# %%
# Now you can add the rest of your code here
import matplotlib.pyplot as plt
plt.figure()
plt.show()

Then you can run the script interactively by clicking on the Run Cell button in the top left corner of the code cell.

2. Instead of showing the Matplotlib code, you can also just save the plot to a file and open it in a new tab in VS Code.

import matplotlib.pyplot as plt
plt.figure()
plt.savefig("my_plot.png")

3. Local Dev containers

Visual Studio Code Dev Containers allow you to open the repository in a containerized development environment. It relies on Docker to virtualize a pre-configured Ubuntu system with all necessary dependencies installed. This allows you to execute code also on a natively unsupported host operating system (e.g. Windows) or to avoid installing all dependencies on your host system. Please note that we are currently not providing support for the Dev Container setup.

In the following, we will describe the basic setup and usage of the Dev Container. You find a more comprehensive guide here. Please also consult the Docker documentation for more information on Docker.

3.1 Install Docker

Please install Docker Desktop on Windows / Mac / Ubuntu. If you are one Windows, please make sure to install and enable the WSL 2 backend in the Docker Desktop settings.

3.2 Allocate sufficient resources to Docker

Docker runs in a Virtual Machine (VM) while limiting the available resources such as CPU cores, memory, and storage, which might result in insufficient resources for executing the code in this repository. Accordingly, you might need to allocate more resources to the Docker VM than provided by default.

We recommend at least 4 CPUs, 6 GB of memory, and 32 GB of disk space. However, allocating too many resources to the VM might slow down / freeze your host system.

3.2.1 Configure resource limits on Windows hosts with WSL 2 backend

If you have followed the provided indications and configured Docker to use WSL2, you will find out that you have no access to the advanced settings in Docker that allow you to limit the RAM and number of CPUs of your system (WSL does not provide a means to set the disk space). To change the resources of your containers when Docker is using WSL as backend, you need to make use of a .wslconfig file. To do so, you need to take the following steps (you require Visual Studio Code installed, see section 3.3 of this manual).

• In Visual Studio Code select in the top toolbar File -> New File... and name it .wslconfig. VS Code will prompt you to indicate the location to which you have to save the file. The file should be located under your user folder:

C:\Users\<your_user_name>\.wslconfig

• Once the file is open in VS Code, you need to make sure that the endline character of the file is LF. You can do that in the blue Status Bar at the bottom of the VS Code window. Towards the right there should be a piece of text indicating CRLF or LF. If the text displayed is LF then you don't need to perform any further action. If the text displayed is CRLF, then click on it and select LF in the command palette at the top of the window.

• Copy and paste the following text into your .wslconfig file. Notice that if the computational resources of your computer are limited you may need to reduce some of these values (e.g. if you only have 4GB of RAM, you would need to reduce the memory value, although that can result in issues while running the code). If you have a more powerful computer, feel free to increase the values without forgetting to leave some resources for your host system.

[wsl2]

memory=6GB

processors=4

• Save the changes (Ctrl + S) and close the window. If you have already installed Docker you will need to restart your computer for the changes to be effective.

• You can verify that the changes are taking effect by running the command below. The value displayed directly below total should match the value you have assigned to memory in your .wslconfig file.

free --giga

3.2.2 Configure resource limits on macOS / Ubuntu hosts

Please open Docker Desktop and go to Settings -> Resources -> Advanced and modify the resources as described as in the Docker Desktop macOS and Linux manuals.

3.3 Install Visual Studio Code

Please install Visual Studio Code and subsequently also the Dev Container extension. With the Dev Containers extension installed, you will see a new green status bar item at the far bottom left.

3.4 Launch the Dev Container

Now, please open the folder with the cloned repository in Visual Studio Code. You will see a green status bar item at the far bottom left. Click on it and select Reopen in Container. Alternatively, you might also be automatically prompted to reopen the folder in a container. This will use the settings in the .devcontainer folder to build a new Dev container and then open a new VS Code window with the repository opened in the containerized environment.

3.5 Install Python dependencies

After VS Code has been openend in the container, you will see a terminal window at the bottom of the screen. The process of installing Python dependencies should be started automatically. If not, please run the following command in the terminal:

./01-pip-install.sh

3.6 Usage

You can open the Jupyter notebooks in the editor and then use the integrated Jupyter notebook extension to execute them. If you are prompted to select a kernel, please choose the kernel Python 3.10.x /usr/local/bin/python.

Alternatively, you can also start a Jupyter notebook server in the VS Code terminal, for which port-forwarding should be configured automatically:

./10-start-notebook-as-student.sh

4. Jupyter notebook - tips & tricks

4.1 Reloading functions implemented in another notebook

When changing the content of functions implemented in a Jupyter notebook and used in other notebooks, it might (sometimes) be necessary to save all notebooks and then restarting the notebook kernel(s). This procedure will allow the function in all notebooks relying on it to be re-loaded.

4.2 Validating your implementation

You are able to validate the syntax of your code, the removal of all NotImplementedError exceptions, and the passing of all public tests by running the following command in the top level directory of the repository:

AUTOGRADING=true nbgrader validate assignment/**/**.ipynb