Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linux Install Documentation improvement & latest drivers #54

Merged
merged 13 commits into from
Oct 15, 2024
13 changes: 7 additions & 6 deletions scripts/linux/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Copyright (C) 2023 Advanced Micro Devices, Inc. All rights reserved.
# SPDX-License-Identifier: MIT

FROM ubuntu:24.04 as base
FROM ubuntu:24.04

ARG BUILD_TEMPDIR
ARG LIC_MAC=00:00:00:00:00:00
Expand All @@ -16,7 +16,7 @@ SHELL ["/bin/bash", "-c"]
RUN apt-get update
RUN apt-get install -y sudo adduser perl

RUN deluser --remove-home $(getent passwd | awk -F: -v uid="$USER_ID" '$3 == uid {print $1}') || echo "Ignoring error, as uid:${USER_ID} does not exist"
RUN deluser --remove-home $(getent passwd | awk -F: -v uid="$USER_ID" '$3 == uid {print $1}') || echo "Ignoring error, as uid:${USER_ID} does not exist"
RUN groupadd -g ${GROUP_ID} ${GROUP_NAME} || echo "Ignoring error, as ${GROUP_NAME} already exists"
RUN useradd -u $USER_ID -g $GROUP_ID -ms /bin/bash riallto
RUN adduser riallto sudo
Expand All @@ -38,8 +38,8 @@ ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8
RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC sudo -E apt-get install -y tzdata

RUN git clone https://github.com/amd/xdna-driver.git --recursive --depth=1
RUN sudo chmod +x /home/riallto/xdna-driver/xrt/src/runtime_src/tools/scripts/xrtdeps.sh
RUN git clone https://github.com/amd/xdna-driver.git --recursive --depth=1
RUN sudo chmod +x /home/riallto/xdna-driver/xrt/src/runtime_src/tools/scripts/xrtdeps.sh
RUN sudo /home/riallto/xdna-driver/xrt/src/runtime_src/tools/scripts/xrtdeps.sh -docker

COPY $BUILD_TEMPDIR/root/debs npu_ubuntu_debs
Expand All @@ -50,7 +50,7 @@ RUN sudo apt-get install -y python3-pip ocl-icd-opencl-dev uuid-dev
RUN sudo -E dpkg -i npu_ubuntu_debs/xrt_*-amd64-xrt.deb
RUN sudo -E dpkg -i npu_ubuntu_debs/xrt_plugin*-amdxdna.deb

RUN git clone https://github.com/AMDResearch/Riallto -b v1.1 --depth=1
RUN git clone https://github.com/AMDResearch/Riallto --depth=1

SHELL ["/bin/bash", "-l", "-c"]

Expand Down Expand Up @@ -85,10 +85,11 @@ RUN sudo -E echo 'sudo ip link set vmnic0 addr ${LIC_MAC} || true' >> /opt/mlir_

# Install Jupyterhub/notebooks
RUN source ~/.bashrc && python3 -m pip install jupyterhub
RUN source ~/.bashrc && python3 -m pip install notebook
RUN source ~/.bashrc && python3 -m pip install notebook

#Cleanup a bit
RUN rm -rf /home/riallto/*.tar.gz
ENV SHELL=/bin/bash

CMD ["/bin/bash", "-l", "-c", "jupyterhub-singleuser", "--allow-root"]

89 changes: 75 additions & 14 deletions scripts/linux/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,28 +20,89 @@ You can follow the steps [here](https://docs.docker.com/engine/install/ubuntu/).

3. __Obtain a license file for Riallto.__

Please follow the [guide here](https://riallto.ai/prerequisites-aie-license.html#prerequisites-aie-license)
Please follow the [guide here](https://riallto.ai/prerequisites-aie-license.html#prerequisites-aie-license) to get the license.

4. __Disable secure boot from your BIOS settings.__ For now we are using an unsigned kernel version requiring that secure boot is disabled before it can be used. To disable secure boot there is a [guide](https://learn.microsoft.com/en-us/windows-hardware/manufacture/desktop/disabling-secure-boot?view=windows-11) from Microsoft here, but often the steps depend on your hardware manufacturer.
4. __Disable secure boot from your BIOS settings.__

5. Clone this the Riallto repository and then navigate to the installation script `git clone https://github.com/AMDResearch/Riallto.git && cd Riallto/scripts/linux/`.
For now we are using an unsigned kernel version requiring that secure boot is disabled before it can be used. To disable secure boot there is a [guide](https://learn.microsoft.com/en-us/windows-hardware/manufacture/desktop/disabling-secure-boot?view=windows-11) from Microsoft here, but often the steps depend on your hardware manufacturer.

6. __Run `./setup_riallto_linux.sh <your license file>`.__
This command will check the kernel version and if the xdna-driver has been installed. If the kernel is not 6.10 or the NPU device drivers are missing it will build them within a docker and install them on the host machine. This takes about 10 minutes to run and after completing successfully the user will be asked to restart.
5. __Clone the `Riallto` repository and then navigate to the installation script.__

```sh
git clone https://github.com/AMDResearch/Riallto.git -b v1.1
cd Riallto/scripts/linux/
```

```{note}
We recommend using the tag v1.1 to get the verified version of Riallto. Install from `main` to get the latest drivers and code, however, this is not thoroughly verified.
```

6. __Upgrade Linux Kernel.__

```sh
./setup_riallto_linux.sh <your license file> <username (optional)>
```

This command will check the kernel version and if the `xdna-driver` has been installed. If the Linux kernel is not `6.10` or the NPU device drivers are missing, it will build them within a docker and install them on the host machine. This takes about 10 minutes to run and after completing successfully the user will be asked to restart.

7. __Reboot the machine.__
To finish upgrading the kernel to `6.10`.

8. __Re run the `./setup_riallto_linux.sh <your license file>` script.__
This will build the Riallto docker and will take about 20 minutes.
This will finish upgrading the Linux kernel to `6.10`.

8. __Install Riallto by creating its Docker container.__

```sh
./setup_riallto_linux.sh <your license file> <username (optional)>
```

This will build the Riallto Docker and will take about 20 minutes.

```{note}
You can specify a username to indicate the ownership of the files created inside the Docker container. If no username is specified, the default username is `$USER`.
```

## Running Tests

Verify the correct installation of Riallto. Inside the `Riallto/scripts/linux` directory, you will find the `run_pytest.sh` script.

```sh
./run_pytest.sh
```

This script will run a suit of tests using `pytest` to verify the operation of your NPU device and the Riallto installation. This will take about 50 minutes.

## Running Riallto

Launch a Jupyterlab server from a Docker container allowing you to use Riallto notebooks from the provided directory (usually Riallto/notebooks). Inside the `Riallto/scripts/linux` directory, you will find the `launch_jupyter.sh` script.

```sh
./launch_jupyter.sh ../../notebooks
```

### Running from different users

Once the Riallto Docker container is created for one user, other users can run Riallto without having to re install. Other users do not need elevated privileges, but they need to be part of the `docker` group.

## Running Riallto / Running Tests
Inside this directory there are a few scripts.
1. Make sure user is in the docker group.

* `launch_jupyter.sh <notebooks directory>` - will launch a jupyterlab server from a docker container allowing you to use Riallto notebooks in the directory provided (usually Riallto/notebooks).
* `run_pytest.sh` - will run a suit of pytests to test the operation of your NPU device and the Riallto installation.
```sh
sudo usermod -aG docker <user> ; exit
```

2. Clone the Riallto repository and then navigate to the Linux scripts directory.

```sh
git clone https://github.com/AMDResearch/Riallto.git -b v1.1
cd Riallto/scripts/linux/
```

3. Launch a Jupyterlab server using Riallto notebooks.

```sh
./launch_jupyter.sh ../../notebooks
```

### FAQ
## FAQ

### If you get the following error on boot, `error: bad shim signature`

Expand All @@ -60,7 +121,7 @@ If you see `loading /lib/firmware/amdnpu/1502_00/npu.sbin failed with error -22`
Delete the NPU firmware:

```sh
sudo rm -rf /lib/firmware/amdnpu/1502_00/npu.sbin
sudo rm -rf /lib/firmware/amdnpu/
```

Then, start the installation from scratch.
16 changes: 11 additions & 5 deletions scripts/linux/setup_riallto_linux.sh
Original file line number Diff line number Diff line change
Expand Up @@ -73,8 +73,8 @@ fi
# Check to make sure that a license file has been provided and that
# a MAC address can be extracted from it for adding into the docker
# image
if [ "$#" -ne 1 ]; then
echo "Usage $0 <Xilinx license file>"
if [ "$#" -lt 1 ]; then
echo "Usage $0 <Xilinx license file> <username (optional)>"
exit 1
fi
LIC_FILE="$1"
Expand Down Expand Up @@ -176,9 +176,15 @@ build_tmp=./_work
rm -rf $build_tmp
mkdir -p $build_tmp

USER_ID=`id -u`
GROUP_ID=`id -g`
GROUP_NAME=`id -g -n`
USER_NAME="$USER"

if [ $# -eq 2 ]; then
USER_NAME="$2"
fi

USER_ID=`id -u $USER_NAME`
GROUP_ID=`id -g $USER_NAME`
GROUP_NAME=`id -g -n $USER_NAME`

## Checks to make sure that all the required tarballs and license are in the directory
if [ ! -f "./pynqMLIR-AIE.tar.gz" ]; then
Expand Down
10 changes: 5 additions & 5 deletions scripts/linux/xdna-driver-builder/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Copyright (C) 2023 Advanced Micro Devices, Inc. All rights reserved.
# SPDX-License-Identifier: MIT

FROM ubuntu:24.04 as base
FROM ubuntu:24.04

ARG DEBIAN_FRONTEND=noninteractive
ARG KERNEL_HEADERS
Expand All @@ -24,8 +24,8 @@ RUN apt-get install -y zstd
COPY _work /root/

RUN apt-get install -y libelf1t64
RUN apt-get install -y kmod
RUN apt-get install -y linux-base
RUN apt-get install -y kmod
RUN apt-get install -y linux-base

RUN dpkg -i /root/$KERNEL_HEADERS
RUN dpkg -i /root/$KERNEL_HEADERS_GENERIC
Expand All @@ -34,9 +34,9 @@ RUN dpkg -i /root/$KERNEL_IMAGE

RUN git config --global url."https://github.com/".insteadOf "[email protected]:"
RUN cd /root && git clone https://github.com/amd/xdna-driver.git --recursive
RUN cd /root/xdna-driver && git checkout 7682e0b4b6c435d6700faef3dc27cd709324e27f
RUN cd /root/xdna-driver && git checkout main

RUN apt-get install -y cmake jq pkg-config wget libdrm-dev
RUN apt-get install -y cmake jq pkg-config wget libdrm-dev
RUN apt-get install -y python3-pip
RUN apt-get install -y ocl-icd-opencl-dev

Expand Down
Loading