AIonFPGA

AI High-Performance Solution on FPGA

Abstract

In a world of self-driving cars and automated quality control in manufacturing, real-time image classification is becoming increasingly important. Artificial intelligence, and deep learning in particular, are achieving excellent classification accuracies, but there are some challenges.

For one thing, high-resolution image acquisition systems require a lot of processing power. For another, a large labeled dataset of training data is required to train deep convolutional neural networks.

A solution for the former is to use field-programmable gate arrays (FPGAs) as hardware accelerators. Therefore, an embedded system featuring a multiprocessor system-on-chip with an integrated FPGA is deployed. The second problem is approached with data augmentation to artificially increase the size of the labeled dataset.

This allowed the deployed convolutional neural network to achieve a Top-1 accuracy of 97.2 % and a Top-5 accuracy of 99.5 %. In addition, the throughput of the image classification chain reached 41.1 fps for color images of 1280×1024 px.

Using data augmentation significantly improved the real-world classification performance by reducing the impact of ambient light. Furthermore, it completely eliminated the need to collect additional data samples.

Repository Structure

This repository is structured as follows:

.
├── doc                 # Documentation
│   ├── fact-sheet
│   ├── poster
│   ├── presentation
│   ├── project-plan
│   └── thesis          # Bachelor Thesis
│
├── mpsoc               # OS / DPU / Model
│   ├── build-pynq
│   ├── cnn-model       # Quantization
│   ├── dpu
│   └── pynq-setup
│
└── sw                  # Software
    ├── inference
    │   ├── aionfpga    # Inference Application
    │   └── camera      # Camera Library
    ├── packages
    │   └── fhnwtoys    # Package fhnwtoys
    ├── training        # Model
    └── verification

Downloads

Documents

The compiled .pdf files can be downloaded from here directly:

Name	Download	SHA-256 checksum
Project Plan	.pdf	ca5c77def20f017892c571859c7772a77c04b8ebc5d4bef795865e88b3c6474f
Bachelor Thesis	.pdf	818df883f765bed2fa2e500bd5064ae3236327c9e5a2626e270d5e65cc38c8f0
Presentation	.pdf	2e3b39469f03a21843b4881c65a7d82646388275b02431f400497ac3d9e51925
Fact Sheet	.pdf	b85f36d1627cad99a8532ecc9528f8b2400884795de159473b460e02a38514ba
Poster	.pdf	8605bac39041a88e24b22f1d9ff3b48a6822ba567db58a8b4c57e0f313d0df62

Compilation

The .pdf files can be built by running make in the respective directory:

$ make build clean

Software

Name	Download	SHA-256 checksum
Camera Library	.so	f3182088466f0b83008e450eb1ce2bf8fa49996659b8dc2617379498735bf5b2
Saved Model	.zip	d54530104d07541d01fefe949ed7b5767533bcafba8ac65b19a765e1547bea49
Frozen Graph	.pb	2ca336ebf425bb668925599aedb67197557b3082694dfa79aeac130b576a4523
Quantized Model	.elf	9ff451d90434959103951976859c2be538e2d89b06f886ac267ea05bff0ac436
DPU	.bit .hwh .xclbin	478f05f4b1a9a35121b8cf3824cc096ba514157be8aa402950668746027e3d0c 0c64dc4a6f8912893d6fa3967ed937d2ec10abd4e1d81800386e6540eed877ab 29e5adb0218e1ead172f36f8c88f805d3f3d039bb1ccb6041286e0e21ba86dcf

Previous Project

This bachelor thesis is the continuation of the following previous project: P5-AIonFPGA

License

Copyright © 2020 Dominik Müller and Nico Canzani

This project is licensed under the terms of the Apache License 2.0 - see the LICENSE file for details