Airsim: Search and Rescue SAR at sea with UAV integrated Artificial Intelligence

[cite] Do Trong, T., Hai, Q. T., Duc, N. T., & Thanh, H. T. (2021, January). A Novelty Approach to Emulate Field Data Captured by Unmanned Aerial Vehicles for Training Deep Learning Algorithms Used for Search-and-Rescue Activities at Sea. In 2020 IEEE Eighth International Conference on Communications and Electronics (ICCE) (pp. 288-293). IEEE.

1. Introduction

The method used to build a simulated marine environment using Unreal Engine 4 and collect datasets from the environment via cameras from UAV supported by Airsim. After that, the Faster-RCNN Inception-V2 is transferred to the new dataset from the simulation environment. Finally, the victim search algorithm is proposed for the scenario of an accident and send it to the GPS location of the victims from the UAV. The Search and Rescue system diagram is illustrated below:

2. Setup

Install Airsim and Unreal Engine. Install CUDA and cuDNN Install the required libraries:

pip install airsim
pip install tensorflow-gpu==1.14.0
pip install opencv-python
pip install numpy
conda install -c anaconda cudnn
conda install -c anaconda cudatoolkit

We provide a configuration file for the UAV settings.json and Ocean Environment.

3. Data collection

Step 1: Start the Unreal Engine and open the project containing the Ocean environment.

Step 2: Go to the Map folder. Then choose the Ocean environment with the proper settings before starting the flight simulation.

Step 3: Initiate flight by dropping the UAV into the simulation environment. Click Play on Unreal Engine to do this.

Step 4: Open Visual Studio Code and open to the directory containing the Python script source code.

python Get_Dataset.py --radius 10 --altitude 8 --speed 4 --center "10,0" --iterations 1

The parameters to set are:

• radius (m): Additional radius for the UAV after each round orbit is completed.
• altitude (m): UAV flight altitude.
• speed (m/s): UAV flight speed.
• center (NED): It is the center of the search area where the UAV carries orbit around this center to search for victims.
• interations (int): The number of circular orbits that the UAV will fly.

The table below illustrates the number of images obtained by interation.

Radius	Altitude	Speed	Center	Interation	Images
5	8	4	(10,0)	1	23
5	8	4	(10,0)	2	53
5	8	4	(10,0)	3	87
5	8	4	(10,0)	4	154
5	8	4	(10,0)	5	189
5	8	4	(10,0)	6	253

Step 5: Configure flight parameters for UAVs in the command. For each environment, this command parameter will be customized to get different number of images.

Several images were obtained from the simulation environment.

Our data set collects itself as Victims on Ocean. You can use this to train a victim detection model for yourself.

4. Training Victims Detector

To train a model to detect victims in simulated ocean environment, you can follow this guide English or Vietnamese.

5. Victims Detection

To detect the victim at sea, run the following command:

python Victim_detector.py