Introduction to robotics project 2022/2023
Project for the course of Introduction to Robotics, University of Trento 2022/2023
Overall, the task of the project is to develop the code for a robotic arm capable of moving Megablocks of different classes to precise locations despite their orientation. Each of the 10 block types must be recognized and classified properly even if it’s placed upside down or lying on its side. The final task requires blocks to be moved around and piled up to build a hardcoded building such as a castle, a tower or more.
This project relies on the locosim repository that manages the simulation of the robot and the environment. It spawns the blocks and the environment and manages the gripper. The locosim repository is a submodule of this repository, so make sure you have it installed.
- Ubuntu 20.04
- ROS Noetic
- Gazebo 11
- Python 3.8
- OpenCV 4.5.2
- Numpy 1.19.5
- Scipy 1.7.1
- Scikit-learn 0.24.2
- Scikit-image 0.18.3
- Matplotlib 3.4.2
- Pandas 1.3.2
- Tensorflow 2.5.0
- Keras 2.5.0
- Pytorch 1.9.0
- Torchvision 0.10.0
- CUDA 11.2
- CUDNN 8.1.1
- Clone the repository
- Clone the submodules, make sure you have installed locosim:
git submodule update --init --recursive
- Install the requirements
- Build a ros workspace
- Source the workspace
- Open your .bashrc file
nano ~/.bashrc
- Add the following line
alias ur5='python3 -i ~/${your_catkin_ws}/src/locosim/robot_control/lab_exercises/lab_palopoli/ur5_generic.py'
- Save and exit
- Source the .bashrc file
source ~/.bashrc
- Open another terminal and launch the ur5 script from locosim, we've used an alias to launch the script The ur5 will launch roscore, if you're looking to test the single nodes separately, you must launch roscore
ur5
- Open another terminal and launch the move node
rosrun cpp_publisher move
- Open another terminal and launch the planner
rosrun cpp_publisher planner
- Open another terminal and launch the vision node
rosrun py_publisher vision
The rviz and gazebo simulation should start and the robot start moving. You can run the script run.sh instead of points from 2 to 4.
- Follow the steps in the locosim repository to install all drivers for the real robot
- Set all REAL_ROBOT flags for very node to True
- Open a terminal and launch the alias created following the steps in the locosim repository
robot_launch
- Open another terminal and launch the ur5 script from locosim, we've used an alias to launch the script. Roscore isn't necessary as ur5 will launch it
ur5
- Open another terminal and launch the move node
rosrun cpp_publisher move
- Open another terminal and launch the planner
rosrun cpp_publisher planner
- Open another terminal and launch the vision node
rosrun py_publisher vision
The move node is responsible for moving the robot in the simulation, it's written in C++ and it's based on the ur5 script from locosim. The move node is launched by rosrun cpp_publisher move. The move node publishes the current position of the robot on the topic /ur5/position and it subscribes to the topic /ur5/goal to receive the goal position of the robot.
The planner node is responsible for planning the path of the robot, it's written in C++ and it's based on the ur5 script from locosim. The planner node is launched by rosrun cpp_publisher planner. The planner node subscribes to the topic /ur5/position to receive the current position of the robot and it publishes the goal position of the robot on the topic /ur5/goal.
The vision node is responsible for detecting the blocks in the simulation, it's written in Python. The vision node is launched by rosrun py_publisher vision. The vision node subscribes to the topics:
- /ur5/zed_node/left/image_rect_color to receive the image from the camera.
- /ur5/zed_node/point_cloud/cloud_registered to recieve the point cloud from the camera and to calculate the 3D position of the block.
Than the node publishes the position of the blocks to the planner on the topic vision/vision_detection.