REP: 100 Title: ROS Stack Separation Author: Tully Foote <[email protected]>, Ken Conley <[email protected]>, Brian Gerkey <[email protected]> Status: Final Type: Standards Track Content-Type: text/x-rst Created: 18-Sep-2010 Post-History: 18-Sep-2010, 20-Oct-2010, 20-Nov-2010
While most of this REP describes information in terms of the legacy build system rosbuild, several aspect like the separation of the monolithic ROS stack are still valid with the catkin build system.
The monolithic ROS stack, as released in ROS 1.0 and ROS 1.2, is being separated into several, smaller stacks: ros, ros_comm, rx, and documentation. These smaller stacks will enable installation on more platforms, better decoupling of design, and easier release. It is also hoped that the base ros stack, which will contain the packaging and build system, will be useful in other software projects that don't make use of the ROS communication system.
New packages are marked with an asterisk *.
- mk
- rosbash
- rosbuild
- rosboost_cfg
- rosclean
- roscreate
- rosdep
- rosemacs
- roslib
- rospack
- rosmake
- rosunit*
ROS client libraries
- roslang
- rospy
- roslisp
- roscpp
- roscpp_serialization*
- roscpp_traits*
- rostime*
ROS graph tools
- rosbag
- rosbagmigration
- rosconsole
- rosgraph
- roslaunch
- rosmsg
- rosmaster
- rosnode
- rosout
- rosparam
- rosrecord
- rosservice
- rostest
- rostopic
- topic_tools
ROS core ontology
- roscore_migration_rules
- rosgraph_msgs*
- std_msgs
- std_srvs
Libraries and Utilities
- message_filters
- roswtf
- xmlrpcpp
ros_release is an existing ROS stack used for libraries and scripts related to releasing and installing ROS stacks. Two Python modules from roslib will be moved into it as new packages.
- rosdistro*
- vcstools*
The rx stack contains GUI-related tools for ROS, which are generally wxWindows-based. The 'rx' prefix is commonly used to denote ROS tools that provide a graphical user interface.
- rxbag
- rxdeps
- rxgraph
- rxtools
- wxPython_swig_interface
- wxswig
- xdot
- rosdoc
The documentation stack will only contain the rosdoc codebase in the initial release. Work is currently being done on rosdoc to separate it into a plugin-based model so that specific documentation capabilities can be added or removed as needed. For example, documentation needs for ROS.org are being moved into the rosdoc_rosorg package and will migrate to another stack. This work is independent of this REP.
The following packages will be removed as part of this separation. The rosrecord package was superceded by rosbag in ROS 1.2. The genmsg_cpp stack is being removed in favor of Python-based message generators. This is discussed further in the backward compatibility section.
- genmsg_cpp
- gtest
Header will be moved from roslib into the std_msgs package.
Clock, and Log will be moved from the roslib package to the rosgraph_msgs package.
For backwards compatibility, the generated source code files for these messages will be released with the roslib package in ROS 1.4 but marked as deprecated.
roscpp makes use of several header and cpp files in roslib. These will be moved into new packages within the ros_comm stack.
- roscpp_traits
- roscpp_serialization
- rostime
These packages use the generic 'ros' namespace, so the expected impact on existing packages is expected to be minor.
Several roslib python modules will be moved to rosgraph and elsewhere.
The following modules will be moved to rosgraph:
- roslib.genpy
- roslib.gentools
- roslib.message
- roslib.msgs
- roslib.network
- roslib.rostime
- roslib.srvs
- roslib.xmlrpc
The following modules will be moved to rosdistro (new package):
- roslib.distro
The following modules will be moved to vcstools (new package):
- roslib.vcs
The following modules will be heavily refactored:
- roslib.scriptutil
The following modules will be removed (already deprecated):
- roslib.masterapi
The main role of rosunit is to replace the bare test functionality of rostest. The bare test function wraps unit tests when they are executed and adds a configurable timeout. It also produces an Ant JUnit XML file from the results. This functionality is generally useful in the ROS build system and is also necessary to run unit tests on the ros stack.
The rosunit package will get the following resources from rostest:
rostest/bin/cleanunitrostest/bin/test-results-dirsrostest/bin/coverage-htmlrostest/bin/rostest-check-results
These scripts are used internally by build tools and do not have high visibility. They will be given newer, more descriptive names after the move.
The following script will be refactored, with some of its required routines moved to rosunit:
rostest/src/rostest/rostestutil.py
rosbuild will be converted to use scripts moved to the rosunit package.
For scripts that currently cannot be moved to the rosunit package (e.g. rostest itself), the behaviors are defined below.
rosbuild_add_roslaunch_check(): this macro requires use of the
roslaunch-check.py. This script will be moved to the roslaunch
package, which remains with the ros_comm stack. To minimize impact on
existing packages, rosbuild will continue to provide the
rosbuild_add_roslaunch_check() macro, but it will throw an error if it
cannot find roslaunch.
There is more discussion of these issues, see rosbuild testing features.
All executables, unless otherwise marked for removal, currently in $ROS_ROOT/bin will remain there. Binary executables will be converted to bash scripts using rosrun.
Several main motivations have been identified for this separation
- Making the "core" ROS lighter weight, which enables - Easier porting to other platforms - Smaller footprint
- Enabling projects to use the ROS packaging and build system separate from the communication infrastructure
- Easier maintainability
- Better decoupling of design interests between the packaging and communication architectures
rosbuild explicitly uses the following packages:
- rosbuild
- rospack
- gtest
- roslib
- rostest
The use of rostest is an issue, because rostest will be in the ros_comm stack, while rosbuild will be in the ros stack. rosbuild uses the following tools from rostest:
- (*)
bin/rostest-check-results - (*)
bin/test-results-dir - (*)
bin/rostest-results - (*)
bin/coverage-html - (+)
bin/roslaunch-check.py - (X)
bin/rostest
The tools marked (*) will be moved to the new rosunit package, but the rest will remain in ros_comm packages.
Options for handling this separation include:
- Follow the
rosbuild_add_roslaunch_check()model: leave everything where it is, and ifbin/rostestisn't available, throw an error if the user calls any ofrosbuild_add_gtest*()orrosbuild_add_pyunit*().- Separate out the
bin/rostest --barebehavior and put it somewhere lower, such as rosunit.
We have chosen to implement Option 2, as it is the better long term solution. Conceptually, rosunit implements unit testing functionality, and rostest implements integration testing functionality. rosunit has no graph dependencies, though it does make use of the roslaunch process manager. This will be resolved in the short term by copying the process manager out of roslaunch. In the future, we can envision an implementation of roslaunch that does not depend on the graph and could be used as a library for rosunit.
Three msg IDL files are currently stored in the roslib package as they are common to all ROS client libraries. The roslib package must go into the base ros stack as it contains many of the libraries used by the package and build tools, but we wish to keep the ros stack decoupled from client libraries issues.
The Clock and Log msg files are not considered to be user facing. They are also specific to the ROS middleware implementation and would pollute a more generic ontology like std_msgs. The designation of a new rosgraph_msgs package clearly associates these msg files as being specific to representing state of the ROS graph. We originally intended to create a roslang_msgs package, but decided that a rosgraph_msgs was a better categorization. In the future, we might add messages for the ROS Master state and other graph data.
The Header msg file is user facing and will go into std_msgs. It would be preferable to be able to move the Header msg out of the ROS stacks entirely, but that is out of scope for this proposal.
The impact of changing the location of these messages requires further study. The Header message is generally not referred to as "roslib/Header", which will hopefully minimize the impact in the ROS code base, despite it being commonly used. The Clock and Log messages generally have limited use outside of ROS client libraries (e.g. simulators, GUI tools), and will hopefully be easy to migrate.
The roslib libraries selected for moving are generally to support ROS communication. Leaving them in the ros stack would create a coupling between ros_comm and the ros stack that would make future development difficult. Moving of roslib python modules is expected to have low external impact as these libraries are generally for internal use. It is expected to have high impact internally as many tool make use of these representations.
The genmsg_cpp package currently lives at the root of the ROS dependency tree so that it can be compiled before any messages are generated. This design causes many problems, though the main one of concern is that it introduces client-library-specific information in the base ROS stack. In order to meet our goals of decoupling the ROS packaging system from the ROS communication system, this implementation artifact needs to be removed. Despite its name, genmsg_cpp is no longer used for roscpp and only affects roslisp, rosoct, and rosjava. The effect on these packages is described in the Backwards Compatibility section.
Client libraries are instead expected to develop Python-based message generators (e.g. rospy and roscpp) or dynamically convert the msg IDL to code (e.g. roslua).
The placement of the rosdoc package was considered for both the ros and documentation stacks. The rosdoc tool is a generally useful tool as it provides a mechanism for auto-generating documentation across a ROS-package-based code base. Comparison to similar tools, like javadoc, shows that these documentation tools are generally distributed with the main system.
Thus, moving the rosdoc package to its own, separate "documentation" stack does generate problems from a user perspective. In particular, they may be confused that it is missing from the ros stack. This concern is balanced by multiple motivations:
- rosdoc has heavy-weight dependencies (rxdeps, graphviz, doxygen, sphinx, latex).
- Future implementations of rosdoc will likely break it into multiple packages in a plugin model (e.g. rosdoc_sphinx, rosdoc_rosorg, etc...).
- The ROS Distribution system and packaging system can provide the documentation stack in common configurations.
The inclusion of ros_comm and rx executables with the ros stack is unfortunate. It does not significantly increase the size of the ros stack as binary executables will be replaced with bash scripts, but it does add non-functioning executables to a bare ros installation.
Our debian package system does not allow another ROS stack to build into $ROS_ROOT/bin. Work on a ROS install target (REP 102 [1]) will hopefully provide a clean solution in the future.
This separation is expected to break the existing, experimental rosoct and rosjava libraries due to the removal of the genmsg_cpp removal.
The roslisp message generator is currently implemented in genmsg_cpp, but is in the process of being converted to the Python-based approach.
For additional backwards compatibility, libraries for msg handling as well as core msg types (Header, Log, and Clock) will be left in the roslib package for the ROS 1.4 release. These will be marked as deprecated and removed in the ROS 1.5 release.
Early planning and cleanup for this separation went into the ROS 1.2 release, including rewriting the roscpp message generator in Python. For the ROS 1.4 release, the rx and documentation stacks are expected to be cleanly separated. For reasons discussed above, clean separation of the ros and ros_comm stacks are not expected for the ROS 1.4 release. In particular, the changes related to 'moving of roslib Python modules' is not expected to be fully completed by the ROS 1.4 release due to the difficulty in moving and renaming Python modules.
All preference will be given to maintaining a stable system over the separation. Although it is desirable to have a perfectly clean separation, the majority of our design goals even if this is not achieved in the ROS 1.4 release.
| [1] | REP 102 (https://ros.org/reps/rep-0102.html) |
This document has been placed in the public domain.