Architecture
This is an overview of the architecture of Eidolon, illustrated by the diagram below. Most of the code is implemented in Python with some data structures and binding code implemented in C++ and Cython. The overall objective of the architecture is to separate the compiled layers from the Python, implement a GUI-based system which maintains the Model-View-Controller (MVC) separation between the components, and provide a clean plugin interface permitting extension code to access and augment components of the system.
Eidolon relies on only a few external compiled dependencies, namely Qt and Ogre3D. The first of these two is accessed through PyQt, which is found in the binding layer, and is not therefore represented by code in this layer.
Ogre3D (Open-source Graphics Rendering Engine) is a cross-platform rendering library using OpenGL or DirectX. It provides facilities for accessing these APIs, managing resources, and implementing a fast flexible rendering pipeline. Eidolon integrates with Ogre3D by subtyping Ogre classes in OgreRenderTypes to present simplified wrappers for renderer objects, as well as provide a simple scene manager interface. This module is the only component of Eidolon where third-party compiled code is directly interfaced at the C++ level.
The RenderTypes module implements the C++ types and data structures used throughout the C++ and Python system.
These include the math types vec3, rotator, transform, and the {Vec3|Real|Color|Index}Matrix data structures collectively implemented by the template type Matrix<T>.
It also provides Figure and its subtypes, Material, Camera, RenderScene, and others which are abstract interface types implementing no behaviour themselves.
These collectively define the abstract interface between the binding layer and the underlying renderer with the expectation that they will be implemented by another module to actually provide the concrete types.
This is done by OgreRenderTypes whose members inherit these interface types to wrap Ogre concepts and objects behind the abstraction layer. How this relates to the binding layer is explained below. Although only the Ogre bindings are present, if another rendering engine is to be used instead then types can be similarly defined to implement the interface types for it without having to change any other part of the code
OgreRenderTypes also provides the implementation for getRenderAdapter() which in its declaration in RenderTypes.h returns an instance of RenderAdapter, but in the definition returns an instance of OgreRenderAdapter.
This function is the entry point to the rendering module since RenderAdapter is used to create a RenderScene, which in turn is the factory for all other renderer-related objects, eg. figures.
If another renderer binding is present, changing the object this function returns will change which binding is used.
RenderAdapter and the other renderer components explained more thoroughly in Renderer.
C++ objects are accessed in Python through binding code compiled against the CPython API. PyQt, Numpy, Scipy, and other third-party libraries do this in their own ways which Eidolon only imports as Python packages.
To interface with the renderer and associated C++ modules defined in the C++ layer, Cython is used to create wrapper types accessible in Python. The wrapper classes provide the same interface as the C++ types they represent wherever possible given the limitations of the language, so the Doxygen documentation for these is almost always correct in Python.
Renderer.pyx defines these wrappers as Cython classes.
Internally each class holds a pointer to the wrapped object, except for vec3, color, rotator, transform, Ray, and Config which have local instances instead for efficiency.
Methods are defined in the classes which wrap the equivalent method of the wrapped object, doing the argument unwrapping and return value wrapping needed to interface with Python.
The lifetime of these objects is entirely maintained by Python, the wrapped C++ types must be defined in a way which permits this and doesn't require ownership by non-wrapped C++ objects.
This isn't a problem for the local instance types like vec3 since they maintain no internal pointers to other objects nor are they pointed to by any objects.
Cython requires an interface file be defined for native code before it can be accessed, this is provided by RenderTypes.pxd.
An important property of this file is that the interface definitions are for the types in RenderTypes specifically and do not mention Ogre types.
This implies that the binding classes in Render.pyx do not reference Ogre in any way, but still store references to Ogre objects acquired through calling getRenderAdapter() which returns an Ogre-specific object.
For example, the Cython type RenderAdapter will wrap the OgreRenderAdapter instance getRenderAdapter() returns, although its interface expects an instance of RenderAdapter as defined in RenderTypes.h.
The file RenderTypes.pxd is thus part of the abstraction layer hiding Ogre-specific types behind an interface of C++ types and their Cython equivalents.
This relationship between types, abstraction layer, and Cython bindings is illustrated here for a few of the principle types:
The Cython bindings are thus programmed to the interface provided by RenderTypes but use instances of types in OgreRenderTypes. With these types in place, the Cython module is compiled into a shared object loaded at runtime as a Python module. Ogre objects implemented in C++ are thus accessed in Python through the abstract layer of RenderTypes which filter out any complex programming details specific to that renderer.
Most of the logic for Eidolon is implemented in the Python layer, with non-Python components accessed through interfaces are previously described. The architectural focus of this layer is to maintain MVC separation between concerns while providing a plugin interface permitting extension and augmentation of core system.
The MVC separation between model, view, and controller is defined in terms of module relationship and contents:
-
Model -- The model is defined by the base datatypes in the Renderer module imported through Cython from C++, and the SceneObject class and its subtypes. All loaded data is defined exclusively by SceneObject instances in Eidolon, which use the *Matrix types in Renderer to store mesh or image data. SceneObject is responsible only for storage, using only the simple algorithms needed for data representation. SceneObjectRepr instances represent the visualization of the data in their SceneObject parent objects. These also do not perform any calculation but store the data needed by the renderer.
-
View -- The view is represented by the GUI which is primarily defined in VisualizerUI.py plus a few secondary source files. There is UI code in a few other places outside these files for defining a few minor UI elements or for responding to events. The intent is to concentrate the core of the UI in the source files for that purpose but because the view is a cross-cutting concern these few places access PyQt for the sake of coherency.
-
Controller -- The controller is represented by algorithms for processing data, UI driver code, and components implementing the core system. The SceneManager object is the root of control for the system, it provides the interface to add scene objects, execute tasks asynchronously, manipulate the UI and rendering objects, interface with plugins, and other roles. To avoid being a god object it's functionality is primarily concerned with calling into plugins to perform operations on scene objects, and controller objects which manage cameras and other assets. Plugins provide the interface between the manager and scene objects, all of which must always be associated with a plugin. The algorithms for manipulating and generating data are mostly in the ImageAlgorithms.py and MeshAlgorithms.pyx source files, these are accessed by plugins to generate representations and calculate fields.
SceneManager and its components is the top of the module hierarchy in Eidolon, they collectively control the UI, plugins, and scene objects. Plugins are a group of different objects bring various functionalities into the system, these control the algorithms and scene objects involved in creating and manipulating data, as well as providing UI panels and minimal UI control code when necessary. Next in the hierarchy is the UI module and SceneObject module which are always controlled by those above and do not themselves implement important behaviour. The Utils.py and SceneUtils.py files contain a variety of functions and types used throughout the system, they represent a catch-all basket of things at the bottom of the system.
Plugins are an essential component to Eidolon. They implement the code necessary for importing and exporting data, generating representations from scene objects, introducing special-purpose algorithms and datatypes, defining custom GUI components, and driving the UI for scene objects and representations. A plugin is represented at runtime by instances of ScenePlugin subtypes which override inherited methods to interface their introduced code with the system, the protocol for which is defined by these methods whose documentation describes their use. Plugins contain many cross-cutting concerns so necessarily end up containing UI and control code, as well as mixing other concepts kept more separate in the main system.
A plugin is typically loaded at initialization time at which point they can access and augment the UI, modify enum data structures containing bindings between concepts, or otherwise modify the system however they wish. Python is very permissive in how an object can be modified while live, so this in conjunction with dynamic typing permits plugins to change just about any aspect of the system.
A SceneObject or SceneObjectRepr instance is entirely controlled by its associated plugin. The code for generating the visualization data is either in the plugin or accessed through it, as well as the code for loading and saving the data. ScenePlugin's two main subtypes, MeshScenePlugin and ImageScenePlugin, implement the code for their respective SceneObject types.