Architecture

At a high level, Anklang consists of an audio synthesis engine written in C++ with the ability to load synthesis plugins and a user interface (UI) implemented with HTML/CSS/JavaScript (web technologies).

Synthesis Core

The synthesis engine source code can be found in ase/. It is written in C++20 and implements the audio drivers, a CLAP plugin interface, synthesis threads, a thread-safe non-blocking memory allocator, serialization interfaces, etc. The engine executable can run in headless mode, supports auto-play and OPUS/FLAC/WAV export, so it can run without UI for audio rendering or unit tests.

User Interface

The UI source code resides in ui/ with web components living under ui/b/. The Anklang synthesis API (ase/api.hh) is exported via a JSON-IPC-Mechanism through a WebSocket interface. The HTML/CSS/JS files that make up the UI are served through the WebSocket. This allows operation of the UI in a local browser, any modern WebKit, Firefox or Chrome should work fine for the rendering needs. The UI currently consists of a number of files the browser loads at startup, and after that all communication with the engine goes through JSON RPC calls that the JavaScript code initiates. In order for Anklang to "feel" like a normal DAW for its users, we install the "anklang" executable, that is actually an Electronjs binary that starts the synthesis engine and runs the UI. That is just one way to run the UI however, you can also just start lib/AnklangSynthEngine directly and browse the UI at http://127.0.0.1:1777/.

Considerations

While the implementation of a DAW synthesis engine is certainly very involved, implementing user interfaces is nevertheless a major part of most applications, if not the most time involved. Other (and previous) projects of the two main authors involved synthesis UIs implemented in LessTif, XCB, Gtk+ or Qt. For the most part, implementing UIs with those toolkits is more laborious and fragile than using WWW technologies which have become ubiquitous, immensely powerful and have ever improving tooling. A very good overview of the motivations to use WWW technologies for desktop applications is given in the talk Embrace Modern Technology: Using HTML 5 for GUI in C++ by Borislav Stanimirov at CppCon 2019.

For now, Anklang facilitates WWW technologies to render its UI locally, basically using it instead of "native" toolkits like Gtk+ or Qt. However, due to outside inquiry, here are our considerations regarding remote use of the user interface.

Remote UI:
For now, the WebSocket server only listens on localhost, but the UI could be used remotely via tunneling or adjusting the code to listen on all network interfaces. But that would still render and play the audio on the host that runs AnklangSynthEngine. What we might implement in the future is to use an Opus WebRTC stream as output instead of ALSA. That'd allow running the UI remotely and listening to the output on the machine that runs the web browser. That would probably be nice for demoing Anklang (but not really work well for interactive low latency audio rendering needs). This also needs serious security considerations, since the UI can tell the engine to do file IO, the engine probably needs sandboxing (docker) and resource limits if the port was to be opened up to the internet.

Remote Plugin UIs
The above scenario would only work with headless plugins. Plugins with their own UI can only really work well when Anklang is used in connection with a local browser. One avenue that could be explored to facilitate remote pluigin UI use would be a setup that runs the plugin UIs in a VNC server and provides remote access to those vie a web browser based vnc-client. That'd probably provide a mixed user experience though, since plugin UIs are usually not optimized for remote rendering.

Remote API:
Another thing that would be possible is to use the API through just nodejs (remote or not) without the UI, that means all the interfaces and methods the UI normally uses are readily available for arbitrary script execution or REPLs. To an extend this already works via the JavaScript console implemented in modern browsers. When the browser console is open, it is possible to just type in Ase.server to get a handle at the global singleton through which all of ase/api.hh is accessible (but you need to await the result of all method calls, since all calls are implemented via async RPC calls). Also, api.hh is far from stable, if serious external uses for this interface would arise, we would effectively need to follow proper library-like semantic versioning for our API.