Merge branch 'master' into lull-bigio

# By Balazs Racz (267) and others
# Via Balazs Racz (18) and GitHub (3)
* master: (305 commits)
  Fixes clang-format style file to create the desired brace linebreak for switch/case statements.
  Adds callbacks to javascript connection monitors (#634)
  TCAN4550 Instrumentation Addition (#629)
  Adds feature to EntryModel to set the leading zeros. (#632)
  Minor fixes on the train node application: (#633)
  Adds print features for service mode packets to DccDebug. (#631)
  Fixes nullptr bug in CallableFlow. (#630)
  TCAN4550 Bus Off Recovery (#628)
  Adds support for partial read command in memconfig_utils. (#627)
  Allows calling invoke_subflow_and_ignore_result outside of stateflows. (#626)
  Specifies the XML encoding to be utf-8. (#625)
  Adds implementation of factory_reset command.  (#624)
  Fixes syntax error in clang-format file.
  Adds implementation of byte buffer. (#622)
  Adds byte_stream.md, a design documentation for a proposed feature. (#621)
  Adds missing makefiles in build directory.
  Inverts the run direction of the capture counter in the STM32 Dcc Decoder. (#620)
  Adds debug log to the common DccDecoder class. (#619)
  Makes microdelay use C linkage. (#618)
  Adds shared code for STM32 implementation of the crash blinker. (#617)
  ...

# Conflicts:
#	applications/nucleo_io/targets/freertos.armv6m.st-stm32f091rc-nucleo-dev-board/config.hxx
#	applications/nucleo_io/targets/freertos.armv7m.st-stm32f303re-nucleo-dev-board/config.hxx

.clang-format

@@ -43,5 +43,26 @@ SpacesInCStyleCastParentheses: false
 SpaceAfterControlStatementKeyword: true
 SpaceBeforeAssignmentOperators: true
 ContinuationIndentWidth: 4
+
+# This big block of settings works around indentation after an extern "C"
+BraceWrapping: {
+  AfterCaseLabel:  true,
+  AfterClass:      true,
+  AfterControlStatement: true,
+  AfterEnum:       true,
+  AfterFunction:   true,
+  AfterNamespace:  true,
+  AfterObjCDeclaration: true,
+  AfterStruct:     true,
+  AfterUnion:      true,
+  AfterExternBlock: false,
+  BeforeCatch:     true,
+  BeforeElse:      true,
+  IndentBraces:    false,
+  SplitEmptyFunction: true,
+  SplitEmptyRecord: false,
+  SplitEmptyNamespace: false }
+BreakBeforeBraces: Custom
+IndentExternBlock: false
 ...
 

.github/workflows/ArduinoBuild.yml

@@ -0,0 +1,49 @@
+# ArduinoBuild.yml
+#
+# Github workflow script that compiles all examples from the OpenMRNLite
+# Arduino library.
+#
+# Copyright (C) 2021  Balazs Racz
+#
+
+# Name of workflow
+name: ArduinoBuild
+on:
+  push:
+    paths:
+      - '**.ino'
+      - '**rduino**'
+      - '**ArduinoBuild.yml'
+  pull_request:
+jobs:
+  build:
+    name: Build ESP32 examples
+    runs-on: ubuntu-latest
+    steps:
+    - name: Checkout
+      uses: actions/checkout@master
+
+    - name: Generate OpenMRNLite library
+      run: |
+        mkdir --parents $HOME/Arduino/libraries/OpenMRNLite
+        $GITHUB_WORKSPACE/arduino/libify.sh $HOME/Arduino/libraries/OpenMRNLite $GITHUB_WORKSPACE -f -l
+        rm -f $GITHUB_WORKSPACE/arduino/examples/Stm*/build_opt.h
+
+    - name: Compile all STM32 examples
+      uses: ArminJo/[email protected]
+      with:
+        platform-url: https://raw.githubusercontent.com/stm32duino/BoardManagerFiles/master/STM32/package_stm_index.json
+        arduino-board-fqbn: STM32:stm32:Nucleo_144:pnum=NUCLEO_F767ZI,upload_method=MassStorage,xserial=generic,usb=CDCgen,xusb=FS,opt=osstd,rtlib=nano
+        sketch-names: Stm32*.ino
+        build-properties: '{ "All": "-DHAL_CAN_MODULE_ENABLED" }'
+        debug-compile: true
+
+    - name: Compile all ESP32 examples
+      uses: ArminJo/[email protected]
+      with:
+        platform-url: https://raw.githubusercontent.com/espressif/arduino-esp32/gh-pages/package_esp32_index.json
+        arduino-board-fqbn: esp32:esp32:node32s
+        sketch-names: ESP*.ino
+        debug-compile: true
+
+

DISTCC.md

@@ -0,0 +1,180 @@
+# Using distcc with OpenMRN compilation framework
+
+## Overview
+
+### What is this?
+
+This feature allows using a powerful remote computer to perform most of the
+compilation steps when compiling OpenMRN-based projects.
+
+### Why?
+
+It is much faster.
+
+OpenMRN compiles a large number of c++ files that can take a lot of CPU to
+compile. If your workstation has constrained CPU (e.g. a laptop or a virtual
+machine with limited number of CPUs), you can use a powerful remote machine to
+offload most of the compilation steps.
+
+### How does it work?
+
+We use the opensource distcc package (`[email protected]:distcc/distcc.git`). We
+create an SSH link to the remote machine, and start the distcc daemon. We wrap
+the local compilation command in distcc. Distcc will preprocess the source file
+locally, then transmit the preprocessed source to the remote machine over the
+SSH link. The daemon running on the powerful machine will then invoke the
+compiler, create the .o file, and transmit the .o file back over the SSH
+link. The linking steps then happen locally.
+
+## Prerequisites
+
+- You have to have the same compiler that OpenMRN uses available on the remote
+  machine. We have a [specific step](#configuring-the-compilers) for telling
+  distcc and OpenMRN what the matching compilers are.
+- You need to start the SSH link before trying to compile things. If you forget
+  this, local compilation will be performed instead.
+- A small change needs to be made to the distcc sources, because it does not
+  support a compiler flag that we routinely pass to armgcc. For this reason you
+  need to compile it from source.
+
+### Installing
+
+#### Installing distcc
+
+1. `sudo apt-get install distcc` on both the local and remote machine.
+2. Clone the distcc sources from `[email protected]:distcc/distcc.git`. Check the
+   INSTALL file for an apt-get commandline near the very top for compile-time
+   dependencies to be installed.
+3. edit `src/arg.c`. Find the place where it checks for `"-specs="`
+   argument. Comment out the early return. (Not doing anything in that if is
+   what you want.)
+4. edit `src/serve.c`. Find where it checks for `"-specs="`, and disable that
+   check.
+4. edit `src/filename.c`. Find where it checks for `"cxx"`, and add another
+   line so that it supports `"cxxtest"` as well.
+5. run through the compile steps of distcc, see the INSTALL file. These are
+   normally:
+   ```
+   ./autogen.sh
+   ./configure
+   make
+   ```
+6. Copy `distcc` to `~/bin`, and copy `distccd` to the remote machine at `~/bin`
+
+#### Configuring the compilers
+
+When setting up compilers, we need to ensure that we can determine what the
+remote compiler will be called for any given local compiler that we have. In
+addition to this, we need to be able to call the same compiler underthe same
+command on both machines, because the local machine will be calling the
+compiler for preprocessing and the remote machine will be calling it for
+compilation.
+
+For any given compiler, you need to make a symlink on both the local machine
+and the remote machine:
+
+```bash
+cd ~/bin
+ln -sf $(realpath /opt/armgcc/gcc-arm-none-eabi-8-2018-q4-major/bin/arm-none-eabi-gcc) armgcc-2018-q4-gcc
+ln -sf $(realpath /opt/armgcc/gcc-arm-none-eabi-8-2018-q4-major/bin/arm-none-eabi-g++) armgcc-2018-q4-g++
+```
+
+Do this on BOTH the local and remote machine. The real path may differ but the
+name must be exactly the same.
+
+Only on the remote machine, the compiler also needs to be added to the
+masquerade directory, otherwise distccd server will refuse to execute it.
+
+```bash
+cd /usr/lib/distccd
+sudo ln -sf ../../bin/distcc armgcc-2018-q4-g++
+sudo ln -sf ../../bin/distcc armgcc-2018-q4-gcc
+```
+
+#### Setting parameters
+
+For the OpenMRN compilation scripts to find distcc and the remote machine, we
+store a few parameters in files under `~/.distcc`.
+
+```bash
+mkdir -p ~/.distcc
+echo 3434 > ~/.distcc/port
+echo my-fast-remote-machine.myplace.com > ~/.distcc/ssh_host
+echo '127.0.0.1:3434/20,lzo' > ~/.distcc/hosts
+```
+
+The port number will be used with SSH port forwarding. The remote machine is
+used by the ssh tunnel setup script.
+
+The "/20" part of the hosts line is the capacity of the remote machine. This
+one says it's good for 20 parallel jobs, which I picked for a 6-core (12
+hyperthread) machine with lots of RAM.
+
+You can configure a lot of other things in the hosts file. `man distcc` for the
+full documentation. It is also possible to configure the local CPU to run some
+number of jobs while the remote CPU runs other builds. It is possible to use
+multiple remote machines as well.
+
+## Using
+
+After every restart of your machine you need to start the ssh tunnel:
+
+```bash
+~/openmrn/bin/start-distcc.sh
+```
+
+This will set up the SSH tunnel to the remote host and start the distccd server
+on the far end of the tunnel.
+
+Then compile OpenMRN stuff with very high parallelism:
+
+```bash
+~/openmrn/applications/io_board/target/freertos.armv7m.ek-tm4c123gxl$ make -j21
+```
+
+### How do I know if it worked?
+
+OpenMRN build system will automatically detect that you have the SSH tunnel
+running to distcc and use distcc.
+
+If it uses distcc, then you will see the compilation done remotely. Check the
+beginning of most lines:
+
+```
+distcc armgcc-2018-q4-gcc -c -Os -fno-strict-aliasing [...] STM32L4xx_HAL_Driver/Src/stm32l4xx_hal_usart.c -o stm32l4xx_hal_usart.o
+```
+
+If it is done locally, then you don't see distcc but see the execution of the
+compiler directly:
+
+```
+/opt/armgcc/gcc-arm-none-eabi-8-2018-q4-major/bin/arm-none-eabi-gcc -c -Os -fno-strict-aliasing [...] STM32L4xx_HAL_Driver/Src/stm32l4xx_hal_usart.c -o stm32l4xx_hal_usart.o
+```
+
+### How much does it help?
+
+With distcc (parallelism of 21, all remotely; this is on a W-2135 Xeon CPU @
+3.70GHz):
+
+```bash
+   text	   data	    bss	    dec	    hex	filename
+ 113808	    312	   3768	 117888	  1cc80	io_board.elf
+/opt/armgcc/default/bin/arm-none-eabi-objdump -C -d -h io_board.elf > io_board.lst
+
+real    0m44.704s
+user    2m41.551s
+sys     0m29.235s
+```
+
+Without distcc (parallelism of 9, all locally; this is on an intel i7-8665U CPU
+@ 1.90GHz -- this is a 15W TDP mobile CPU with 4 cores, 8 hyperthreads):
+
+```bash
+   text	   data	    bss	    dec	    hex	filename
+ 113808	    312	   3768	 117888	  1cc80	io_board.elf
+/opt/armgcc/default/bin/arm-none-eabi-objdump -C -d -h io_board.elf > io_board.lst
+
+real    2m8.602s
+user    14m21.471s
+sys     0m42.488s
+```

Makefile

@@ -38,3 +38,37 @@ js-tests:
 	$(MAKE) -C targets/js.emscripten run-tests
 
 alltests: tests llvm-tests
+
+release-clean:
+	$(MAKE) -C targets/linux.x86 clean
+
+RELNAME=$(shell uname -sm | tr ' A-Z' '.a-z')
+RELDIR=$(OPENMRNPATH)/bin/release/staging-$(RELNAME)
+JSRELDIR=$(OPENMRNPATH)/bin/release/staging-js
+
+include $(OPENMRNPATH)/etc/release.mk
+
+# These are the applications that are packaged into the binary release.
+$(call RELEASE_BIN_template,hub,applications/hub/targets/linux.x86)
+$(call RELEASE_BIN_template,memconfig_utils,applications/memconfig_utils/targets/linux.x86)
+$(call RELEASE_BIN_template,bootloader_client,applications/bootloader_client/targets/linux.x86)
+$(call RELEASE_BIN_template,send_datagram,applications/send_datagram/targets/linux.x86)
+
+$(call RELEASE_JS_template,openmrn-bootloader-client,applications/bootloader_client/targets/js.emscripten)
+$(call RELEASE_JS_template,openmrn-hub,applications/js_hub/targets/js.emscripten)
+
+release-bin:
+	rm -rf $(RELDIR)/*
+	mkdir -p $(RELDIR)
+	+$(MAKE) -C . release-bin-all
+	cp LICENSE.md $(RELDIR)/LICENSE.txt
+	cd $(RELDIR); zip -9r ../applications.$(RELNAME).zip . 
+
+release-js:
+	rm -rf $(JSRELDIR)/*
+	mkdir -p $(JSRELDIR)
+	+$(MAKE) -C . release-js-all
+	cp LICENSE.md $(JSRELDIR)/win/LICENSE.txt
+	cp LICENSE.md $(JSRELDIR)/macos/LICENSE.txt
+	cd $(JSRELDIR)/win; zip -9r ../../applications.win.zip .
+	cd $(JSRELDIR)/macos; zip -9r ../../applications.macos.zip .

RELEASE.md

@@ -0,0 +1,85 @@
+# Release process
+
+This document details how releases need to be built and published for OpenMRN.
+
+## Purpose and content of releases
+
+Releases fulfill two purposes:
+
+- Precompiled binaries are made available for download. This is important for
+  those that do not have facilities or the experience to compile OpenMRN
+  applications from source.
+
+- (At a later point) a packaged library is made available with include headers
+  and `*.a` files that enable importing OpenMRN into a different compilation
+  environment, such as various IDEs for embedded development.
+
+The following is out of scope of this documentation:
+
+- Releases for OpenMRNLite (Arduino compatible library) are documented in the
+  [arduino/RELEASE.md](arduino/RELEASE.md) file.
+
+## Requirements for building releases
+
+- You need to be able to compile OpenMRN binaries. You need a linux host (or
+  VM). The necessary packages have to be installed:
+
+  - beyond standard `g++` you will need `sudo apt-get install
+    libavahi-client-dev`
+
+- You need both a linux.x86_64 and a raspberry pi host.
+
+- On the linux host you should install node.js, npm, emscripten, and the
+  packager:
+
+  `sudo npm install -g pkg`
+
+## Release numbering
+
+Release numbers are marked as major.minor.patch, such as `v2.10.1`.
+
+- Major release numbers are incremented once per year (2020 is 0, 2021 is 1,
+  2022 is 2, ...).
+
+- Minor release numbers are 10, 20, 30, 40 for given quarters, then incremented
+  by one if there are multiple releases built within a quarter.
+
+- Patch release numbers start at 1, and are only incremented if the same
+  release needs to be re-built with a patch.
+
+
+## How to build
+
+All of the make commands here need to be run in the openmrn directory
+(toplevel).
+
+1. Start with a clean checkout. Run `make release-clean`.
+
+2. On the linux.x86_64 host, run
+
+   `make -j5 release-bin`
+
+3. Copy `openmrn/bin/release/applications.linux.x86_64.zip` as one of the
+   artifacts.
+
+3. On the raspberry pi host, do the same:
+
+   `make release-clean`
+
+   `make -j4 release-bin`
+
+4. Copy `openmrn/bin/release/applications.linux.armv7l.zip` as one of the
+   artifacts. Rename it to `applications.linux.armv7l-raspberry-pi.zip`
+
+5. On the linux.x86_64 host, build the javascript binaries. Run
+
+   `make -j5 release-js`
+
+6. Copy `openmrn/bin/release/applications.win.zip` and
+   `openmrn/bin/release/applications.macos.zip`.
+
+7. On the OpenMRN GitHub project, select Releases, create a new release, select
+   create a new tag in the form of `release-v2.10.1`. Upload the release
+   artifacts that you collected.
+
+8. Publish the release.

applications/Makefile

@@ -25,6 +25,8 @@ SUBDIRS = \
 	train \
 	tcp_blink_client \
 	usb_can \
+    time_client \
+    time_server
 
 
 include $(OPENMRNPATH)/etc/recurse.mk