Restore output plugin and further code quality improvements to ruby

CHANGELOG.md

@@ -1,14 +1,23 @@
 # Change Log
 
-## 2.7.3 (Future)
+## 2.7.3
 
-Future
+26th October 2021
 
 Log Courier
 
 - Add new [`add timezone name field`](docs/log-courier/Configuration.md#add-timezone-name-field) configuration that adds the timezone name such as `UTC` or `Europe/London` for use with the Logstash Date Filter. The existing `timezone` / `event.timezone` (ECS) fields were in a format the filter could not use (#345)
 - Fix race that might cause a file to use a configuration other than the first configuration it matches (#367)
 
+Logstash Input Plugin
+
+- Removed `peer_recv_queue` configuration as it is unused. Only a single payload is received and processed at any one time by the plugin.
+
+Logstash Output Plugin
+
+- Now maintained again and updated to use latest log-courier ruby implementation which includes protocol handshake support
+- Added support for `tcp` only output
+
 ## 2.7.2
 
 21st October 2021

docs/LogstashIntegration.md

@@ -3,29 +3,30 @@
 - [Logstash Integration](#logstash-integration)
   - [Overview](#overview)
   - [Installation](#installation)
-  - [Configuration](#configuration)
+  - [Input Configuration](#input-configuration)
+  - [Output Configuration](#output-configuration)
 
 ## Overview
 
-Log Courier is built to work seamlessly with [Logstash](http://logstash.net). It
-communicates via an input plugin called "courier".
+Log Courier can be used to send events to [Logstash](http://logstash.net) by installing an input plugin.
 
-(NOTE: An output plugin exists for Logstash to Logstash transmission but is archived
-and no longer maintained and its use is not advised.)
+Additionally, Logstash can send events to Log Carver by installing an output plugin.
 
 ## Installation
 
-Simply run the following commands as the user Logstash was installed with to
-install the latest stable version of the Log Courier plugin.
+Simply run the following commands as the user Logstash was installed with to install the latest stable version of the Log Courier input plugin.
 
     cd /path/to/logstash
     ./bin/logstash-plugin install logstash-input-courier
 
-Once the installation is complete, you can start using the plugin!
+To install the output plugin, run the following
 
-## Configuration
+    cd /path/to/logstash
+    ./bin/logstash-plugin install logstash-output-courier
+
+## Input Configuration
 
-The 'courier' input plugin will now be available. An example configuration follows.
+An example configuration for the `courier` input plugin is below:
 
     input {
         courier {
@@ -37,20 +38,40 @@ The 'courier' input plugin will now be available. An example configuration follo
 
 The following options are available:
 
-- transport - "tcp", "tls", "plainzmq" or "zmq" (default: "tls")
+- transport - "tcp" or "tls" (default: "tls")
 - address - Interface address to listen on (defaults to all interfaces)
 - port - The port number to listen on
 - ssl_certificate - Path to server SSL certificate (tls)
 - ssl_key - Path to server SSL private key (tls)
 - ssl_key_passphrase - Password for ssl_key (tls, optional)
 - ssl_verify - If true, verifies client certificates (tls, default false)
-- ssl_verify_default_ca - Accept client certificates signed by systems root CAs
-(tls)
-- ssl_verify_ca - Path to an SSL CA certificate to use for client certificate
-verification (tls)
+- ssl_verify_default_ca - Accept client certificates signed by systems root CAs (tls)
+- ssl_verify_ca - Path to an SSL CA certificate to use for client certificate verification (tls)
+- min_tls_version - Sets the minimum TLS version when transport is "tls", defaults to 1.2, minimum is 1.0 and maximum 1.3
+- max_packet_size - The maximum packet size to accept (default 10485760, corresponds to Log Courier's `spool max bytes`)
+- add_peer_fields - Add "peer" field to events that identifies source host, and "peer_ssl_dn" for TLS peers with client certificates
+
+## Output Configuration
+
+An example configuration for the `courier` output plugin is below:
+
+    output {
+        courier {
+            addresses       => ['127.0.0.1']
+            port            => 12345
+            ssl_certificate => "/opt/logstash/ssl/logstash.cer"
+        }
+    }
+
+The following options are available:
+
+- transport - "tcp" or "tls" (default: "tls")
+- addresses - Address to connect to in array format (only one value is supported at the moment)
+- port - Port to connect to
+- ssl_ca - Path to SSL certificate to verify server certificate
+- ssl_certificate - Path to client SSL certificate (optional)
+- ssl_key - Path to client SSL private key (optional)
+- ssl_key_passphrase - Password for ssl_key (optional)
 - min_tls_version - Sets the minimum TLS version when transport is "tls", defaults to 1.2, minimum is 1.0 and maximum 1.3
-- max_packet_size - The maximum packet size to accept (default 10485760,
-corresponds to Log Courier's `"spool max bytes"`)
-- peer_recv_queue - The size of the internal queue for each peer
-- add_peer_fields - Add "peer" field to events that identifies source host, and
-"peer_ssl_dn" for TLS peers with client certificates
+- spool_size - Maximum number of events to spool before a flush is forced (default 1024)
+- idle_timeout - Maximum time in seconds to wait for a full spool before flushing anyway (default 5)

lc-lib/core/version.go

@@ -19,5 +19,5 @@ package core
 // LogCourierVersion is the library version number
 const LogCourierMajorVersion uint32 = 2
 const LogCourierMinorVersion uint32 = 7
-const LogCourierPatchVersion uint32 = 0
-const LogCourierVersion string = "2.7.0"
+const LogCourierPatchVersion uint32 = 3
+const LogCourierVersion string = "2.7.3"

ruby/log-courier/.gitignore

@@ -1,3 +1,2 @@
+/lib/log-courier/rspec/tmp
 /pkg
-/spec/tmp
-/vendor

ruby/log-courier/.rubocop.yml

@@ -1,5 +1,7 @@
 AllCops:
   NewCops: enable
+Layout/LineLength:
+  Max: 140
 Metrics/AbcSize:
   Enabled: false
 Metrics/BlockLength:
@@ -10,13 +12,11 @@ Metrics/ClassLength:
   Enabled: false
 Metrics/CyclomaticComplexity:
   Enabled: false
-Metrics/LineLength:
-  Max: 140
 Metrics/PerceivedComplexity:
   Enabled: false
 Metrics/MethodLength:
   Enabled: false
-Naming/Filename:
+Naming/FileName:
   Enabled: false
 Style/FrozenStringLiteralComment:
   Enabled: false

ruby/log-courier/Gemfile.lock

@@ -1,24 +1,24 @@
 PATH
   remote: .
   specs:
-    log-courier (2.7.2)
+    log-courier (2.7.3)
       cabin (~> 0.6)
       multi_json (~> 1.10)
 
 GEM
   remote: https://rubygems.org/
   specs:
-    ast (2.4.0)
+    ast (2.4.2)
     cabin (0.9.0)
     diff-lcs (1.4.4)
-    jaro_winkler (1.5.4-java)
-    jrjackson (0.4.13-java)
+    jrjackson (0.4.11-java)
     multi_json (1.15.0)
-    parallel (1.19.1)
-    parser (2.7.0.4)
-      ast (~> 2.4.0)
+    parallel (1.21.0)
+    parser (3.0.2.0)
+      ast (~> 2.4.1)
     rainbow (3.0.0)
-    rake (13.0.1)
+    rake (13.0.6)
+    regexp_parser (2.1.1)
     rexml (3.2.5)
     rspec (3.10.0)
       rspec-core (~> 3.10.0)
@@ -33,16 +33,19 @@ GEM
       diff-lcs (>= 1.2.0, < 2.0)
       rspec-support (~> 3.10.0)
     rspec-support (3.10.2)
-    rubocop (0.80.1)
-      jaro_winkler (~> 1.5.1)
+    rubocop (1.22.2)
       parallel (~> 1.10)
-      parser (>= 2.7.0.1)
+      parser (>= 3.0.0.0)
       rainbow (>= 2.2.2, < 4.0)
+      regexp_parser (>= 1.8, < 3.0)
       rexml
+      rubocop-ast (>= 1.12.0, < 2.0)
       ruby-progressbar (~> 1.7)
-      unicode-display_width (>= 1.4.0, < 1.7)
-    ruby-progressbar (1.10.1)
-    unicode-display_width (1.6.1)
+      unicode-display_width (>= 1.4.0, < 3.0)
+    rubocop-ast (1.12.0)
+      parser (>= 3.0.1.1)
+    ruby-progressbar (1.11.0)
+    unicode-display_width (2.1.0)
 
 PLATFORMS
   java