Added a prometheus metrics exporter to Ghost

[cmraible]

ref https://linear.app/tryghost/issue/ENG-1505/add-prometheus-metrics-server-to-allow-monitoring-ghost-metrics

Summary

This commit includes two main components: a prometheus client class to collect metrics from Ghost, and a standalone metrics server that exposes a /metrics endpoint at a separate port (9416 by default) from the main Ghost app.

The prometheus client is a very thin wrapper around prom-client. We could use prom-client directly, but this approach should make it easier to switch to a different prometheus client package (or make our own) if we ever need to down the line.

The list of default metrics this enables is specified in an e2e test here. This also gives us the ability to create and collect custom metrics, although none are included in this commit yet.

Configuration

The prometheus client and the metrics server are both disabled by default, but can be enabled by setting the metrics_server:enabled flag to true.

You can also define a custom host and port using metrics_server:host and metrics_server:port.

Why not expose the /metrics endpoint in one of the existing express apps?

The standalone express app exists for two main reasons:

1. We don't want these metrics to be public, and the easiest way to accomplish that is to expose the /metrics endpoint at a different port that won't be exposed to the internet.

2. Creating a standalone express instance decouples the metrics endpoint from the Ghost server, so if Ghost is not responding for whatever reason, we should still be able to scrape metrics to understand what's going on internally.

Impact on Boot & Shut down time

The prometheus client is initialized early in the boot process so we can collect metrics during the boot sequence. Testing locally has shown that this increases boot time by ~20ms. The metrics server which exposes the /metrics endpoint is not initialized until after the background services, and it is not awaited, to avoid impacting boot time. None of this code, including the requires, will run if the metrics_server:enabled flag is set to false (or not set).

Shutting down the metrics server is added as a cleanup task for the main Ghost server instance, and is setup to shut down with 0 grace period to avoid impacting shut down time.

[codecov]

Codecov Report

Attention: Patch coverage is 94.93671% with 4 lines in your changes missing coverage. Please review.

Project coverage is 73.83%. Comparing base (607dee2) to head (69eeee6).
Report is 52 commits behind head on main.

Files with missing lines	Patch %	Lines
ghost/core/core/shared/prometheus-client.js	91.66%	3 Missing ⚠️
ghost/core/core/boot.js	97.67%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #21193      +/-   ##
==========================================
+ Coverage   73.70%   73.83%   +0.13%     
==========================================
  Files        1285     1286       +1     
  Lines       76010    76128     +118     
  Branches    10114    10137      +23     
==========================================
+ Hits        56021    56209     +188     
+ Misses      19060    18983      -77     
- Partials      929      936       +7     

Flag	Coverage Δ
admin-tests	46.57% <ø> (+0.47%)	⬆️
e2e-tests	81.71% <94.93%> (+0.04%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.