Core API, formatting by env, req support, tear down

[myovchev]

Summary

• Add structured logging API - logDebug, logInfo, logWarnandlogError` (can be used in any module):

self.logError('event-type');
self.logError('event-type', { some: 'data' });
self.logError('event-type', 'some message', { some: 'data' });
// prepend `req` to also log valuable request data, followed by any of the above arguments
self.logError(req, ...);

• Custom loggers can now be destroyed - provide async destroy() method.
• Default logger now stringifies objects.
• Production and non-production default policy is applied for structured logging.
• Filtering by severity and event type per module can be configured for structured logging.
• Format logs for readability in non-production environments.

Closes PRO-4479, PRO-4480, PRO-4481, PRO-4482, PRO-4483, PRO-4486, PRO-4487.

What are the specific steps to test this change?

See the above mentioned tickets acceptance criteria.

What kind of change does this PR introduce?

(Check at least one)

• Bug fix
• New feature
• Refactor
• Documentation
• Build-related changes
• Other

Make sure the PR fulfills these requirements:

• It includes a) the existing issue ID being resolved, b) a convincing reason for adding this feature, or c) a clear description of the bug it resolves
• The changelog is updated
• Related documentation has been updated
• Related tests have been updated

If adding a new feature without an already open issue, it's best to open a feature request issue first and wait for approval before working on it.

Other information:

[linear]

PRO-4479 As a developer I can call the new structured logging API methods

Technical strategy

See Structured logging API section of the Tech Design, which can be found here: https://github.com/apostrophecms/tech-designs/blob/main/3.x/structured-logging/design.md#structured-logging-api

As well as:

• Omitting the data object
• Interpolation

Acceptance Criteria

• Calls to the new log methods reach the terminal in the format described in the tech design.
• Unit tests verify the above. (Inject a custom logger object for your tests)
• See "Interpolation" above

[boutell]

Direction feels good. I know you're not asking for review yet but I did spot a few small things.

[linear]

PRO-4480 As a developer I can pass req to the new structured logging API to automatically include more data about the request

Technical Strategy

See Logging related to a request in the Tech Design.

Acceptance Criteria

• The specified properties of req arrive in the log output in the format specified in the tech design.
• Unit tests verify the above.

PRO-4483 As a developer I can ensure my custom logger is properly shut down

Please see tech design:

https://github.com/apostrophecms/tech-designs/blob/main/3.x/structured-logging/design.md#custom-loggers

Acceptance Criteria:

• If a custom logger has a destroy method, it is awaited when Apostophe shuts down.
• Unit test verifies the above.

PRO-4482 As a devops person I can decide which log levels and events merit my attention

See Filtering in the Tech Design for technical details. Be sure to read through "Default Policy" as well as the environment variable override option.

Acceptance criteria

• Default policy: When NODE_ENV is not production, we log everything.
• Default policy: when NODE_ENV is production, we log only warnand error severities.
• Custom policy: see the tech design for the possibilities.
• Unit tests verify all of the above.

PRO-4481 As a devops person I can configure Apostrophe to send only objects to the configured logger

See Pure object mode in the Tech Design for the technical approach.

Acceptance Criteria

• When messageAs is set as an option to the apostrophecms/log module, self.apos.util.info and friends don't pass the message as a first argument to the logger. Instead they pass only an object argument, with the message moved to the property specified by messageAs.
• Unit tests verify the above.

PRO-4486 As a developer I can visually scan structured logs in the terminal

See Formatting of logs in development versus production in the Tech Design for the technical strategy.

Acceptance criteria

• In both production and development (regardless of NODE_ENV), any arguments passed to the default logger (the backend of util.info, util.error, etc) should be output, separated by a space. If any of those arguments are objects they should be output as JSON.
• If NODE_ENV is not production then JSON should be formatted in a friendly way: JSON.stringify(object, null, 2)
• See separate ticket for the production case.
• Unit tests verify the above.

PRO-4487 As a devops person I can analyze structured logs with tools that expect one log entry per line and minimal performance impact at runtime

See Formatting of logs in development versus production in the Tech Design for the technical strategy.

Acceptance criteria

• In both production and development (regardless of NODE_ENV), any arguments passed to the default logger (the backend of util.info, util.error, etc) should be output, separated by a space. If any of those arguments are objects they should be output as JSON.
• If NODE_ENV is production then JSON should be formatted without breaking the "one line per event" rule: JSON.stringify(object) with no indentation etc.
• See separate ticket for the development case.
• Unit tests verify the above.

[boutell]

Generally great. One or two functional concerns and some performance recommendations.