How to use json logger with timestamp, console + file and log also non-json data?

[Ks89]

Please tell us about your environment:

• winston version?
  • winston@2
  • winston@3
• node -v outputs: 10.15.1
• Operating System? (Windows, macOS, or Linux) macOS
• Language? (all | TypeScript X.X | ES6/7 | ES5 | Dart) ES6/7

What is the problem?

I'm trying to use winston 3 to log in this way:

1. write logs both in console and files (with daily rotate)
2. write logs in console and files with this format: "timestamp - level - text message - json or string"
3. write in console with colours based on levels (everything must be coloured, both message and payload)
4. write json data in console and files in a readable and good way
5. write strings, number, arrays as second param of logger.debug. For instance: "00000 - warn - hello - [1,2,4]". In this example I'm logging an array and not a json.

How can I do that? This is my current code:

'use strict';

const {
  format,
  createLogger,
  transports,
  addColors
} = require('winston');

require('winston-daily-rotate-file');

const configLevels = {
  levels: {
    error: 0,
    debug: 1,
    warn: 2,
    data: 3,
    info: 4,
    verbose: 5,
    silly: 6,
    custom: 7
  },
  colors: {
    error: 'red',
    debug: 'green',
    warn: 'yellow',
    data: 'grey',
    info: 'white',
    verbose: 'cyan',
    silly: 'magenta',
    custom: 'yellow'
  }
};

addColors(configLevels.colors);

const logger = createLogger({
  levels: configLevels.levels,
  format: format.combine(
    // format.timestamp({
    //   format: 'YYYY-MM-DD HH:mm:ss'
    // }),
    format.simple()
    // format.printf(info => `${info.timestamp} ${info.level}: ${info.message}`)
  ),
  transports: [
    new transports.DailyRotateFile({
      filename: 'logs/myapplication-errors-%DATE%.log',
      datePattern: 'YYYY-MM-DD',
      zippedArchive: false,
      maxSize: '10m',
      maxFiles: '10',
      level: 'error',
      handleExceptions: true
    }),
    new transports.DailyRotateFile({
      filename: 'logs/myapplication-warnings-%DATE%.log',
      datePattern: 'YYYY-MM-DD',
      zippedArchive: false,
      maxSize: '10m',
      maxFiles: '10',
      level: 'warn',
      handleExceptions: true
    }),
    new transports.DailyRotateFile({
      filename: 'logs/myapplication-%DATE%.log',
      datePattern: 'YYYY-MM-DD',
      zippedArchive: false,
      maxSize: '10m',
      maxFiles: '10',
      handleExceptions: true
    })
  ],
  level: 'silly',
  exitOnError: false
});

if (process.env.NODE_ENV === 'dev') {
  logger.add(new transports.Console({
    handleExceptions: true,
    format: format.combine(
      format.colorize({
        all: true
      }),
      format.simple()
    )
  }));
}

logger.info('Hello there. How are you?');


// they aren't working
// --------------
logger.info('Hello there.', `i'm not json but I want to be logged`);

logger.info('Hello there.', `first non json`, `another non json`);

logger.info('Hello there.', 5);

logger.info('Hello there.', [1, 2, 'hello']);

logger.info('Hello there.', () => {});
// --------------

logger.custom('hello');
logger.info('Hello again logs with a small object', {
  pippo: {
    pluto: 'hello',
    ooo: 2.4,
    sasas: [1, 2, 3],
    ole: {
      test: true
    }
  }
});
logger.warn('some foobar level-ed message');
logger.error('some baz level-ed message');
logger.silly('some bar level-ed message');

logger.warn('very nested object with functions and other stuff', {
  pippo: {
    pluto: 'hello',
    zzzz: {
      a: 5,
      b: 2,
      c: {
        c: 1
      },
      d: [1, 2, 3]
    },
    ooo: 2.4,
    functionDemo: function() {
      return 'hello';
    },
    sasas: [1, {
      a: 1
    }, 3],
    ole: {
      test: [
        [
          [{
            a: 5
          }], {
            b: 3
          }
        ]
      ]
    }
  },
  a: [],
  lambdaFunction: test => console.log('hi ' + test),
  set: new Set(),
  regexp: /ab+c/
});

// uncomment to see logged exceptions in console and file
// throw new Error('Hello, winston!');

If you try to run this code you can't achieve all requirements, for instance point 5 is not working.

Also, if I try to enable timestamps with:

format.timestamp({
   format: 'YYYY-MM-DD HH:mm:ss'
}),

the result is a log like this:

info: Hello again logs with a small object {"pippo":{"pluto":"hello","ooo":2.4,"sasas":[1,2,3],"ole":{"test":true}},"timestamp":"2019-06-26 12:33:55"} 

with this 'crazy' "timestamp":"2019-06-26 12:33:55" inside my json payload.

So, I should do something like "format.printf(info => myFunction(info))", for instance "format.printf(info => ${info.timestamp} ${info.level}: ${info.message})
)," to do this,
but how can I combine format.simple() and format.printf without breaking json logger and logging on file like this: "timestamp - level - text message - json or string"?

I also tried to implement a custom logic to remove Symbol and other stuff from the object in format.printf, but it's quite complicated and probably inefficient. Which is the right way to do this?
Do you have any suggestions? Am I missing something or is there an issue somewhere?

Also, about point 3 of the list above, how to color everything in console with the json data?

Thanks

[Ks89]

Also, if not available, is there a format.something() to print json, level and timestamp by default? I think that a log without timestamps it's a little bit useless

[fourpastmidnight]

I've come across what seems to be a pretty severe issue with Winston and defining multiple transports on a logger: #1933. I think you may be running into the same issue. For the record, it is possible to have different formats per transport.

When creating a logger, you can set a "base" or "global" format. For example, when creating the logger, I set the format option to format.timestamp({ format: 'YYYY-MM-DD HH:mm:ss.SSSZZ' }). Then, for each transport, I define another format (or, in my case, a pipeline of formatters via format.combine(...). It appears (I haven't looked at winston's source code yet for this) that Winston will format.combine the logger format with the transport formatter(s), resulting in the final output. Just beware that there appears to be a fairly significant issue with using multiple transports in combination with log entry metadata that you pass in. Only the first transport will get the metadata. The metadata gets silently dropped for all subsequent transports.

[fourpastmidnight]

There is a comment on #1430 which resolves the issue with info "losing data" between transports.