Logging timestamps in ms or sec is problematic

[adriangb]

The logging data model states that timestamps should be an integer in nanoseconds. We use DataDog and this is problematic because their backend can't process these logs without quite a bit of customization (and without that customization, it drops them completely).

Could someone point me to the benefits/discussion of logging in ns as opposed to (what I think is more standard) seconds or milliseconds? Is there any hope to expand it to be int64 | float where floats are assumed to be seconds or something?

[SorianoMarmol]

+1

[trask]

are you sending the logs to DataDog over OTLP? if so, it sounds like a bug in the DataDog OTLP implementation, see:

https://github.com/open-telemetry/opentelemetry-proto/blob/ac3242b03157295e4ee9e616af53b81517b06559/opentelemetry/proto/logs/v1/logs.proto#L137-L140

[adriangb]

That is not what I was doing. This was over a year ago and I no longer work on that project but at the time we were sending the logs to stdout where I believe the datadog agent running in the cluster eventually picked them up.

[trask]

if not OTLP, which logs exporter were you using?

[adriangb]

Like I said, no exporter, if I recall correctly we'd write to stdout then have the "old school" pipeline of taking stdout and shipping it to DataDog where DataDog parses the text into structured data. This was a time when I believe there wasn't even stability around exporters for logs.

In any case, I no longer work at that company and I think the point here was more that picking a time unit that differs from what I understand most pre-existing logging implementations use by default caused friction.