XRay trace ID conversion

[errordeveloper]

I am looking at XRay exporter code, and I'm not quite sure how it's actually meant to work.

opentelemetry-collector-contrib/exporter/awsxrayexporter/internal/translator/segment.go

Lines 257 to 304 in ac26625

	// convertToAmazonTraceID converts a trace ID to the Amazon format.
	//
	// A trace ID unique identifier that connects all segments and subsegments
	// originating from a single client request.
	// * A trace_id consists of three numbers separated by hyphens. For example,
	// 1-58406520-a006649127e371903a2de979. This includes:
	// * The version number, that is, 1.
	// * The time of the original request, in Unix epoch time, in 8 hexadecimal digits.
	// * For example, 10:00AM December 2nd, 2016 PST in epoch time is 1480615200 seconds,
	// or 58406520 in hexadecimal.
	// * A 96-bit identifier for the trace, globally unique, in 24 hexadecimal digits.
	func convertToAmazonTraceID(traceID pdata.TraceID) (string, error) {
	const (
	// maxAge of 28 days. AWS has a 30 day limit, let's be conservative rather than
	// hit the limit
	maxAge = 60 * 60 * 24 * 28
	// maxSkew allows for 5m of clock skew
	maxSkew = 60 * 5
	)
	var (
	content = [traceIDLength]byte{}
	epochNow = time.Now().Unix()
	traceIDBytes = traceID.Bytes()
	epoch = int64(binary.BigEndian.Uint32(traceIDBytes[0:4]))
	b = [4]byte{}
	)
	// If AWS traceID originally came from AWS, no problem. However, if oc generated
	// the traceID, then the epoch may be outside the accepted AWS range of within the
	// past 30 days.
	//
	// In that case, we return invalid traceid error
	if delta := epochNow - epoch; delta > maxAge || delta < -maxSkew {
	return "", fmt.Errorf("invalid xray traceid: %s", traceID.HexString())
	}
	binary.BigEndian.PutUint32(b[0:4], uint32(epoch))
	content[0] = '1'
	content[1] = '-'
	hex.Encode(content[2:10], b[0:4])
	content[10] = '-'
	hex.Encode(content[identifierOffset:], traceIDBytes[4:16]) // overwrite with identifier
	return string(content[0:traceIDLength]), nil
	}

So XRay IDs are supposed to be much longer (35 bytes, not 16), however the code seems to ignore the length and just checks the epoch bytes... I'm not sure if this well intended, or a bug?

[Aneurysm9]

I believe this is correct. Skipping the timestamp validation and just looking at the traceID construction:

	content[0] = '1'
	content[1] = '-'
	hex.Encode(content[2:10], b[0:4])
	content[10] = '-'
	hex.Encode(content[identifierOffset:], traceIDBytes[4:16]) // overwrite with identifier

	return string(content[0:traceIDLength]), nil

traceIDLength is a constant 35. content starts as [traceIDLength]byte{}.

• 2 bytes for 1-
• 8 bytes for a hex-encoded representation of the 4 byte timestamp
• 1 byte for -
• 24 bytes for a hex-encoded representation of the 12 bytes in the traceID following the timestamp

This fills all 35 bytes of content and then returns it as a string.

[errordeveloper]

In my case I am just ingesting spans with generated IDs (similarly to what kspan does). XRay exporter seems to reject these spans, I was hoping it would try convert the IDs by prepending a timestamp... what am I missing?

[Aneurysm9]

Unfortunately the traceID must be generated with the first 32 bits representing a timestamp if the spans are to be sent to XRay. It is not feasible later to convert a random traceID to one that contains a timestamp as different spans from the same trace may end up being processed differently and thus with different timestamps injected into their traceID, which would break the trace.

kspan complicates things further in that it does not use randomly generated traceIDs, but instead constructs a traceID from a hash of object data. It may be possible to find a way to use a consistent timestamp there, but it's not immediately obvious.

[errordeveloper]

@Aneurysm9 thanks for explaining this. It makes sense to me.

I think that the documentation needs to be clearer about what exactly happens. Right now it's not very clear from the readme. When I looked at the code linked above, the function name (convertToAmazonTraceID) implied conversion does take place and the documentation didn't make it very clear. Also, even the function argument is pdata.TraceID, so it must be attempting conversion, and that is actually the case as you described, however it's not sufficiently explicit that conversion will be done for trace IDs that do contain at timestamp already.

[Aneurysm9]

It looks like it was originally creating new traceIDs with current timestamps, but that was changed in #1366 because it wasn't doing the right thing. Unfotunately, the README wasn't also updated.

@mxiamxia are you, or someone from your team, able to review and update the documentation around this?