OtlpHttpExporter responds with proto: illegal wireType 7

[CMThF]

Hi!
I am trying to export to an elastic cloud APM with http otlp exporters. It works from java services just fine.

With C++, I get the following response:

ERROR: Export 45 trace span(s) error: 1(../exporters/otlp/src/otlp_http_exporter.cc:L107)
OTLP HTTP Client Export failed,
Status:400, Header:.Content-Length: 29.,.Content-Type: application/x-protobuf.,
.Date: Mon, 02 Jan 2023 14:00:21 GMT.,.X-Cloud-Request-Id: XxCPi-yxRSGz_HynRfKu3g.,.X-Found-Hand ling-Cluster: 7e0c1b4e4059481092f8e4956d438e4e., .X-Found-Handling-Instance: instance-0000000001. ,
Body:....proto: illegal wireType 7(../exporters/otlp/src/otlp_http_client.cc:L116)

The response reads like there is an issue with the content format. Like, maybe the proto format is wrong? Or just the error response is not parsed correctly?

Url and authentication are okay.
Elastic states that it wants protobuf content:

APM Server supports both the (OTLP/gRPC) and (OTLP/HTTP) protocol with ProtoBuf payload. APM Server does not yet support JSON Encoding for OTLP/HTTP.

opentelemetry-cpp docs state that it is using http/protobuf:

OpenTelemetry(otlp) Exporter: sends the data to the OpenTelemetry Collector using protobuf/gRPC or protobuf/HTTP.

I am using OtlpHttpExporter with BatchSpanProcessor for tracing, and OtlpHttpMetricExporter with PeriodicExportingMetricReader for Metrics, release 1.8.1. Url and Auth token are set via environment variables.

What am I doing wrong here?

[lalitb]

@CMThF

• Are you directly exporting from C++ OTLP/HTTP exporter to Elastic, or connecting using Collector instance ?
• Do you know the otel-proto version used by Elastic ?
• Are you seeing issue with both traces and metrics ?

[CMThF]

Hi @lalitb!
Thank you for looking into this!

I am exporting directly from C++ OTLP/HTTP exporter to Elastic

Elastic uses these proto definitions v1 according to their docs here.

Yes, metrics have the same issue:

[OTLP HTTP Client] ERROR: Export 1 metric(s) error: 1
OTLP HTTP Client
] Export failed, Status:400, Header:.Content-Length: 29.,.Content-Type: application/x-protobuf.,
.Date: Tue, 03 Jan 2023 08:52:01 GMT.,.X-Cloud-Request-Id: kXtE8wMcSIqrGma3pbtYtQ.,.X-Found-Handling-Cluster: 7e0c1b4e4059481092f8e4956d438e4e., .X-Found-Handling-Instance: instance-0000000001.,
Body:....proto: illegal wireType 7

[marcalff]

@CMThF

Please double check the exporter options:

OtlpHttpExporterOptions::content_type is by default HttpRequestContentType::kJson,
so it should be changed to HttpRequestContentType::kBinary to send a protobuf message in binary format.

Same for metrics.

[lalitb]

Yeah good point, looks like Elastic doesn't support JSON format for OTLP/HTTP - https://www.elastic.co/guide/en/apm/guide/current/open-telemetry.html#open-telemetry-otlp-limitations

[CMThF]

Man, of course! It defaults to kJson! I will try that immediately! Many thanks!

[CMThF]

Works like a charm! Sorry I did not notice that json ofc is the default in the options. Thanks for pointing that out!

[marcalff]

Thanks for providing so many details, it helps to find the root cause.