Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add stdout memory mode #6774

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
/*
* Copyright The OpenTelemetry Authors
* SPDX-License-Identifier: Apache-2.0
*/

package io.opentelemetry.exporter.logging.otlp.internal;

import io.opentelemetry.sdk.common.export.MemoryMode;

public class OtlpStdoutExporterBuilderUtil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing internal boilerplate javadoc.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why did the build not fail?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's not a build check for it right now. Do you know if opentelemetry-java-instrumentation managed to add this automation yet?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened new issue here: #6798

public OtlpStdoutExporterBuilderUtil() {}

Check warning on line 11 in exporters/logging-otlp/src/main/java/io/opentelemetry/exporter/logging/otlp/internal/OtlpStdoutExporterBuilderUtil.java

View check run for this annotation

Codecov / codecov/patch

exporters/logging-otlp/src/main/java/io/opentelemetry/exporter/logging/otlp/internal/OtlpStdoutExporterBuilderUtil.java#L11

Added line #L11 was not covered by tests

public static void validate(MemoryMode memoryMode, boolean wrapperJsonObject) {
if (memoryMode == MemoryMode.REUSABLE_DATA && !wrapperJsonObject) {
throw new IllegalArgumentException(
"Reusable data mode is not supported without wrapperJsonObject");
zeitlinger marked this conversation as resolved.
Show resolved Hide resolved
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,12 @@

package io.opentelemetry.exporter.logging.otlp.internal.logs;

import io.opentelemetry.exporter.internal.otlp.logs.LogsRequestMarshaler;
import io.opentelemetry.exporter.internal.marshal.Marshaler;
import io.opentelemetry.exporter.internal.otlp.logs.LogReusableDataMarshaler;
import io.opentelemetry.exporter.internal.otlp.logs.ResourceLogsMarshaler;
import io.opentelemetry.exporter.logging.otlp.internal.writer.JsonWriter;
import io.opentelemetry.sdk.common.CompletableResultCode;
import io.opentelemetry.sdk.common.export.MemoryMode;
import io.opentelemetry.sdk.logs.data.LogRecordData;
import io.opentelemetry.sdk.logs.export.LogRecordExporter;
import java.util.Collection;
Expand All @@ -33,11 +35,14 @@ public final class OtlpStdoutLogRecordExporter implements LogRecordExporter {
private final Logger logger;
private final JsonWriter jsonWriter;
private final boolean wrapperJsonObject;
private final MemoryMode memoryMode;

OtlpStdoutLogRecordExporter(Logger logger, JsonWriter jsonWriter, boolean wrapperJsonObject) {
OtlpStdoutLogRecordExporter(
Logger logger, JsonWriter jsonWriter, boolean wrapperJsonObject, MemoryMode memoryMode) {
this.logger = logger;
this.jsonWriter = jsonWriter;
this.wrapperJsonObject = wrapperJsonObject;
this.memoryMode = memoryMode;
}

/** Returns a new {@link OtlpStdoutLogRecordExporterBuilder}. */
Expand All @@ -53,8 +58,12 @@ public CompletableResultCode export(Collection<LogRecordData> logs) {
}

if (wrapperJsonObject) {
LogsRequestMarshaler request = LogsRequestMarshaler.create(logs);
return jsonWriter.write(request);
return new LogReusableDataMarshaler(memoryMode) {
zeitlinger marked this conversation as resolved.
Show resolved Hide resolved
@Override
public CompletableResultCode doExport(Marshaler exportRequest, int numItems) {
return jsonWriter.write(exportRequest);
}
}.export(logs);
} else {
for (ResourceLogsMarshaler resourceLogs : ResourceLogsMarshaler.create(logs)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't seem to be the case that if wrapperJsonObject = false, we can't take advantage of reuseable memory mode. Can we include wrapperJsonObject as an argument to LogReuseableDataMarshaler and adjust the logic accordingly?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's not so easy, because

takes a list and I didn't follow what has to be modified in there

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok let's punt on it for now

CompletableResultCode resultCode = jsonWriter.write(resourceLogs);
Expand Down Expand Up @@ -87,6 +96,7 @@ public String toString() {
StringJoiner joiner = new StringJoiner(", ", "OtlpStdoutLogRecordExporter{", "}");
joiner.add("jsonWriter=" + jsonWriter);
joiner.add("wrapperJsonObject=" + wrapperJsonObject);
joiner.add("memoryMode=" + memoryMode);
return joiner.toString();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,11 @@
import static java.util.Objects.requireNonNull;

import io.opentelemetry.exporter.logging.otlp.OtlpJsonLoggingLogRecordExporter;
import io.opentelemetry.exporter.logging.otlp.internal.OtlpStdoutExporterBuilderUtil;
import io.opentelemetry.exporter.logging.otlp.internal.writer.JsonWriter;
import io.opentelemetry.exporter.logging.otlp.internal.writer.LoggerJsonWriter;
import io.opentelemetry.exporter.logging.otlp.internal.writer.StreamJsonWriter;
import io.opentelemetry.sdk.common.export.MemoryMode;
import java.io.OutputStream;
import java.util.logging.Logger;

Expand All @@ -27,6 +29,7 @@ public final class OtlpStdoutLogRecordExporterBuilder {
private final Logger logger;
private JsonWriter jsonWriter;
private boolean wrapperJsonObject = true;
private MemoryMode memoryMode = MemoryMode.IMMUTABLE_DATA;

public OtlpStdoutLogRecordExporterBuilder(Logger logger) {
this.logger = logger;
Expand All @@ -44,6 +47,17 @@ public OtlpStdoutLogRecordExporterBuilder setWrapperJsonObject(boolean wrapperJs
return this;
}

/**
* Set the {@link MemoryMode}. If unset, defaults to {@link MemoryMode#IMMUTABLE_DATA}.
*
* <p>When memory mode is {@link MemoryMode#REUSABLE_DATA}, serialization is optimized to reduce
* memory allocation.
*/
public OtlpStdoutLogRecordExporterBuilder setMemoryMode(MemoryMode memoryMode) {
this.memoryMode = memoryMode;
return this;
}

/**
* Sets the exporter to use the specified output stream.
*
Expand Down Expand Up @@ -71,6 +85,7 @@ public OtlpStdoutLogRecordExporterBuilder setOutput(Logger logger) {
* @return a new exporter's instance
*/
public OtlpStdoutLogRecordExporter build() {
return new OtlpStdoutLogRecordExporter(logger, jsonWriter, wrapperJsonObject);
OtlpStdoutExporterBuilderUtil.validate(memoryMode, wrapperJsonObject);
return new OtlpStdoutLogRecordExporter(logger, jsonWriter, wrapperJsonObject, memoryMode);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

package io.opentelemetry.exporter.logging.otlp.internal.logs;

import io.opentelemetry.exporter.internal.ExporterBuilderUtil;
import io.opentelemetry.sdk.autoconfigure.spi.internal.ComponentProvider;
import io.opentelemetry.sdk.autoconfigure.spi.internal.StructuredConfigProperties;
import io.opentelemetry.sdk.logs.export.LogRecordExporter;
Expand All @@ -31,6 +32,7 @@ public String getName() {
@Override
public LogRecordExporter create(StructuredConfigProperties config) {
OtlpStdoutLogRecordExporterBuilder builder = OtlpStdoutLogRecordExporter.builder();
ExporterBuilderUtil.configureExporterMemoryMode(config, builder::setMemoryMode);
return builder.build();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

package io.opentelemetry.exporter.logging.otlp.internal.logs;

import io.opentelemetry.exporter.internal.ExporterBuilderUtil;
import io.opentelemetry.sdk.autoconfigure.spi.ConfigProperties;
import io.opentelemetry.sdk.autoconfigure.spi.logs.ConfigurableLogRecordExporterProvider;
import io.opentelemetry.sdk.logs.export.LogRecordExporter;
Expand All @@ -20,6 +21,7 @@ public final class OtlpStdoutLogRecordExporterProvider
@Override
public LogRecordExporter createExporter(ConfigProperties config) {
OtlpStdoutLogRecordExporterBuilder builder = OtlpStdoutLogRecordExporter.builder();
ExporterBuilderUtil.configureExporterMemoryMode(config, builder::setMemoryMode);
return builder.build();
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,12 @@

package io.opentelemetry.exporter.logging.otlp.internal.metrics;

import io.opentelemetry.exporter.internal.otlp.metrics.MetricsRequestMarshaler;
import io.opentelemetry.exporter.internal.marshal.Marshaler;
import io.opentelemetry.exporter.internal.otlp.metrics.MetricReusableDataMarshaler;
import io.opentelemetry.exporter.internal.otlp.metrics.ResourceMetricsMarshaler;
import io.opentelemetry.exporter.logging.otlp.internal.writer.JsonWriter;
import io.opentelemetry.sdk.common.CompletableResultCode;
import io.opentelemetry.sdk.common.export.MemoryMode;
import io.opentelemetry.sdk.metrics.Aggregation;
import io.opentelemetry.sdk.metrics.InstrumentType;
import io.opentelemetry.sdk.metrics.data.AggregationTemporality;
Expand Down Expand Up @@ -37,18 +39,21 @@ public final class OtlpStdoutMetricExporter implements MetricExporter {
private final Logger logger;
private final JsonWriter jsonWriter;
private final boolean wrapperJsonObject;
private final MemoryMode memoryMode;
private final AggregationTemporalitySelector aggregationTemporalitySelector;
private final DefaultAggregationSelector defaultAggregationSelector;

OtlpStdoutMetricExporter(
Logger logger,
JsonWriter jsonWriter,
boolean wrapperJsonObject,
MemoryMode memoryMode,
AggregationTemporalitySelector aggregationTemporalitySelector,
DefaultAggregationSelector defaultAggregationSelector) {
this.logger = logger;
this.jsonWriter = jsonWriter;
this.wrapperJsonObject = wrapperJsonObject;
this.memoryMode = memoryMode;
this.aggregationTemporalitySelector = aggregationTemporalitySelector;
this.defaultAggregationSelector = defaultAggregationSelector;
}
Expand All @@ -69,15 +74,24 @@ public Aggregation getDefaultAggregation(InstrumentType instrumentType) {
return defaultAggregationSelector.getDefaultAggregation(instrumentType);
}

@Override
public MemoryMode getMemoryMode() {
return memoryMode;
}

@Override
public CompletableResultCode export(Collection<MetricData> metrics) {
if (isShutdown.get()) {
return CompletableResultCode.ofFailure();
}

if (wrapperJsonObject) {
MetricsRequestMarshaler request = MetricsRequestMarshaler.create(metrics);
return jsonWriter.write(request);
return new MetricReusableDataMarshaler(memoryMode) {
@Override
public CompletableResultCode doExport(Marshaler exportRequest, int numItems) {
return jsonWriter.write(exportRequest);
}
}.export(metrics);
} else {
for (ResourceMetricsMarshaler resourceMetrics : ResourceMetricsMarshaler.create(metrics)) {
CompletableResultCode resultCode = jsonWriter.write(resourceMetrics);
Expand Down Expand Up @@ -110,6 +124,13 @@ public String toString() {
StringJoiner joiner = new StringJoiner(", ", "OtlpStdoutMetricExporter{", "}");
joiner.add("jsonWriter=" + jsonWriter);
joiner.add("wrapperJsonObject=" + wrapperJsonObject);
joiner.add("memoryMode=" + memoryMode);
joiner.add(
"aggregationTemporalitySelector="
+ AggregationTemporalitySelector.asString(aggregationTemporalitySelector));
joiner.add(
"defaultAggregationSelector="
+ DefaultAggregationSelector.asString(defaultAggregationSelector));
zeitlinger marked this conversation as resolved.
Show resolved Hide resolved
return joiner.toString();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,11 @@
import static java.util.Objects.requireNonNull;

import io.opentelemetry.exporter.logging.otlp.OtlpJsonLoggingMetricExporter;
import io.opentelemetry.exporter.logging.otlp.internal.OtlpStdoutExporterBuilderUtil;
import io.opentelemetry.exporter.logging.otlp.internal.writer.JsonWriter;
import io.opentelemetry.exporter.logging.otlp.internal.writer.LoggerJsonWriter;
import io.opentelemetry.exporter.logging.otlp.internal.writer.StreamJsonWriter;
import io.opentelemetry.sdk.common.export.MemoryMode;
import io.opentelemetry.sdk.metrics.InstrumentType;
import io.opentelemetry.sdk.metrics.export.AggregationTemporalitySelector;
import io.opentelemetry.sdk.metrics.export.DefaultAggregationSelector;
Expand Down Expand Up @@ -40,6 +42,7 @@ public final class OtlpStdoutMetricExporterBuilder {
private final Logger logger;
private JsonWriter jsonWriter;
private boolean wrapperJsonObject = true;
private MemoryMode memoryMode = MemoryMode.IMMUTABLE_DATA;

public OtlpStdoutMetricExporterBuilder(Logger logger) {
this.logger = logger;
Expand All @@ -57,6 +60,17 @@ public OtlpStdoutMetricExporterBuilder setWrapperJsonObject(boolean wrapperJsonO
return this;
}

/**
* Set the {@link MemoryMode}. If unset, defaults to {@link MemoryMode#IMMUTABLE_DATA}.
*
* <p>When memory mode is {@link MemoryMode#REUSABLE_DATA}, serialization is optimized to reduce
* memory allocation.
*/
public OtlpStdoutMetricExporterBuilder setMemoryMode(MemoryMode memoryMode) {
this.memoryMode = memoryMode;
return this;
}

/**
* Sets the exporter to use the specified output stream.
*
Expand Down Expand Up @@ -114,10 +128,12 @@ public OtlpStdoutMetricExporterBuilder setDefaultAggregationSelector(
* @return a new exporter's instance
*/
public OtlpStdoutMetricExporter build() {
OtlpStdoutExporterBuilderUtil.validate(memoryMode, wrapperJsonObject);
return new OtlpStdoutMetricExporter(
logger,
jsonWriter,
wrapperJsonObject,
memoryMode,
aggregationTemporalitySelector,
defaultAggregationSelector);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ public String getName() {
@Override
public MetricExporter create(StructuredConfigProperties config) {
OtlpStdoutMetricExporterBuilder builder = OtlpStdoutMetricExporter.builder();
ExporterBuilderUtil.configureExporterMemoryMode(config, builder::setMemoryMode);
ExporterBuilderUtil.configureOtlpAggregationTemporality(
config, builder::setAggregationTemporalitySelector);
ExporterBuilderUtil.configureOtlpHistogramDefaultAggregation(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ public final class OtlpStdoutMetricExporterProvider implements ConfigurableMetri
@Override
public MetricExporter createExporter(ConfigProperties config) {
OtlpStdoutMetricExporterBuilder builder = OtlpStdoutMetricExporter.builder();
ExporterBuilderUtil.configureExporterMemoryMode(config, builder::setMemoryMode);
ExporterBuilderUtil.configureOtlpAggregationTemporality(
config, builder::setAggregationTemporalitySelector);
ExporterBuilderUtil.configureOtlpHistogramDefaultAggregation(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,12 @@

package io.opentelemetry.exporter.logging.otlp.internal.traces;

import io.opentelemetry.exporter.internal.marshal.Marshaler;
import io.opentelemetry.exporter.internal.otlp.traces.ResourceSpansMarshaler;
import io.opentelemetry.exporter.internal.otlp.traces.TraceRequestMarshaler;
import io.opentelemetry.exporter.internal.otlp.traces.SpanReusableDataMarshaler;
import io.opentelemetry.exporter.logging.otlp.internal.writer.JsonWriter;
import io.opentelemetry.sdk.common.CompletableResultCode;
import io.opentelemetry.sdk.common.export.MemoryMode;
import io.opentelemetry.sdk.trace.data.SpanData;
import io.opentelemetry.sdk.trace.export.SpanExporter;
import java.util.Collection;
Expand All @@ -32,11 +34,14 @@ public final class OtlpStdoutSpanExporter implements SpanExporter {
private final Logger logger;
private final JsonWriter jsonWriter;
private final boolean wrapperJsonObject;
private final MemoryMode memoryMode;

OtlpStdoutSpanExporter(Logger logger, JsonWriter jsonWriter, boolean wrapperJsonObject) {
OtlpStdoutSpanExporter(
Logger logger, JsonWriter jsonWriter, boolean wrapperJsonObject, MemoryMode memoryMode) {
this.logger = logger;
this.jsonWriter = jsonWriter;
this.wrapperJsonObject = wrapperJsonObject;
this.memoryMode = memoryMode;
}

/** Returns a new {@link OtlpStdoutSpanExporterBuilder}. */
Expand All @@ -52,8 +57,12 @@ public CompletableResultCode export(Collection<SpanData> spans) {
}

if (wrapperJsonObject) {
TraceRequestMarshaler request = TraceRequestMarshaler.create(spans);
return jsonWriter.write(request);
return new SpanReusableDataMarshaler(memoryMode) {
@Override
public CompletableResultCode doExport(Marshaler exportRequest, int numItems) {
return jsonWriter.write(exportRequest);
}
}.export(spans);
} else {
for (ResourceSpansMarshaler resourceSpans : ResourceSpansMarshaler.create(spans)) {
CompletableResultCode resultCode = jsonWriter.write(resourceSpans);
Expand Down Expand Up @@ -86,6 +95,7 @@ public String toString() {
StringJoiner joiner = new StringJoiner(", ", "OtlpStdoutSpanExporter{", "}");
joiner.add("jsonWriter=" + jsonWriter);
joiner.add("wrapperJsonObject=" + wrapperJsonObject);
joiner.add("memoryMode=" + memoryMode);
return joiner.toString();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,11 @@
import static java.util.Objects.requireNonNull;

import io.opentelemetry.exporter.logging.otlp.OtlpJsonLoggingSpanExporter;
import io.opentelemetry.exporter.logging.otlp.internal.OtlpStdoutExporterBuilderUtil;
import io.opentelemetry.exporter.logging.otlp.internal.writer.JsonWriter;
import io.opentelemetry.exporter.logging.otlp.internal.writer.LoggerJsonWriter;
import io.opentelemetry.exporter.logging.otlp.internal.writer.StreamJsonWriter;
import io.opentelemetry.sdk.common.export.MemoryMode;
import java.io.OutputStream;
import java.util.logging.Logger;

Expand All @@ -27,6 +29,7 @@ public final class OtlpStdoutSpanExporterBuilder {
private final Logger logger;
private JsonWriter jsonWriter;
private boolean wrapperJsonObject = true;
private MemoryMode memoryMode = MemoryMode.IMMUTABLE_DATA;

public OtlpStdoutSpanExporterBuilder(Logger logger) {
this.logger = logger;
Expand All @@ -44,6 +47,17 @@ public OtlpStdoutSpanExporterBuilder setWrapperJsonObject(boolean wrapperJsonObj
return this;
}

/**
* Set the {@link MemoryMode}. If unset, defaults to {@link MemoryMode#IMMUTABLE_DATA}.
*
* <p>When memory mode is {@link MemoryMode#REUSABLE_DATA}, serialization is optimized to reduce
* memory allocation.
*/
public OtlpStdoutSpanExporterBuilder setMemoryMode(MemoryMode memoryMode) {
this.memoryMode = memoryMode;
return this;
}

/**
* Sets the exporter to use the specified output stream.
*
Expand Down Expand Up @@ -71,6 +85,7 @@ public OtlpStdoutSpanExporterBuilder setOutput(Logger logger) {
* @return a new exporter's instance
*/
public OtlpStdoutSpanExporter build() {
return new OtlpStdoutSpanExporter(logger, jsonWriter, wrapperJsonObject);
OtlpStdoutExporterBuilderUtil.validate(memoryMode, wrapperJsonObject);
return new OtlpStdoutSpanExporter(logger, jsonWriter, wrapperJsonObject, memoryMode);
}
}
Loading
Loading