Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

opentelemetry python trace export does not work well on fork #4221

Closed
yurneroma opened this issue Oct 9, 2024 · 3 comments
Closed

opentelemetry python trace export does not work well on fork #4221

yurneroma opened this issue Oct 9, 2024 · 3 comments

Comments

@yurneroma
Copy link

What are you trying to achieve?
I init a tracerProvider in main process . and then i start a new sub process, i children process i start a span. but the span will not export to the endpoint.
here is my code:

import os
from opentelemetry import  trace

from opentelemetry.exporter.otlp.proto.http.trace_exporter import (
    OTLPSpanExporter,
)
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from multiprocessing import Process
import multiprocessing as mp

def init_tracer():
    resource = Resource.create(
        attributes={
            "service.name": "api-service",
            # If workers are not distinguished within attributes, traces and
            # metrics exported from each worker will be indistinguishable. While
            # not necessarily an issue for traces, it is confusing for almost
            # all metric types. A built-in way to identify a worker is by PID
            # but this may lead to high label cardinality. An alternative
            # workaround and additional discussion are available here:
            # https://github.com/benoitc/gunicorn/issues/1352
            "worker": os.getpid(),
        }
    )

    trace.set_tracer_provider(TracerProvider(resource=resource))
    # This uses insecure connection for the purpose of example. Please see the
    # OTLP Exporter documentation for other options.
    span_processor = BatchSpanProcessor(
            OTLPSpanExporter(endpoint="http://tempo.mytempo.cn:4318/v1/traces")
    )

    trace.get_tracer_provider().add_span_processor(span_processor)

def worker_func(name):
    info("worker function")

import time
def info(title):
    with trace.get_tracer(__name__).start_as_current_span('multi-span') as span:
        span.set_attribute("pid", os.getpid())
        print(f"info function {title} running, span context : {trace.get_current_span().get_span_context()}")
        time.sleep(3)




if __name__ == '__main__':
    init_tracer()
    print('----------------------')

    mp.set_start_method('fork')
    p = Process(target=worker_func, args=('bob',))
    p.start()
    p.join()

What did you expect to see?
i can reinitialize the global TracerProvider, then the BatchSpanProcessor.worker() function will work well in subprocess.
Additional context.
i have seen the implementation. it seems the SynchronousMultiSpanProcessor does not be copy when i initialize the TracerProvider in main process and then fork. so when the child process end, the SynchronousMultiSpanProcessor will not call shutdown in child process. and then the BatchSpanProcessor in child process will not call shutdown() function, and so the BatchSpanProcessor.worker() will not be joined by shutdown() function. so it does not work well in my code.
if we have a function that can reinitialize the TracerProvider in child process. it will work well. just like i use a spawn method to new a child process. i can new a TracerProvider in child process.
Add any other context about the problem here. If you followed an existing documentation, please share the link to it.

@dmathieu
Copy link
Member

dmathieu commented Oct 9, 2024

This seems like something for the otel-python repository.

@trask trask transferred this issue from open-telemetry/opentelemetry-specification Oct 11, 2024
@yurneroma
Copy link
Author

This seems like something for the otel-python repository.
thanks, i write another issue under the otel-python repo. link: #4215

@arminru
Copy link
Member

arminru commented Oct 15, 2024

Duplicate of (the more comprehensive) #4215

@arminru arminru closed this as not planned Won't fix, can't repro, duplicate, stale Oct 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants