Suppressing auto-instrumented span creation locally

[carlpett]

This might be a bit of an unusual case, but hoping there's a way to accomplish it. One of our services has a background cron scheduler which polls our database for updates very frequently. Due to reasons of legacy, we can't really change this at the moment. This leads to most of our traces from this service being just a single database lookup span.
Is there a way to suppress sampling within a specific scope? We're using auto-instrumentation, so we don't have direct control over the code which starts the trace.

Basically we have something like this:

schedule.scheduleJob('*/2 * * * * *', reloadAuth)

Where reloadAuth is a MongoDB find operation.

[carlpett]

I did find api.suppressInstrumentation, but I haven't managed to make it work yet. I might still be a bit confused regarding contexts and that is the entire problem?
Just chucking api.suppressInstrumentation(api.context.active()) into the first line of reloadAuth isn't sufficient, at least.

[Flarna]

opentelemetry-instrumentation-mongoose is hosted on a different repo. Looking into the code it seems it creates a new trace if no span is active.

There was a discussion a while ago to agree on a config option for all instrumentations but this has not been finalized till now (see open-telemetry/opentelemetry-js-contrib#344).

Maybe better to ask there if it is possible to add a configuration to disable creation of traces.

A workaround could be to change your reloadAuth() function to something like this:

function reloadAuth() {
  // use of ROOT_CONTEXT instead context.active() is fine here as you don't want to pass anything from current context into this call tree
  // you could even create a 'suppressContext = opentelemetry.suppressInstrumentation(ROOT_CONTEXT)' once and reuse it
  return opentelemetry.context.with(opentelemetry.suppressInstrumentation(ROOT_CONTEXT), OurMongooseSchema.find, OurMongooseSchema, { /* match */ });
}

difference is that your code only creates a new context with suppress set but doesn't use it for anything. Above code executes OurMongooseSchema.find() with this context set as active.

fyi @blumamir

[blumamir]

Hi @carlpett ,
We usually add config option like requireParentSpan or ignoreOrphandSpans to the instrumentation that does what you want:
https://github.com/aspecto-io/opentelemetry-ext-js/blob/master/doc/conventions.md#ignoreorphanedspans

Looks like it's missing from mongoose. Created an issue for that: aspecto-io/opentelemetry-ext-js#106

I'll add it in the following days, If you like, you are welcome to create a PR yourself and we'll be happy to review it. You can see how it's done in other instrumentation libraries like sequelize

[blumamir]

aspecto-io/opentelemetry-ext-js#108

[blumamir]

published: https://github.com/aspecto-io/opentelemetry-ext-js/releases/tag/opentelemetry-instrumentation-mongoose%400.4.1

[carlpett]

Sorry for the lack of response, and big thanks for the fix 🙏