Skip to content
Elena Zannoni edited this page Oct 23, 2024 · 32 revisions

DTrace on Linux

Introduction

This is the Linux port of the Solaris Dynamic Tracing Framework.

Contact us

IRC: irc.libera.chat #linux-dtrace channel

Please consider joining our development list: dtrace at lists.linux.dev (preferred) (or dtrace-devel at oss.oracle.com)

Source code and dependencies

DTrace is a userspace utility. The latest development code can be found in the 2.0-branch-dev branch of this dtrace-utils repository. The various releases can also be found here.

DTrace for Linux depends on libctf (part of newer binutils, with full functionality in version 2.36.1 or later) or libdtrace-ctf. While libctf is preferred, building against libdtrace-ctf is fully supported. Libdtrace-ctf can be found at: https://github.com/oracle/libdtrace-ctf

DTrace for Linux makes use of BPF library functions that are compiled at build time. It depends on BPF support in GCC and binutils to generate the pre-compiled BPF function library.

In addition, DTrace for Linux makes use of 3 tracing support features that are not commonly available in the Linux kernel:

  • CTF data generation at compile time: this provides important datatype information for kernel and kernel module symbols. (NOTE: we are going to try to upstream this once we work on CTFv4 and BTF see the proposal from LPC 2024: https://lpc.events/event/18/contributions/1926/ and the CTF todo list here: https://sourceware.org/binutils/wiki/CTF/Todo)
  • NOT NEEDED ANY LONGER: waitfd() system call: this provides a mechanism to monitor for state changes of processes that are being traced. The needed functionality is now done inside dtrace itself w/o the need for kernel changes.
  • NOT NEEDED ANY LONGER (October 2024): /proc/kallsyms module grouping support: this adds annotation with module names to all symbols that are part of a kernel module even if the module is configured to be compiled into the kernel proper. This allows for consistent ways to refer to probes by module and function (or probe) name. After many attempts to upstream this, we the patches have been MERGED into 6.12-rc1: https://lore.kernel.org/lkml/172720929019.4186044.5884636763066616855.pr-tracker-bot@kernel.org/. The merged patchset is slightly modified from the initial one.

These additional support features for tracing are available at: https://github.com/oracle/dtrace-linux-kernel/tree/v2/5.14.9 (as new kernels are released we'll forward port the code, look for additional branches).

Use of Branches in this Repository

  • devel: this is where patches which are reviewed and accepted (on the dtrace at lists.linux.dev mailing list) are added
  • 2.0-branch-dev: Patches are merged from the devel branch into this branch when they are included into a release of DTrace
  • 2.0-branch: This has the releases of DTrace for Oracle Linux. This branch includes additional patches specific to the builds for Oracle Linux.

Latest Release

The latest release of DTrace is 2.0.1 (released May 2024). Release notes are here. Dtrace 2.0.1 Release Notes

Implementation Status

You can see where we are with the implementation in this wiki page which lists the items that are done and those still to be finished to reach parity with the Version 1 of DTrace (the one not based on BPF): Implementation Plan

Project History

DTrace for Linux started in 2011. Since then and until 2019, DTrace for Linux required a set of custom kernel modules and core kernel support functions for its implementation. This design continued through DTrace version 1.2.1.

That version is still maintained and it supports nearly all language features available in Solaris DTrace, and a great many of the providers, including FBT, USDT and pid. Perf events are available as the perf provider.

The source code for the kernel changes for these DTrace versions is available in the dtrace-linux-kernel Github repo in the v1/* branches, ported to the most recent versions of releases Linux kernel.

These versions of DTrace require the libdtrace-ctf type storage library.

Starting from 2020, with release 2.0.0, we have re-implemented DTrace for Linux utilizing BPF and other core kernel tracing functionality. These more recent versions of DTrace are mentioned at the beginning of this page (https://github.com/oracle/dtrace-utils/wiki/DTrace-on-Linux#source-code-and-dependencies). All DTrace functionality is being moved from kernel space to userspace, and it is being re-implemented. Please refer to the Release Notes in this wiki, to see what functionality is incrementally being added.

Presentation Materials

Availability

Kernel OL release Arch Repository
5.15 UEK7 OL9 X86 https://yum.oracle.com/repo/OracleLinux/OL9/UEKR7/x86_64/index.html
5.15 UEK7 OL9 AArch64 dtrace RPM: https://yum.oracle.com/repo/OracleLinux/OL9/baseos/latest/aarch64/index.html
5.15 UEK7 OL9 AArch64 dtrace-devel & dtrace-testsuite RPMS: https://yum.oracle.com/repo/OracleLinux/OL9/appstream/aarch64/index.html
5.15 UEK7 OL8 X86 http://yum.oracle.com/repo/OracleLinux/OL8/UEKR7/x86_64/index.html
5.15 UEK7 OL8 AArch64 dtrace RPM: https://yum.oracle.com/repo/OracleLinux/OL8/baseos/latest/aarch64/index.html
5.15 UEK7 OL8 AArch64 dtrace-devel & dtrace-testsuite RPMS: https://yum.oracle.com/repo/OracleLinux/OL8/appstream/aarch64/index.html
5.4 UEK6 OL8 X86 http://yum.oracle.com/repo/OracleLinux/OL8/UEKR6/x86_64/index.html
5.4 UEK6 OL8 AArch64 dtrace RPM: https://yum.oracle.com/repo/OracleLinux/OL8/baseos/latest/aarch64/index.html
5.4 UEK6 OL8 AArch64 dtrace-devel & dtrace-testsuite RPMS: https://yum.oracle.com/repo/OracleLinux/OL8/appstream/aarch64/index.html
5.4 UEK6 OL7 X86 http://yum.oracle.com/repo/OracleLinux/OL7/UEKR6/x86_64/index.html
5.4 UEK6 OL7 AArch64 http://yum.oracle.com/repo/OracleLinux/OL7/UEKR6/aarch64/index.html

Licensing

The license for the various components of DTrace, post-0.6.1, is:

Documentation

Oracle docs:

Training Videos: