Skip to content

DTrace Patches for the Linux Kernel

Jump to bottom
Elena Zannoni edited this page Oct 23, 2020 · 14 revisions

This repository hosts the upstream Linux kernel, plus the necessary patches to run DTrace. There are two flavors of patches, one set that works with the implementation of DTrace based on BPF, and the other set that works with the implementation of DTrace which doesn't use BPF and relies on a few kernel modules specific to DTrace.

Kernel Patches for BPF Based DTrace

This set of kernel patches is for the BPF based version of DTrace and applies to DTrace 2.0.0 and higher releases:

  • Add /proc/kallmodsyms which provides a mapping between kernel symbol and module name that only changes when the kernel source code is changed. This mapping does not change simply because a module becomes built into the kernel. It also provides size information to determine whether an address is within a symbol or outside it, especially given that there could be huge gaps between symbols.

  • dwarf2ctf, a tool which runs whenever 'make ctf' is run, extracting information on the kernel's types and global variables from the DWARF-format debug information in the kernel build tree, deduplicating it, and emitting it in Sun's Compact Type Format into an mmappable type archive named vmlinux.ctfa, which is installed at 'make install' time into /lib/modules/$(uname -r)/kernel/. NOTE: this tool is not necessary if you have a CTF capable toolchain.

  • waitfd(): new syscall implementing waitpid() over fds. fd = waitfd(P_PID, some_pid, WEXITED | WSTOPPED, 0); This returns a file descriptor which becomes ready whenever waitpid() would return, and when read() returns the return value waitpid() would have returned.

  • Usage of the Toolchain-based CTF support: kernel is built with -gt for CTF generation using GCC. Add two new CONFIGs for a CTF enabled toolchain and a kernel build with -gt/-lctf respectively:

    • new config HAVE_CTF_TOOLCHAIN which is set if the toolchain supports CTF generation.
    • new config CTF_USING_BINUTILS to control whether CTF generation is to be done using the GNU Toolchain. This config is unset by default. This adds the option to use a new CTF deduplicator based on the libctf linking machinery.

These two configs do not need to be set explicitly, but rather, they are automatically set when a CTF capable toolchain is being detected.

Kernel Patches for Non-BPF Based DTrace

These kernel patches are necessary for DTrace 1.2.1-1 and earlier releases.

The set of kernel patches includes some of the patches listed above plus others to provide the functionality of the various providers. In this version of DTrace, such functionality is implemented in the kernel, in the following DTrace specific modules: dtrace.ko, dt_test.ko, fasttrap.ko, fbt.ko, profile.ko, sdt.ko, systrace.ko. The complete list is as follows:

  • /proc/kallmodsyms which provides a mapping between kernel symbol and module name that only changes when the kernel source code is changed. This mapping does not change simply because a module becomes built into the kernel. It also provides size information to determine whether an address is within a symbol or outside it, especially given that there could be huge gaps between symbols.

  • dwarf2ctf, a tool which runs whenever 'make ctf' is run, extracting information on the kernel's types and global variables from the DWARF-format debug information in the kernel build tree, deduplicating it, and emitting it in Sun's Compact Type Format into an mmappable type archive named vmlinux.ctfa, which is installed at 'make install' time into /lib/modules/$(uname -r)/kernel/.

  • waitfd(): new syscall implementing waitpid() over fds. fd = waitfd(P_PID, some_pid, WEXITED | WSTOPPED, 0); This returns a file descriptor which becomes ready whenever waitpid() would return, and when read() returns the return value waitpid() would have returned.

  • dtrace: core and x86: This implements DTrace's core kernel (linked-in) components, including platform-dependent portions for x86.

  • dtrace: modular components and x86 support: the core DTrace module (including the entire DIF interpreter and support for all built-in D variables and functions) and one test provider, dt_test.ko. This adds also an x86 implementation of the architecture-dependent parts.

  • dtrace: systrace provider core components: the core (linked-in) components of the DTrace systrace provider, which intercepts system call invocations. The arch-dependent pieces needed for x86 are also provided.

  • dtrace: systrace provider: the DTrace systrace provider, which intercepts system call invocations.

  • dtrace: sdt provider core components: the core (linked-in) machinery needed for SDT tracepoints.

  • dtrace: sdt provider for x86: the SDT provider itself. It is relatively straightforward except for the code needed to parse the argument strings ultimately derived from SDT DTRACE_PROBE invocations and perf-event prototype definitions.

  • dtrace: profile provider and test probe core components: Only Kconfig changes are needed here: everything else is purely modular.

  • dtrace: profile and tick providers built on cyclics: Probes are constructed dynamically as called upon by the user: some default commonly-used probes for common timing frequencies are provided whether or not called upon.

  • dtrace: USDT and pid provider core and x86 components: the core (linked-in) machinery needed for userspace statically-defined tracepoints (for historical reasons, known as 'fasttrap' by DTrace) and for the pid provider, which allows USDT probes to be dropped at addresses that do not correspond to symbols, at locations named as a symbol plus an offset. Both are implemented in terms of kprobes.

  • dtrace: USDT and pid providers: For historical reasons, these are provided in a module named fasttrap.ko. Much of this is arch-dependent code for jump-table detection and implementation of globbed pid probes ("probe everything you can in this function"), as well as arch-dependent code to look up arguments and code to ensure that dropping a kprobe in a single process does not affect other processes running from the same binary.

  • dtrace: function boundary tracing (FBT) core and x86 components: the core components needed for FBT tracing. Unlike ftrace we allow the tracing of very large numbers of functions at once: the intent is that the system should still be stable when every eligible function in the kernel is traced simultaneously. Functions that are not safe for this (because e.g. they are used in trap handling, or by functions called by the DTrace module itself during probe processing) are (semi-manually) blacklisted from being probed.

  • dtrace: fbt provider, modular components: This uses the fbt machinery added in the previous commit.

  • dtrace, arm: arm64 port: an arm64 implementation of DTrace.

  • dtrace: add SDT probes: add a variety of SDT probes.

Clone this wiki locally