How to Use System Profiling Tools (`perf`, `ftrace`) on Debian 12 Bookworm

System performance is critical for both production environments and development workflows. When applications behave unexpectedly or systems exhibit unexplained latency, profiling tools become essential. On Debian 12 Bookworm, two powerful tools for system profiling and tracing are perf and ftrace.

In this article, we’ll walk through how to use these tools effectively on a Debian 12 system, covering everything from installation to practical use cases.

Why Use System Profiling Tools?

System profiling tools help you analyze how resources are being used—CPU, memory, disk I/O, kernel interactions, and more. This helps in:

• Identifying performance bottlenecks
• Debugging complex runtime issues
• Tuning system behavior
• Understanding application interactions with the kernel

perf vs ftrace

Both perf and ftrace serve profiling purposes but differ in focus:

• perf is a powerful CPU profiling tool built on top of Linux’s Performance Counters subsystem. It focuses on performance statistics and events, both in user space and kernel space.
• ftrace is the official tracing framework of the Linux kernel, ideal for debugging and analyzing kernel internals, syscalls, and scheduling events.

Installing perf and ftrace on Debian 12

1. Update Your System

sudo apt update && sudo apt upgrade

2. Install perf

sudo apt install linux-perf

This installs perf matching your running kernel. If your kernel and headers are mismatched, install the matching headers:

sudo apt install linux-headers-$(uname -r)

3. ftrace Is Built into the Kernel

No separate package is needed. ftrace comes enabled with the default Debian kernel. You can interact with it through the /sys/kernel/debug/tracing directory.

To access ftrace, mount the debugfs:

sudo mount -t debugfs none /sys/kernel/debug

Ensure debugfs is mounted at boot by adding the following to /etc/fstab:

debugfs  /sys/kernel/debug  debugfs  defaults  0  0

Using perf on Debian 12

Overview

perf allows you to analyze CPU usage, sample code paths, inspect hardware events, and monitor software counters. Here’s how to start using it.

Common perf Commands

1. Record CPU Profile

sudo perf record -g ./your_program

This records performance data while the program runs.

• -g captures the call graph (stack trace).

2. View Report

sudo perf report

This opens a TUI (text-based UI) to interactively explore the recorded profile.

3. Live Monitoring

To see CPU events live:

sudo perf top

This is similar to top, but it shows functions and their performance counters.

4. Statistical Summary

sudo perf stat ./your_program

Example output:

 Performance counter stats for './your_program':

       124.730743 task-clock                #    1.003 CPUs utilized
             1,234 context-switches          #    0.010 M/sec
               45 CPU-migrations            #    0.000 M/sec
            12,345 page-faults              #    0.099 M/sec
     1,234,567,890 cycles                   #    9.899 GHz

       0.1243 seconds time elapsed

Use Case: Profiling a High CPU Load

Suppose a program is consuming excessive CPU. Record its execution:

sudo perf record -g -p $(pidof your_program)

Then analyze with:

sudo perf report

You’ll see function names and sample percentages—pinpointing where the program spends most of its CPU time.

Using ftrace on Debian 12

Overview

ftrace gives deep insights into kernel operations. It’s ideal for tracing function calls, syscalls, scheduling activity, and latency.

All configuration and usage happen in the /sys/kernel/debug/tracing/ directory.

Enable DebugFS

Ensure it’s mounted:

sudo mount -t debugfs none /sys/kernel/debug

Explore ftrace Directory

cd /sys/kernel/debug/tracing
ls

You’ll see files like:

• available_tracers
• current_tracer
• trace
• trace_pipe
• set_ftrace_filter
• tracing_on

Basic Workflow

1. Set a Tracer

Check available tracers:

cat available_tracers

Enable the function tracer:

echo function > current_tracer

2. Start Tracing

Turn on tracing:

echo 1 > tracing_on

Let it run for a few seconds, then stop:

echo 0 > tracing_on

3. View Trace Output

cat trace

Sample output:

         bash-2371  [001] ....   121.898016: cpu_idle <-do_idle
         bash-2371  [001] ....   121.898017: cpu_idle <-do_idle

4. Filter by Function

Trace only specific kernel functions:

echo do_sys_open > set_ftrace_filter

Now it’ll only log traces involving do_sys_open.

5. Trace System Calls

Enable the syscall tracer:

echo syscalls > current_tracer

This logs all syscalls made.

6. Real-Time View

sudo cat trace_pipe

This provides live updates as the system runs.

Tip: Use trace-cmd (an optional utility) for easier scripting and trace analysis.

sudo apt install trace-cmd

Practical Example: Tracing a File Open Event

Let’s say you want to trace when nano opens files:

1. Filter only do_sys_openat2 (modern file open syscall):

echo do_sys_openat2 > set_ftrace_filter

2. Start tracing:

echo 1 > tracing_on

3. Run nano in another terminal:

nano /etc/hosts

4. Stop tracing:

echo 0 > tracing_on

5. View trace:

cat trace

You’ll see when nano triggered do_sys_openat2 to open files.

Tips for Both Tools

Use Symbols for Better Insight

Install debug symbols for kernel packages to see detailed function names:

sudo apt install linux-image-$(uname -r)-dbg

Combine Tools

Use perf for profiling hotspots and ftrace to dig into the kernel internals.

Keep Output Manageable

Tracing everything creates huge output. Always filter traces or limit tracing time.

Summary Table: perf vs ftrace

Conclusion

Debian 12 Bookworm offers a solid environment for low-level performance analysis using tools like perf and ftrace. Whether you’re a developer optimizing an app, or a sysadmin diagnosing bottlenecks, these tools provide rich insights into system behavior.

• Use perf when you want to know where the CPU time is going.
• Use ftrace when you need to understand the kernel’s internals and trace fine-grained system behavior.

As always, start small, trace only what you need, and consider saving trace sessions for offline analysis. With some practice, you’ll be able to spot inefficiencies or strange behaviors quickly and precisely.