Strace command in Linux with Examples

What is strace

strace isn't just a logger - it's a tracing tool that can help you with debugging tasks when no other tools can. Most developers are not familiar with it, but it's often the last resort when everything else fails.

Strace can be used in many different cases:

• Observing how user-space code interacts with kernel-space
• Finding out which resources are actually used by a process
• Diagnosing mysterious hangs, permission errors, and missing files

Key idea: every program is a sequence of system calls.

A little about system calls

Every program in every programming language relies on the infrastructure provided by the OS. Part of that infrastructure is system calls - the API that the kernel exposes to userland.

Linux has around 450–500 system calls depending on CPU architecture.

For example, when we want to open a file in Python:

with open("file.txt", "r") as f:
    content = f.read()
print(content)

Under the hood, the Python runtime calls the kernel's open syscall. The kernel provides the abstraction we call a FILE - and all the real machinery happening underneath is hidden from us.

Deep diving into strace

Most useful options

strace [-a column] [-b execve] [-e expr]... [-I n] [-o file] [-O overhead] [-p pid] \
       [-S sortby] [-U columns] [-X format] [--seccomp-bpf] \
       [--syscall-limit=limit] { -p pid | [-DDD] [-E var[=val]]... [-u username] command [args] }

Strace can launch a process directly:

strace cat /etc/hosts

Or attach to an already running process:

strace -p 1

The attach mode is especially useful for stuck or misbehaving processes you can't restart.

Output format

syscall(args) = return_value

Here's a real snippet of strace output for cat /etc/hosts:

execve("/usr/bin/cat", ["cat", "/etc/hosts"], 0x7ffea4c64248 /* 22 vars */) = 0
brk(NULL)                               = 0x5634e6184000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff4010ad000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=26983, ...}) = 0
mmap(NULL, 26983, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff4010a6000
close(3)                                = 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\243\2\0\0\0\0\0"..., 832) = 832
...

The foundation of strace is the ptrace mechanism in the Linux kernel:

• The tracer (strace) attaches to the target process
• It intercepts every syscall at two points: before and after execution
• It reads the arguments and the return value

Keep in mind: strace can slow down execution significantly - sometimes by an order of magnitude. Don't use it in production without understanding the impact.

Filtering

Strace output is noisy. Without filtering it's easy to get lost in hundreds of lines per second.

Filter by category - file operations:

strace -e trace=file myapp

Network operations:

strace -e trace=network myapp

Or filter by specific syscalls:

strace -e trace=open,write,read myapp

Useful options

Show execution time for each syscall:

strace -T myapp

read(3, ..., 1024) = 512 <0.000034>

Trace subprocesses (fork/clone):

strace -f myapp

Without -f, you only see the main process. This can be very misleading when debugging multi-process applications - a child process might be the one actually failing.

Save output to a file:

strace -o trace.log myapp

Useful when the output is too large to read in the terminal, or when you need to share it.

Print a summary of syscall statistics:

strace -c myapp

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 38.00    0.000456          45        10           read
 22.00    0.000264          33         8           mmap
 ...

This is great for performance profiling - you can immediately see which syscalls are consuming the most time.

Real-world use cases

"Why is my app ignoring the config file?"

strace -e trace=openat myapp 2>&1 | grep config

Strace will show you exactly which paths the app is trying to open - and whether they return ENOENT (file not found) or EACCES (permission denied).

"Why is my app hanging?"

strace -p <pid>

Attach to the process and watch what syscall it's stuck on. Nine times out of ten it's waiting on read, epoll_wait, futex, or a network socket - and you'll see it immediately.

"My app works locally but fails in a container"

Containers often restrict syscalls via seccomp profiles. strace will reveal which syscall is getting blocked (it returns EPERM or ENOSYS), helping you pinpoint exactly what needs to be allowed.

Summary

strace is one of those tools that feels intimidating at first but becomes indispensable once you've used it to solve a real problem. It cuts through abstraction layers and shows you the raw conversation between your program and the kernel.

The learning curve is shallow - you don't need to memorize syscall tables. Just run it, look for errors (= -1), and follow the trail. Most bugs reveal themselves within the first few lines.

# When in doubt:
strace -f -e trace=file,network -o trace.log ./your-broken-app

That single command has saved me hours more times than I can count.

Ready to put strace into practice? Test your skills with our interactive Linux task where a service ignores its config file - can you trace the real source of its configuration and fix it?