Skip to content

libbpf-tools: Fix ringbuf leaks that leak prior events to userspace #5506

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
vdasu wants to merge 1 commit into
base: master
Choose a base branch
from
+17 −0
Open

libbpf-tools: Fix ringbuf leaks that leak prior events to userspace #5506

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions libbpf-tools/compat.bpf.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,4 +42,16 @@ static __always_inline long submit_buf(void *ctx, void *buf, __u64 size)
return bpf_perf_event_output(ctx, &events, BPF_F_CURRENT_CPU, buf, size);
}

/* Zero a buffer returned by reserve_buf so callers don't have to write every
* field of the event struct, and so cross-record stack/ringbuf residue is
* not emitted to userspace. __noinline lands as a single BPF-to-BPF
* subprogram shared across call sites; volatile defeats LLVM's loop-idiom
* recognition, which would otherwise re-lower this back to __builtin_memset
* (which the BPF backend cannot inline for buffers larger than ~256 bytes). */
static __noinline void zero_buf(void *p, __u64 sz)

@ekyooo ekyooo May 18, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two concerns with zero_buf():

  • opensnoop is a high-frequency probe and struct event is ~8 KB — per-event zeroing may add noticeable overhead?
  • The variable-bounded loop requires Linux 5.3+. Kernels below that would reject the program at load time?

@vdasu vdasu Jun 4, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I missed this.

  1. Yes, I zeroing out will add some overhead. Copilot's suggestion to optimize the for loop should speed things up significantly. Would you like me to run some tests to see the ovehead zero_buf adds? Also, another way to speed things up would be to zero out only the hole in the struct. If we do this, we have will have optimized ringbuf zero functions for each eBPF program that leaks data.
  2. This should not be a problem for libbpf-tools since it requires CO-RE which requires kernel version 5.4+. Also, at this point in time, it looks like most libbpf-tools programs require version 5.5+ (This issue (Allow libbpf-tools to run on old kernels #4231) seems to address the backwards compatibility problem, but at this point in time, most programs are still not backward compatible).

{
for (__u64 i = 0; i < sz; i++)
*(volatile char *)((char *)p + i) = 0;
Comment on lines +49 to +54

@vdasu vdasu May 4, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ekyooo I can commit this optimization if you think it is necessary.

}

#endif /* __COMPAT_BPF_H */
1 change: 1 addition & 0 deletions libbpf-tools/filelife.bpf.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,7 @@ int BPF_KRETPROBE(vfs_unlink_ret)
eventp = reserve_buf(sizeof(*eventp));
if (!eventp)
return 0;
zero_buf(eventp, sizeof(*eventp));

eventp->tgid = unlink_event->tgid;
eventp->delta_ns = unlink_event->delta_ns;
Expand Down
1 change: 1 addition & 0 deletions libbpf-tools/mountsnoop.bpf.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ static int probe_exit(void *ctx, int ret)
eventp = reserve_buf(sizeof(*eventp));
if (!eventp)
goto cleanup;
zero_buf(eventp, sizeof(*eventp));

@ekyooo ekyooo May 18, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For mountsnoop, userspace already accesses only the active union arm via switch(e->op), so stale bytes in inactive arms are never read. Is "previously emitted record leak" the right framing for this case?

Would it be simpler to check the return value of bpf_probe_read_user_str and skip submit_buf on failure, rather than zeroing the whole buffer? Any case where a partial event is still useful?

@vdasu vdasu May 28, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale bytes are never read in the current userspace program. However, the ringbuf does contain stale bytes from prior events that are accessible from userspace. Any other consumer of the same ringbuf (e.g., a custom loader) sees the full record submitted to userspace, not just the bytes in the active arm.

I confirmed the leak empirically by reading the ringbuf directly (i.e., not through mountsnoop.c) on Linux 6.8. When you first reserve ringbuf memory (bpf_ringbuf_reserve(size)), the kernel returns a zeroed-out chunk of memory. However, since the ringbuf is 256 KB and each event is 8768 bytes, the buffer holds about ~29 events before the write position wraps. From event 30 onwards, each write lands on cycled slots, and the inactive-arm bytes contain fully decodable previous records. Across 200 events under a mount -t tmpfs / umount loop, 11,396 bytes of non-zero residue leaked, including PIDs, command names, and mount paths like /run/user/<UID>/credentials/mnt-test.mount, which reveal which user triggered earlier mount operations.

Regarding checking the return value of bpf_probe_read_user_str, that is good practice, but it's orthogonal to this leak. When the active arm is smaller than mount (e.g., FSMOUNT writes 12 bytes, UMOUNT writes 4104 bytes), the trailing bytes of the union (8720 - active arm size) are never targeted by any helper. Even if every bpf_probe_read_user_str succeeds and we never skip submit_buf, those bytes remain unwritten.

Partial events might be useful, since they are better than silently dropping the event. However, I believe that is a separate question since it would not fix the leak.

@ekyooo ekyooo Jun 5, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I confirmed the leak empirically by reading the ringbuf directly (i.e., not through mountsnoop.c) on Linux 6.8. When you first reserve ringbuf memory (bpf_ringbuf_reserve(size)), the kernel returns a zeroed-out chunk of memory. However, since the ringbuf is 256 KB and each event is 8768 bytes, the buffer holds about ~29 events before the write position wraps. From event 30 onwards, each write lands on cycled slots, and the inactive-arm bytes contain fully decodable previous records. Across 200 events under a mount -t tmpfs / umount loop, 11,396 bytes of non-zero residue leaked, including PIDs, command names, and mount paths like /run/user//credentials/mnt-test.mount, which reveal which user triggered earlier mount operations.
Regarding checking the return value of bpf_probe_read_user_str, that is good practice, but it's orthogonal to this leak. When the active arm is smaller than mount (e.g., FSMOUNT writes 12 bytes, UMOUNT writes 4104 bytes), the trailing bytes of the union (8720 - active arm size) are never targeted by any helper.

Your observation is valid, and this seems to be an expected result of mountsnoop’s design.
Can you clarify the exact issue you are trying to solve?

  1. Previously emitted records were already delivered to userspace. In that sense, is this really a leak in the security sense?
  2. When using mountsnoop.c, inactive union fields are not read, so there is no practical issue. Do you have another real consumer besides mountsnoop that is affected?

@vdasu vdasu Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you are right. It is not a leak in the security sense, i.e., not a KALSR-breaking leak. The information that is left over and submitted to the ringbuf is not privileged, and it was sent to userspace. It concerns a (potential) confidentiality issue (e.g., residue from a record about one mount/user can surface in a later record about a different mount/user) and general output-sanitization hygiene. Additionally, it might be problematic if we have custom userspace loaders and multiple userspace loaders accessing the same pinned ringbuf. Regarding other programs, I also observed something similar in opensnoop where the file paths, which are of variable length, contain stale bytes from previous filepaths that were longer.


task = (struct task_struct *)bpf_get_current_task();
eventp->delta = bpf_ktime_get_ns() - argp->ts;
Expand Down
1 change: 1 addition & 0 deletions libbpf-tools/oomkill.bpf.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ int BPF_KPROBE(oom_kill_process, struct oom_control *oc, const char *message)
data = reserve_buf(sizeof(*data));
if (!data)
return 0;
zero_buf(data, sizeof(*data));

data->fpid = bpf_get_current_pid_tgid() >> 32;
data->tpid = BPF_CORE_READ(oc, chosen, tgid);
Expand Down
1 change: 1 addition & 0 deletions libbpf-tools/opensnoop.bpf.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,7 @@ int trace_exit(struct syscall_trace_exit* ctx)
eventp = reserve_buf(sizeof(*eventp));
if (!eventp)
goto cleanup;
zero_buf(eventp, sizeof(*eventp));

/* event data */
eventp->pid = bpf_get_current_pid_tgid() >> 32;
Expand Down
1 change: 1 addition & 0 deletions libbpf-tools/tcppktlat.bpf.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ static int handle_tcp_rcv_space_adjust(void *ctx, struct sock *sk)
eventp = reserve_buf(sizeof(*eventp));
if (!eventp)
goto cleanup;
zero_buf(eventp, sizeof(*eventp));

eventp->pid = pid;
eventp->tid = tid;
Expand Down