Assume presence of PT_CHERI_PCC by bsdjhb · Pull Request #2695 · CTSRD-CHERI/cheribsd

bsdjhb · 2026-07-27T20:28:56Z

csu: Replace references to the captable with the GOT instead
csu: Remove unused rodata_cap_out argument from crt_init_globals
csu: Assume PT_CHERI_PCC and bounded code pointers for pure-cap
libc/csu: Assume bounded code pointers for pure-cap
rtld: Assume PT_CHERI_PCC and bounded code pointers for pure-cap
sys: Assume PT_CHERI_PCC and bounded code pointers for pure-cap

bsdjhb · 2026-07-27T20:29:40Z

This can land after the TLS ABI change from the next branch.

bsdjhb · 2026-07-27T20:36:03Z

+	crt_init_rela(code_cap, data_cap);
+#endif
+#else
 	/* Attempt to bound the data capability to only the writable segment */


I really wish we could decide we didn't care about hybrid as much and remove all of this. I guess lld doesn't know how to set the bounds in caprelocs for code pointers that appear in hybrid binaries? Alternatively we could just use more naive bounds by coming up with an equivalent data_cap as is used for the pure-cap case by just bounding to the entire file mapping and not worrying about permissions as much. Probably that is fine to do actually since caprelocs also have proper bounds for data symbols? (Though I guess there is always the size == 0 case?) Still, it's doubtful we gain much by narrowing the bounds of data_cap/rodata_cap narrower than just the bounds of the entire mapping for hybrid.

LLD can set whatever bounds it likes, and knows hybrid from purecap, but the problem is it has no way to know what the start and end of the address space is going to be, as that depends on the execution environment.

bsdjhb · 2026-07-27T20:36:47Z

-#ifdef __CHERI_PURE_CAPABILITY__
-handle_relocs:
+#ifdef CHERI_INIT_RELA
+	crt_init_rela(code_cap, cheri_perms_clear(cheri_ddc_get(),


A wider data_cap would let me move this back out of the purecap vs hybrid #ifdef since it could just use data_cap directly for hybrid.

Is there a reason not to? It's not like hybrid has any kind of coherent principled design to it...

I've added a commit at the end of the branch that takes this approach.

bsdjhb · 2026-07-27T20:37:28Z

 	caddr_t pcc_cap;
 	Elf_Addr addr;

+	/* XXX: RISC-V kernels may not have phdrs in memory. */


I still need this for the non-EFI boot case for RISC-V kernels where the phdrs aren't available as module data. :(

At this point PT_CHERI_PCC is a solely dynamic linking thing for bounding the result of symbol resolution, so really belongs in .dynamic, which is mapped...

Well, with the MODINFO_PHDR thing, we do now have phdrs, just not when direct-booting in QEMU (which I'm still doing). Possibly that is also true on FPGA as well though?

Yeah, FPGAs also direct boot like QEMU, and as long as that's a supported boot flow I don't think it's ok to say you can't load modules there...

Modules always have phdrs though, it is only the kernel image itself that does not due to the "first byte must be an instruction" requirement.

Sure, but you're not going to get very far if your loaded module can't reference any function symbols in the kernel due to not having PCC bounds information for them

Hence the XXX comment here that permits this fallback still (which I've removed in userspace).

Yeah, my point is just we need a proper solution eventually, and it being in .dynamic would be one approach

jrtc27 · 2026-07-27T21:05:59Z

 		ptr = (uintptr_t)cheri_perms_and(code_cap,
 		    function_pointer_permissions_mask);
 		ptr = cheri_address_set(ptr, r->object);
-		if (use_code_bounds && r->size != 0)


Not for now since we'd also need to do the Clang header, but this pattern is probably one to kill off (the "throw your hands up and don't set bounds because it's too hard to do properly without a linker, and/or symbols were missing sizes" approach, as I understand).

I feel like I still see quite a few linker warnings for various kernel modules for zero-sized symbols. Perhaps I should make lld warnings fatal for the kernel and see how painful that is to resolve.

Sure, but those are cases where the size being zero doesn't matter. Note that our Morello fragment handling doesn't special-case zero, it's only actual grotty caprelocs.

jrtc27 · 2026-07-27T21:11:56Z

-	if (obj->npcc_caps == 0)
-		return (true);
+	if (obj->npcc_caps == 0) {
+		_rtld_error("missing required PT_CHERI_PCC");


That's not an error, that just means there's no code:

jrtc27@morello:~$ echo 'int x = 42;' | cc -x c - -o - -shared -nostdlib | llvm-readelf -Wl - | grep PCC jrtc27@morello:~$ echo 'int x = 42; void f(void) {}' | cc -x c - -o - -shared -nostdlib | llvm-readelf -Wl - | grep PCC CHERI_PCC 0x0003a0 0x00000000000003a0 0x00000000000003a0 0x000060 0x010060 R E 0x20

Not something you'd normally get, thanks to run_cxa_finalize in crtbeginS, but if you don't need any crt bits for some data-only object it can be valid.

Perhaps I can gate this on if there is any PT_LOAD with execute permission?

Do we need it though? Won't we just fall back on returning NULL for executable symbols if you somehow manage to build a module with an old toolchain?

For userland, yes. I think it is a bit friendlier to users to get a better error message. I've made some squash commits in the updated branch (will push after some local testing) for both the kernel and userspace to only require it if there is an executable segment. The changes are not very invasive as we are already walking phdrs to count PT_CHERI_PCC and can just check the PT_LOADs in the same loop.

If I wanted to be particularly awkward, I think I'd argue that an object that exports no symbols shouldn't need a PT_CHERI_PCC phdr, even if it has code (such as solely using constructor-y things)

Oh, yeah, that too. I know we don't in the kernel (in part because then you'd need to pass a set of code capabilities for a binary with more than one segment), but forgot we do in rtld for dynamic binaries.

Actually, that's in map_object(). I wonder why we even bother setting that in map_object()... we don't invoke the entry point for DSOs. Oh, I guess for direct exec mode we do use map_object() for the binary, and then we can end up using that. Blech. The kernel does not use narrow bounds here, perhaps map_object() should be doing the same and giving the equivalent bounds to prog_cap for the entry point?

Hm, are we not setting the bounds on the executable entry point for the normal non-direct exec case? That's a bug...

(In that csu will end up running with overly-wide bounds, only narrowing them thanks to passing a function pointer to main to __libc_start1)

Well, in the current form of this series, we are now consistent in always having wide bounds on entry to CSU. Static binaries require this so that they can process relocations. The only way we could set bounds though would be to depend on PT_CHERI_PCC, but I wanted to see if I could make PT_CHERI_PCC only required when resolving dynamic symbols and this is the one case that isn't associated with a relative relocation that is able to encode the bounds.

One might argue that for CHERI we find a way to encode the entry point as a relative relocation so that we can set the bounds there, but that gets a bit squishy. One approach might be to adopt new ELF "classes" (ELF64C, etc.) for purecap where the entry field in the ELF header is capability-sized so it can hold a relative relocation (either a fragment style or cbuildcap-style). That has a lot of cost. Another approach might be to add some sort of ancillary data to store the bounds, but then you might as well just mandate at least one PT_CHERI_PCC covering the entry point? It can also only ever apply to dynamic binaries and not static binaries anyway. It seems simplest to say that the CSU must always explicitly narrow bounds at some point (at the latest main() must use narrow bounds, but narrow bounds are encouraged earlier than that). The CSU code already needs to do this for the static case anyway.

jrtc27 · 2026-07-27T21:22:45Z

 	}

+	if (ef->npcc_caps == 0) {
+		printf("%s: Missing PT_CHERI_PCC segment\n", ef->lf.filename);


Although I'm not aware of any modules actually doing it, even firmware modules where the only code in them is a function pointed to by their struct moduledata's evhand, I think you can have a module with no code in it (and if that means you leave evhand NULL rather than leave a symbolic relocation for it then module_register will use modevent_nop for you)?

I think firmware modules just point to a function in the kernel itself, not their own module handler, but presumably I could similarly gate this on if there are any executable PT_LOADs?

sys/tools/fw_stub.awk generates a handler that calls firmware_(un)register

Bleh. We should probably just be defining a data structure passed to a centralized module handler similar to driver_module_handler in subr_bus.c, but that's an upstream problem.

bsdjhb · 2026-07-28T21:54:35Z

So another thing I could do in the kernel perhaps, is explicitly in the preload case where we can't find phdrs for a preloaded kernel or module, use a special variant that creates a single pcc_cap that is equivalent to the current fallback, and then remove the fallback from the loop. That is, a kernel where we can't find the phdrs would end up with npcc_caps == 1. That I think would avoid requiring PT_CHERI_PCC and just giving the proper runtime failure similar to userspace.

- Use GOT instead of captable. - Remove various CHERI-MIPS-specific details such as no-jump-tables and claiming that all function calls go via the GOT. Calls to non-preemptible functions do not use the GOT.

The fragment-based ABI was added in Morello LLVM in commit 94e1dbac prior to CheriBSD 22.05.

The new ABI was introduced in Morello LLVM commit b2549f5d49be prior to PT_CHERI_PCC.

jrtc27 · 2026-07-30T10:41:56Z

So another thing I could do in the kernel perhaps, is explicitly in the preload case where we can't find phdrs for a preloaded kernel or module, use a special variant that creates a single pcc_cap that is equivalent to the current fallback, and then remove the fallback from the loop. That is, a kernel where we can't find the phdrs would end up with npcc_caps == 1. That I think would avoid requiring PT_CHERI_PCC and just giving the proper runtime failure similar to userspace.

That's cute, yeah, do it once at boot and then you never need to handle the special case again

This is now replaced with PT_CHERI_PCC bounds and the bounds on relative relocations.

init_cap_from_fragment already clears FUNC_PTR_REMOVE_PERMS when it uses the pcc_cap as the base capability.

- Don't bother walking ELF program headers at all for pure-cap always using the PT_CHERI_PCC case. - Always bound code capabilities built from Morello fragments for pure-cap.

…ions PT_CHERI_PCC is now only used to set bounds on dynamically-exported symbols.

For RISC-V kernels without program headers, create a fallback PCC cap that spans the entire kernel.

The main change is that data_cap and rodata_cap now have bounds of the entire executable image. The change primarily aims to reduce code duplication with pure-capability.

bsdjhb · 2026-08-03T13:46:55Z

-#endif
 #else
-	/* Attempt to bound the data capability to only the writable segment */
+	/* Bound the data capability to the executable image */


It would be even simpler (and almost identical code for both cases), if we just used DDC as the data_cap and made no promises about bounds for hybrid. I'm tempted to just do that on the assumption that "if you are doing hybrid, you need to do all the bounds on your own".

bsdjhb · 2026-08-04T18:51:35Z

-		}
-	}
-
-	for (; phnum > 0; phdr++, phnum--) {


This check was wrong when I added it. :( In the normal case this is checking the phdrs of the binary which doesn't indicate anything about if rtld itself has "new" relative relocations. This was only correct in the direct-exec case. Oh well.

bsdjhb requested a review from jrtc27 July 27, 2026 20:29

bsdjhb commented Jul 27, 2026

View reviewed changes