Sync jemalloc with upstream FreeBSD - #2660
Open
brooksdavis wants to merge 38 commits into
Open
Conversation
Member
|
Updating the system malloc implementation is probably something not to land just before 26.05? |
Member
Author
Yeah, definitely not. I've mostly posted it now for CI purposes before I start turning it into tidier commits against upstream FreeBSD. |
Member
Good, just checking 😂 I suspect there's a lot in the diff that can be cleaned up, our jemalloc is rather old code from before we knew what we were doing (and probably has remnants of horrors like offset semantics still). |
brooksdavis
force-pushed
the
jemalloc-update
branch
from
June 4, 2026 19:20
35410a2 to
b1f0be7
Compare
Collaborator
I recently purged some of said horrors in fact. :-/ |
Add back more of the release for jemalloc for easier imports that can be audited for stray changes simply. However, trim out tests and msvc project files since they are easy and account for over half the size of the distro. No functional change. Sponsored by: Netflix (cherry picked from commit bf6039f)
Add the xml source to the FreeBSD changes to the jemalloc.3. No functional change. Sponsored by: Netflix (cherry picked from commit a0dfba6)
jemalloc_preample.h is generated, and FreeBSD has made some changes which we update using configure, etc on version updates. No functional change. Sponsored by: Netflix (cherry picked from commit 718b13b)
Gate the namespace dance on this define. This allows us to bootstrap new versions of jemalloc more easily. One of the steps of import is building to find what symbols need to be private. When we do that, we don't want to do this dance. No functional change. Sponsored by: Netflix (cherry picked from commit 6371645)
…pace When we're not doing the private namespace remap, we don't need to do this pragma. This is needed for the bootstrapping path when we import a new version of jemalloc. No functional change. Sponsored by: Netflix (cherry picked from commit da260ab)
This brings in a merge from jemalloc 5.3.0. It's almost fully scripted, except: Three conflicts resolved by hand: include/jemalloc/internal/test_hooks.h Use the new name src/extent.c Use the new code src/jemalloc.c Use the new code since je_realloc has moved The script is recorded in FREEBSD-upgrade. The old script did svn commands that were basically a rebase of our changes. This update has a series of diff reduction changes before this. Note: I'd planned on fixing the above three conflicts with commits, but ran out of time when I did this work in January. I got discouraged when jemalloc was EOL'd and didn't pick this back up. I did the above by hand to get this into FreeBSD 15.0 This work is a repeat of the work by Minsoo Choo who did all these changes and created a pull request. Given the importance of jemalloc, I audited these changes by redoing them in this series of commits (and with the script that was checked in). I did this to confince myself and anybody else in doubt that there was no supply chain attack. The diffs between this series of commits and Minsoo's work are minor (though the version skew makes adds some noise). Interested parties can independent audit each step, I hope. I've listed Minsoo as a co-author since without his pull request to test again, this wouldn't have been possible. Thanks to brooks@ for help with getting the jemalloc 3 ABI compat symbols right. Co-authored-by: Minsoo Choo <minsoochoo0122@proton.me> Pull Request: freebsd/freebsd-src#1337 Sponsored by: Netflix (cherry picked from commit c43cad8)
The merge markers were added, it seems, in one of my rebases. The HEAD entry is the merged code, as far as I can tell, and is what should be in the base. None of these files are used during the build. Noticed by: jhb Sponsored by: Netflix (cherry picked from commit 8ebb3de)
gcc doesn't like something about the initializer that comes with jemalloc. Since it's vendor code, make this warning not an error for -Werror purposes. Sponsored by: Netflix (cherry picked from commit edf9a2f)
Reviewed by: imp Pull Request: freebsd/freebsd-src#1890 (cherry picked from commit bf4c748)
Reviewed by: imp Pull Request: freebsd/freebsd-src#1890 (cherry picked from commit 80ec82a)
Summary: This avoids a clash with the new macro in <stddef.h> introduced in D53967 Reviewed by: imp Approved by: markj (mentor) MFC after: 1 month Differential Revision: https://reviews.freebsd.org/D53968 (cherry picked from commit 04d000a)
realloc() and rallocx() shares path, and realloc() should set errno to ENOMEM upon OOM failures. PR: 291677 Obtained from: jemalloc (commit 38056fea64c34ca4fef0a16212776eaa4de80b78) Fixes: c43cad8 ("jemalloc: Merge from jemalloc 5.3.0 vendor branch") MFC after: 3 days Pull Request: freebsd/freebsd-src#2059 (cherry picked from commit 5583b64)
PR: 291677 Obtained from: jemalloc (commit 83b075789b4239035931c1ee212576d00153bbf0) Fixes: c43cad8 ("jemalloc: Merge from jemalloc 5.3.0 vendor branch") MFC after: 3 days Pull Request: freebsd/freebsd-src#2059 (cherry picked from commit 2c5cd07)
sys/cheric.h has not included assert.h for quite some time so remove this diff. This reverts the remainder of commit 40ff7ae
It's not in active use and it hard to preserve through the 5.3.0 update.
Fixes: 98a4c94 ("Update jemalloc to version 5.2.1")
This has no effect as JEMALLOC_GCC_ATOMICS has been defined and is prefered by the ifdef ladder that decides which implementation to use.
Seems to work today
… 2^13." This diff had no effect because jemalloc_FreeBSD.h would redefine LG_PAGE as PAGE_SHIFT. This reverts commit 3615c3e.
This feature which allocates an extra page for large allocations and then places the starte of the returned allocation at the beginning in incompatible with CHERI's compressed bounds and the (current) requirement that malloc return pointers with an offset of 0. Rather than disabling it in the local jemalloc_internal_defs.h, do it centrally. Arguably configure should disable this option and fail attempts to enable it on CHERI targets, but we're not there yet.
JEMALLOC_DSS is undefined by jemalloc_FreeBSD.h so we don't need to avoid defining it here.
It's more clear what we mean and arguably more correct even if the target doesn't define ptraddr_t.
There are no other remaining changes to this file.
Changes removed with opt_zero_free
Today we always set allocation bounds so the allocation lookup should be correct.
This doesn't happen in practice because we round up allocation requests. If it did, it's ok now that reservations are padded.
We can do pointer math propertly now so we don't need to use them. There might be some optimization advantages to using them, but that should be left to upstream.
Unclear if it's still relevant and this reduces the diff to upstream.
The compiler correctly concludes there's not provenance confusion so we don't need these any more.
We required the address to be NULL on CHERI to supress attempts to extend the previous extent. That fact that a tagged, out of bounds pointer would be passed to mmap should have the same effect. Note: this function should be updated to use MAP_FIXED|MAP_EXCL on FreeBSD.
brooksdavis
force-pushed
the
jemalloc-update
branch
from
July 17, 2026 15:42
b1f0be7 to
45be085
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry pick all the jemalloc changes from upstream updating CHERI support.
Drop the zero-on-free support we added in 2020 as we're not using it.
Unfortunately, I've had to disable RTREE_LEAF_COMPACT because upstream started using some of the low bits we were stealing. Once I've extracted this set of diffs to the upstreaming branch it may make sense to engage with Meta about this.
Some tidying is likely required and some CHERI CHANGES blocks need to be restored.