Skip to content

stdenv: PURL fetcher introduction #454333

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
raboof merged 1 commit into from
May 26, 2026
Merged

stdenv: PURL fetcher introduction #454333

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions doc/redirects.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -412,6 +412,9 @@
"sec-meta-identifiers-cpe": [
"index.html#sec-meta-identifiers-cpe"
],
"sec-meta-identifiers-purl": [
"index.html#sec-meta-identifiers-purl"
],
"sec-modify-via-packageOverrides": [
"index.html#sec-modify-via-packageOverrides"
],
Expand Down Expand Up @@ -932,6 +935,15 @@
"var-meta-identifiers-possibleCPEs": [
"index.html#var-meta-identifiers-possibleCPEs"
],
"var-meta-identifiers-purl": [
"index.html#var-meta-identifiers-purl"
],
"var-meta-identifiers-purlParts": [
"index.html#var-meta-identifiers-purlParts"
],
"var-meta-identifiers-purls": [
"index.html#var-meta-identifiers-purls"
],
"var-meta-teams": [
"index.html#var-meta-teams"
],
Expand Down
6 changes: 5 additions & 1 deletion doc/release-notes/rl-2611.section.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,11 @@

<!-- To avoid merge conflicts, consider adding your item at an arbitrary place in the list instead. -->

- Create the first release note entry in this section!
- Package-URL (PURL, https://github.com/package-url/purl-spec) metadata identifier has been added for `fetchgit`, `fetchpypi` and `fetchFromGithub` fetchers.
`mkDerivation` has been adjusted to reuse this information.
Package-URLs allow reliably identifying and locating software packages.
Maintainers of derivations using the adapted fetchers should rely on the `drv.src.meta.identifiers.v1.purl` default identifier and can enhance their `drv.meta.identifiers.v1.purls` list once they would like to have additional identifiers.
Maintainers using `fetchurl` for `drv.src` are urged to adapt their `drv.meta.identifiers.purlParts` for proper identification.

## Nixpkgs Library {#sec-nixpkgs-release-26.11-lib}

Expand Down
27 changes: 27 additions & 0 deletions doc/stdenv/meta.chapter.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -334,3 +334,30 @@ A readonly attribute that concatenates all CPE parts in one string.
#### `meta.identifiers.possibleCPEs` {#var-meta-identifiers-possibleCPEs}

A readonly attribute containing the list of guesses for what CPE for this package can look like. It includes all variants of version handling mentioned above. Each item is an attrset with attributes `cpeParts` and `cpe` for each guess.

### Package URL {#sec-meta-identifiers-purl}

[Package-URL](https://github.com/package-url/purl-spec) (PURL) is a specification to reliably identify and locate software packages.
Through identification of software packages, additional (non-major) use cases are e.g. software license cross-verification via third party databases or initial vulnerability response management.
Package-URLs shall default to the `mkDerivation.src`, as the original consumed software package is the single source of truth.

#### `meta.identifiers.purlParts` {#var-meta-identifiers-purlParts}

This attribute contains an attribute set of all parts of the PURL for this package.

* `type` mandatory [type](https://github.com/package-url/purl-spec/blob/18fd3e395dda53c00bc8b11fe481666dc7b3807a/docs/standard/summary.md) which needs to be provided
* `spec` specify the PURL in accordance with the [purl-spec](https://github.com/package-url/purl-spec/blob/18fd3e395dda53c00bc8b11fe481666dc7b3807a/purl-specification.md)
Comment thread
h0nIg marked this conversation as resolved.

#### `meta.identifiers.purl` {#var-meta-identifiers-purl}

An extendable attribute which is built based on `purlParts`.
This is the main identifier of the software package.
For handling edge cases, consider using the list interface [`meta.identifiers.purls`](#var-meta-identifiers-purls).

#### `meta.identifiers.purls` {#var-meta-identifiers-purls}

An extendable list attribute which defaults to a single element equal to [`meta.identifiers.purl`](#var-meta-identifiers-purl).
It provides an interface for additional identifiers of `mkDerivation.src` or for identifiers of vendored dependencies inside `mkDerivation.src`, which maintainers may carefully consider to specify as well.

Additional identifiers are generally not recommended, as they might cause maintenance overhead or diverge.
For example, a source distribution `pkg:github` may be hard to keep correctly aligned with the corresponding binary distribution `pkg:pypi`.
13 changes: 12 additions & 1 deletion pkgs/build-support/fetchgit/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,18 @@ lib.makeOverridable (
${if allowedRequisites != null then "allowedRequisites" else null} = allowedRequisites;
};

inherit preferLocalBuild meta;
inherit preferLocalBuild;

meta = meta // {
identifiers = {
purlParts = {
type = "generic";
# https://github.com/package-url/purl-spec/blob/18fd3e395dda53c00bc8b11fe481666dc7b3807a/types-doc/generic-definition.md
spec = "${name}?vcs_url=${url}@${(lib.revOrTag rev tag)}";
};
}
// meta.identifiers or { };
};

env = {
NIX_PREFETCH_GIT_CHECKOUT_HOOK = finalAttrs.postCheckout;
Expand Down
16 changes: 16 additions & 0 deletions pkgs/build-support/fetchgithub/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,22 @@ decorate (
meta
// {
homepage = meta.homepage or baseUrl;
identifiers = {
purlParts =
if githubBase == "github.com" then
{
type = "github";
# https://github.com/package-url/purl-spec/blob/18fd3e395dda53c00bc8b11fe481666dc7b3807a/types-doc/github-definition.md
spec = "${owner}/${repo}@${(lib.revOrTag rev tag)}";
}
else
{
type = "generic";
# https://github.com/package-url/purl-spec/blob/18fd3e395dda53c00bc8b11fe481666dc7b3807a/types-doc/generic-definition.md
spec = "${repo}?vcs_url=https://${githubBase}/${owner}/${repo}@${(lib.revOrTag rev tag)}";
};
Comment on lines +102 to +105
Copy link
Copy Markdown
Contributor

@samueldr-at-cyberus samueldr-at-cyberus May 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be type = "github" with repository_url?

Copy link
Copy Markdown
Member

@raboof raboof May 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that would be a good improvement.

}
// meta.identifiers or { };
}
// lib.optionalAttrs (position != null) {
# to indicate where derivation originates, similar to make-derivation.nix's mkDerivation
Expand Down
16 changes: 15 additions & 1 deletion pkgs/build-support/fetchpypi/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ lib.makeOverridable (
format ? "setuptools",
sha256 ? "",
hash ? "",
pname,
version,
...
}@attrs:
let
Expand All @@ -60,8 +62,20 @@ lib.makeOverridable (
"hash"
]
);
meta = {
identifiers.purlParts = {
type = "pypi";
# https://github.com/package-url/purl-spec/blob/18fd3e395dda53c00bc8b11fe481666dc7b3807a/types-doc/pypi-definition.md
spec = "${pname}@${version}";
};
};
in
fetchurl {
inherit url sha256 hash;
inherit
url
sha256
hash
meta
;
}
)
4 changes: 4 additions & 0 deletions pkgs/by-name/jq/jq/package.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,5 +139,9 @@ stdenv.mkDerivation (finalAttrs: {
];
platforms = lib.platforms.unix;
mainProgram = "jq";
identifiers.purlParts = {
type = "github";
spec = "jqlang/jq@jq-${finalAttrs.version}";
};
Comment on lines +142 to +145
Copy link
Copy Markdown
Contributor

@samueldr-at-cyberus samueldr-at-cyberus May 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this accurate?

nix-repl> :p jq.src.meta.identifiers.purls  
[ ]

nix-repl> :p jq.meta.identifiers.purls     
[ "pkg:github/jqlang/jq@jq-1.8.1" ]

Since the PURL spec seems to be missing an important detail: whether or not the identity is about the “logical” package, or the provenance of the source file, there's some interpretation that needs to be done...

My current head-canon, especially since there is pkg:deb/* and other misc. provenances, is that PURLs are supposed to represent the provenance of an artifact, rather than what "logical" package something is. And that would seem to align with your implementation on fetchers.

(See the comment for popt about src.meta vs. meta)

So, in this case I'm more interested in the pkg:github PURL usage.

This is, as far as I understand, not a GitHub type PURL.

This is a generic fetch of a tarball that was created via CI and coincidentally uploaded to GitHub.

See these steps:

This is creating an archive that differs from the commit's contents.

Copy link
Copy Markdown
Member

@raboof raboof May 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

while https://github.com/package-url/purl-spec/blob/main/types-doc/github-definition.md is sadly low on details, it is my understanding that you should understand GitHub purls not as literally "this is exactly the source found at this Git repository", but more abstractly "this is the software published by the owners of this repository" - so using this Purl to refer to a tarball that was uploaded to this repo seems OK to me here.

};
})
Copy link
Copy Markdown
Contributor

@samueldr-at-cyberus samueldr-at-cyberus May 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional thread for jq.meta.purl. Let's ignore how this might not build, and focus on the implications for the PURL.

The proper tracking of provenance is, imo, another reason why the PURL should be on the src in this case:

nix-repl> :p (jq.overrideAttrs(_: { src = fetchurl { url = "https://github.com/jqlang/jq/releases/download/jq-1.8.0/jq-1.8.0.tar.gz"; hash = "sha256-kYEVd/kdmmGV/1DCv/7JtyyEKdwF7D6gIv2VwG0rMZw="; }; } )).meta.identifiers.purls     
[ "pkg:github/jqlang/jq@jq-1.8.1" ]

Sure, this is bad form to override the source without overriding the version accordingly, but here it's important not to confidently produce wrong output. The overriding mechanisms will be used in surprising manners, and in the case of ${...}.meta.purl, it almost definitely will produce the wrong result at some point.

Compare this to if it had been set on jq.meta.src, the simple fact of overriding the src attribute would have made it harder to come to this result. Right?

Copy link
Copy Markdown
Member

@raboof raboof May 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's some case to be made that the purl to be encoded here should be the purl without the version component (version is optional in the spec and there's some precedence for this in CVE (https://github.com/CVEProject/cve-schema/blob/main/schema/docs/CVE_Record_Format_bundled.json#L449), as long as the version can be reliably determined by the version field. Of course, that also fails if you override just the source...

4 changes: 4 additions & 0 deletions pkgs/by-name/po/popt/package.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,5 +60,9 @@ stdenv.mkDerivation (finalAttrs: {
maintainers = with lib.maintainers; [ qyliss ];
license = lib.licenses.mit;
platforms = lib.platforms.unix;
identifiers.purlParts = {
type = "github";
spec = "rpm-software-management/popt@popt-${finalAttrs.version}-release";
};
Copy link
Copy Markdown
Contributor

@samueldr-at-cyberus samueldr-at-cyberus May 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Compare:

nix-repl> :p popt.src.meta.identifiers.purls
[ ]

nix-repl> :p popt.meta.identifiers.purls     
[ "pkg:github/rpm-software-management/popt@popt-1.19-release" ]

and

nix-repl> :p hub.src.meta.identifiers.purls
[ "pkg:github/github/hub@38bcd4ae469e5f53f01901340b715c7658ab417a" ]

nix-repl> :p hub.meta.identifiers.purls       
[ ]

Why is the purl on the package here rather than on the src?

Since the PURL spec seems to be missing an important detail: whether or not the identity is about the “logical” package, or the provenance of the source file, there's some interpretation that needs to be done...

My current head-canon, especially since there is pkg:deb/* and other misc. provenances, is that PURLs are supposed to represent the provenance of an artifact, rather than what "logical" package something is. And that would seem to align with your implementation on fetchers.

Though in this particular instance, the src ends-up with no PURL, and the package itself is getting a PURL.

And, I'm not even sure it would be a "valid" PURL. A PURL for a derivation might really should probably be a pkg:nix/drv@... or something or other representing the provenance in a faithful-enough manner. No?

Copy link
Copy Markdown
Member

@raboof raboof May 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the purl on the package here rather than on the src?

The purl is on the popt package because we manually encoded this knowledge. The purl is not on the hub package because we didn't specify it explicitly. In a lot of cases, unless otherwise specified, likely the purl of the package can be derived from the purl of the source, and the purl of the source can be derived from the fetcher. However, we didn't want to bake that assumption into nixpkgs yet: for various reasons it seemed better / more conservative / more precise to leave that up to whatever tool will be consuming these identifiers.

My current head-canon, especially since there is pkg:deb/* and other misc. provenances, is that PURLs are supposed to represent the provenance of an artifact, rather than what "logical" package something is

I'd say it's actually more about 'logical' 'identity' than 'provenance', though indeed they get mixed up somewhat.

A PURL for a derivation might really should probably be a pkg:nix/drv@... or something or other representing the provenance in a faithful-enough manner. No?

My thinking is we might indeed want pkg:nix/drv Purls at some point (package-url/purl-spec#314), but the 'meta' should contain the identification of the package as it's 'known' - similar to how the 'website' field is not the website of the derivation but the website of the software represented by the derivation.

};
})
7 changes: 7 additions & 0 deletions pkgs/development/ruby-modules/gem/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,13 @@ lib.makeOverridable (
attrs.source.remotes or [ "https://rubygems.org" ]
);
inherit (attrs.source) sha256;
meta = {
identifiers.purlParts = {
type = "gem";
# https://github.com/package-url/purl-spec/blob/18fd3e395dda53c00bc8b11fe481666dc7b3807a/types-doc/gem-definition.md
spec = "${gemName}@${version}?platform=${platform}";
};
};
}
else if type == "git" then
fetchgit {
Expand Down
27 changes: 25 additions & 2 deletions pkgs/stdenv/generic/check-meta.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -622,14 +622,37 @@ let
cpe = makeCPE guessedParts;
}
) possibleCPEPartsFuns;

purlParts = attrs.meta.identifiers.purlParts or { };
purlPartsFormatted =
if purlParts ? type && purlParts ? spec then "pkg:${purlParts.type}/${purlParts.spec}" else null;

# search for a PURL in the following order:
purl =
# 1) locally set through API
if purlPartsFormatted != null then purlPartsFormatted else null;

# search for a PURL in the following order:
purls =
# 1) locally overwritten through meta.identifiers.purls (e.g. extension of list)
attrs.meta.identifiers.purls or (
# 2) locally set through API
if purlPartsFormatted != null then [ purlPartsFormatted ] else [ ]
);

v1 = {
inherit cpeParts possibleCPEs;
inherit
cpeParts
possibleCPEs
purls
;
${if cpe != null then "cpe" else null} = cpe;
${if purl != null then "purl" else null} = purl;
};
in
v1
// {
inherit v1;
inherit v1 purlParts;
};

# Expose the result of the checks for everyone to see.
Expand Down
Loading