Introduce support for TUF in Update Server

[doanac]

This is a fairly substantial change that introduces the ability to transparently manage TUF metadata for Updates. The branch produces a fully functioning server with TUF metadata that aklite will consume. There are some features that will be added once this has been merged:

• The ability to refresh the expiration of an old Targets file: issue Introduce mechanism to refresh a targets.json for a specific update #190
• The ability to import TUF keys from a FoundriesFactory: issue Introduce mechanism to import TUF root key from an existing FoundriesFactory #189
• Secure key storage in cloud providers: issue Introduce abstraction for storing TUF private keys in cloud secret managers #192

The current logic for handling timestamp and targets metadata may not jive well with users wanting to move devices between tags. Given the size and scope of this current change, I'd like to delay work on that until I can get this work merged.

I also haven't tested rollback support. The current approach creates a single Target in the targets.json and doesn't not include any other older Targets. I believe aklite will work correctly but this will need to be tested.

This has been reviewed by Claude Sonnet and Claude Opus.

[doanac]

Screenshot of improved Update page:

The Update creation section defaults to:

and expands to:

[vkhoroz]

This is a good feature.

LGTM

[vkhoroz]

It doesn't seem secure enough to only reference the version in the snapshot/timestamp meta.
Our Foundries.io implementation references a version, a length, and a sha256 hash.
I'm not even sure if TUF allows skipping the hash info.

[doanac]

We might reference it, but we don't do it. Here's my timestamp.json for an offline update:

"signed": {
    "_type": "Timestamp",
    "expires": "2026-06-30T18:45:39Z",
    "version": 107,
    "meta": {
      "snapshot.json": {
        "version": 107
      }
    }
  }

I think bypassing the hash is how we allow for condensed targets to work across tags.

[vkhoroz]

The spec says:

: HASHES :: A dictionary that specifies one or more hashes of the metadata file at METAPATH, with the cryptographic hash function as key and the value as HASH, the hexdigest of the cryptographic function computed on the metadata file at METAPATH. For example: { "sha256": HASH, ... }. HASHES is OPTIONAL and can be omitted to reduce the snapshot metadata file size. In that case the repository MUST guarantee that VERSION alone unambiguously identifies the metadata at METAPATH.

In our case, a version alone does not unambiguously identify the metadata, as we may have different metadata for different tags with the same version.
So, I think it is a good hygiene to include its hash into the metadata.

[vkhoroz]

This doesn't seem right to me, unless I'm missing something.

IIUC, the tufVer is derived from the individual targets versions somehow, and then targets.json version is set to tufVer + 10, in a hope that we have some room for re-signing between the current version and the next tufVer. But, the flaw I see is this: if tufVer gets incremented by e.g. 1, or 2, or 5 - it mangles with the re-signing logic.

I guess, a safer approach would be for version = tufVer * 10, so that there is always an allowance of 10 targets.json versions for every single TUF version, regardless of how the tufVer is managed.

[doanac]

I may be missing something subtle, but let me try and explain how I think the flow would be:

• Initial target added. tufVer would be 0, so the first targets.json version would be 10.
• We add another target. getLatestVersions will return 10. So target 2 will have targets.json version (10 + 10 = 20)

Later on we add support to resign a targets.json. This initial update could go from 10 -> 11.

[vkhoroz]

I'm not sure how exactly you were going to implement the whole thing.
As it is, I think we need to factor in that we have multiple tags, and each tag has its own version of targets.

It is not easy to explain, resulting into TL;DR. Sorry for that.

This is how it was handled for the production timestamp in ota-lite:

• Let's assume tags A and B have targets.json at version 10 and tag C has targets.json at version 12.
• We set timestamp version to 10000 for tags A and B, and to version 12000 for tag C.
• When we need to refresh the expires field, we increment version for tags A and B to 10001 and, for tag C to 12001.
• That allows to freely switch between tags A and B, which are on the same version; but don't allow switching from a higher version (tag C) to a lower version (tags A, B).

Now, let's try to extend the same logic to the current case, when you want to allow refreshing snapshot/targets expires field.
I think, we still need to allow switching freely between tags A and B in that case.
So, what we should do is this:

• Set targets and snapshot versions to 100 for tags A/B and 120 for tag C.
• When any targets.json for version 100 expires - we refresh all targets with the same version to 101; and leave version 120 alone. That preserves an ability to switch between tags of the same version. I think we need to be mindful here, so that we don't extend beyond version 109.
• When a new app is released with e.g. version 11 for tag A - its version jumps to 110, i.e. upgrades into a higher league.

If we were simply adding 10 (not multiplying) - all our tags quickly run out of sync when refreshing expiration; and we lose an ability to switch between tags on the same version.

IIUC you were kind of copying a simplified logic from ota-lite for CI targets. But, over there we've had a "primary" targets.json, which was then condensed into tags, so that every tag was getting the same exact version. In update-server we handle each tag's targets separately, meaning that their versions are not synchronized.

[doanac]

Let's assume tags A and B have targets.json at version 10 and tag C has targets.json at version 12.

The idea is that no two tags will ever have the same version. I believe the real issue is that s.getLatestVersions is looking at a single tag. It needs to look across all updates so that things always go forward. In other words, we can't really create an Update in this system that would be like a Target in our current product where you have a Target with tags, ["foo", "bar"].

I also think bumping the targets version should be a manual thing operators do when they want to keep an update around over 90 days. ie, its not a part of the periodic signing daemon.

I'm posting now for you to think on this. I think something still isn't quite right with how this new notion of Updates and tags is working.

[vkhoroz]

I think something still isn't quite right with how this new notion of Updates and tags is working.

Yeah... I think we need to work out a better tag/version management over time.

As for the time being, let's keep things simple:

• Auto-resigning timestamp is a must have, hence the timestamp versioning depends on that requirement. A multiple of 1000 had proven its usefulness.
• As for the targets/snapshot versioning, I'd opt for one of two things for now:
  • A. Keep it simple for now i.e. set version equal to whatever user specified (no multiples or additives). If we later allow users to manually re-sign existing (latest) targets - that's a simple version bump.
  • B. If we plan to implement auto-resigning, similar to how we manage timestamp, I'd add a multiple of e.g. 100 (not additive). But, that can be easily done at that later time in a backward-compatible way.

TBH I did not get how that addition of 10 solves any real-world problem. IMHO it only introduces some mental puzzle and potential code complication.