Skip to content

Add support for OCI image manifests and indexes #584

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
aman-coder03 wants to merge 2 commits into
base: master
Choose a base branch
from
+151 −13
Open

Add support for OCI image manifests and indexes #584

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
bcfccfe
support for OCI image manifests and indexes
aman-coder03 Apr 6, 2026
a4a1082
addressing review comments
aman-coder03 Apr 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ require (
github.com/jmoiron/sqlx v0.0.0-20190319043955-cdf62fdf55f6
github.com/mattn/go-sqlite3 v1.14.0
github.com/opencontainers/go-digest v1.0.0
github.com/opencontainers/image-spec v1.0.2
github.com/pressly/goose v2.6.0+incompatible
github.com/satori/go.uuid v1.2.0
github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72
Expand Down Expand Up @@ -88,7 +89,6 @@ require (
github.com/kardianos/osext v0.0.0-20190222173326-2bc1f35cddc0 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.1 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/opencontainers/image-spec v1.0.2 // indirect
github.com/opencontainers/runc v1.0.2 // indirect
github.com/opencontainers/runtime-spec v1.0.3-0.20210326190908-1c3f411f0417 // indirect
github.com/pkg/errors v0.9.1 // indirect
Expand Down
6 changes: 5 additions & 1 deletion proxy/proxyserver/preheat.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,11 @@ import (
"github.com/uber/kraken/utils/log"
)

var _manifestRegexp = regexp.MustCompile(`^application/vnd.docker.distribution.manifest.v\d\+(json|prettyjws)`)
var _manifestRegexp = regexp.MustCompile(
`^application/vnd\.docker\.distribution\.manifest\.v\d\+(json|prettyjws)` +
`|^application/vnd\.oci\.image\.manifest\.v1\+json` +
`|^application/vnd\.oci\.image\.index\.v1\+json`,
)
Comment on lines +35 to +39
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding OCI index media types to the preheat event filter means Kraken will start processing push events whose manifest.References() typically include only child manifests (not layer/config blobs). With the current process() implementation, this can result in preheating only nested manifests rather than the actual layers unless additional manifest push events are also received. Consider either keeping index/list types out of this filter, or recursively fetching referenced manifests and preheating their references to ensure layers are warmed.

Copilot uses AI. Check for mistakes.

// PreheatHandler defines the handler of preheat.
type PreheatHandler struct {
Expand Down
76 changes: 65 additions & 11 deletions utils/dockerutil/dockerutil.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,9 @@

"github.com/docker/distribution"
"github.com/docker/distribution/manifest/manifestlist"
"github.com/docker/distribution/manifest/ocischema"
"github.com/docker/distribution/manifest/schema2"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/uber/kraken/core"
)

Expand All @@ -35,13 +37,26 @@
return nil, core.Digest{}, fmt.Errorf("read: %s", err)
}

manifest, d, err := ParseManifestV2(b)
if err == nil {
return manifest, d, err
type attempt struct {
name string
fn func([]byte) (distribution.Manifest, core.Digest, error)
}
attempts := []attempt{
{"docker v2 manifest", ParseManifestV2},
{"docker v2 manifest list", ParseManifestV2List},
{"OCI image manifest", ParseManifestOCI},
{"OCI image index", ParseManifestOCIIndex},
}

// Retry with v2 manifest list.
return ParseManifestV2List(b)
var errs []string
for _, a := range attempts {
m, d, err := a.fn(b)
if err == nil {
return m, d, nil
}
errs = append(errs, fmt.Sprintf("%s: %s", a.name, err))
}
return nil, core.Digest{}, fmt.Errorf("unrecognized manifest format: [%s]", strings.Join(errs, "; "))

Check failure on line 59 in utils/dockerutil/dockerutil.go

View workflow job for this annotation

GitHub Actions / lint 

undefined: strings

Check failure on line 59 in utils/dockerutil/dockerutil.go

View workflow job for this annotation

GitHub Actions / build 

undefined: strings

Check failure on line 59 in utils/dockerutil/dockerutil.go

View workflow job for this annotation

GitHub Actions / unit_tests 

undefined: strings
}

// ParseManifestV2 returns a parsed v2 manifest and its digest.
Expand Down Expand Up @@ -71,13 +86,13 @@
if err != nil {
return nil, core.Digest{}, fmt.Errorf("unmarshal manifestlist: %s", err)
}
deserializedManifestList, ok := manifestList.(*manifestlist.DeserializedManifestList)
deserializedManifestIndex, ok := manifestIndex.(*manifestlist.DeserializedManifestList)

Check failure on line 89 in utils/dockerutil/dockerutil.go

View workflow job for this annotation

GitHub Actions / lint 

undefined: manifestIndex

Check failure on line 89 in utils/dockerutil/dockerutil.go

View workflow job for this annotation

GitHub Actions / build 

undefined: manifestIndex

Check failure on line 89 in utils/dockerutil/dockerutil.go

View workflow job for this annotation

GitHub Actions / unit_tests 

undefined: manifestIndex
if !ok {
return nil, core.Digest{}, errors.New("expected manifestlist.DeserializedManifestList")
return nil, core.Digest{}, fmt.Errorf("expected OCI image index, got %T", manifestIndex)

Check failure on line 91 in utils/dockerutil/dockerutil.go

View workflow job for this annotation

GitHub Actions / lint 

undefined: manifestIndex (typecheck)

Check failure on line 91 in utils/dockerutil/dockerutil.go

View workflow job for this annotation

GitHub Actions / build 

undefined: manifestIndex

Check failure on line 91 in utils/dockerutil/dockerutil.go

View workflow job for this annotation

GitHub Actions / unit_tests 

undefined: manifestIndex
}
version := deserializedManifestList.SchemaVersion
version := deserializedManifestIndex.SchemaVersion
if version != 2 {
return nil, core.Digest{}, fmt.Errorf("unsupported manifest list version: %d", version)
return nil, core.Digest{}, fmt.Errorf("unsupported OCI image index version: %d", version)
}
d, err := core.ParseSHA256Digest(string(desc.Digest))
if err != nil {
Expand All @@ -86,6 +101,40 @@
return manifestList, d, nil
}

// ParseManifestOCI returns a parsed OCI image manifest and its digest.
func ParseManifestOCI(bytes []byte) (distribution.Manifest, core.Digest, error) {
manifest, desc, err := distribution.UnmarshalManifest(specs.MediaTypeImageManifest, bytes)
if err != nil {
return nil, core.Digest{}, fmt.Errorf("unmarshal OCI manifest: %s", err)
}
_, ok := manifest.(*ocischema.DeserializedManifest)
if !ok {
return nil, core.Digest{}, errors.New("expected ocischema.DeserializedManifest")
}
d, err := core.ParseSHA256Digest(string(desc.Digest))
if err != nil {
return nil, core.Digest{}, fmt.Errorf("parse digest: %s", err)
}
return manifest, d, nil
}

// ParseManifestOCIIndex returns a parsed OCI image index and its digest.
func ParseManifestOCIIndex(bytes []byte) (distribution.Manifest, core.Digest, error) {
manifestIndex, desc, err := distribution.UnmarshalManifest(specs.MediaTypeImageIndex, bytes)
if err != nil {
return nil, core.Digest{}, fmt.Errorf("unmarshal OCI image index: %s", err)
}
_, ok := manifestIndex.(*manifestlist.DeserializedManifestList)
if !ok {
return nil, core.Digest{}, errors.New("expected manifestlist.DeserializedManifestList for OCI index")
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error message here exposes the internal Docker distribution type name rather than describing the OCI concept being validated. Consider rewording to something user-facing like "expected OCI image index" (and optionally including the actual type via %T) to make troubleshooting easier.

Suggested change
return nil, core.Digest{}, errors.New("expected manifestlist.DeserializedManifestList for OCI index")
return nil, core.Digest{}, fmt.Errorf("expected OCI image index, got %T", manifestIndex)

Copilot uses AI. Check for mistakes.
}
Comment on lines +127 to +130
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ParseManifestOCIIndex asserts the returned type but does not validate SchemaVersion (unlike ParseManifestV2List, which checks SchemaVersion==2 on the same DeserializedManifestList type). For consistency and to avoid accepting unsupported index versions, store the cast result and validate SchemaVersion before returning.

Suggested change
_, ok := manifestIndex.(*manifestlist.DeserializedManifestList)
if !ok {
return nil, core.Digest{}, errors.New("expected manifestlist.DeserializedManifestList for OCI index")
}
deserializedManifestIndex, ok := manifestIndex.(*manifestlist.DeserializedManifestList)
if !ok {
return nil, core.Digest{}, errors.New("expected manifestlist.DeserializedManifestList for OCI index")
}
version := deserializedManifestIndex.SchemaVersion
if version != 2 {
return nil, core.Digest{}, fmt.Errorf("unsupported OCI image index version: %d", version)
}

Copilot uses AI. Check for mistakes.
d, err := core.ParseSHA256Digest(string(desc.Digest))
if err != nil {
return nil, core.Digest{}, fmt.Errorf("parse digest: %s", err)
}
return manifestIndex, d, nil
}

// GetManifestReferences returns a list of references by a V2 manifest
func GetManifestReferences(manifest distribution.Manifest) ([]core.Digest, error) {
var refs []core.Digest
Expand All @@ -100,5 +149,10 @@
}

func GetSupportedManifestTypes() string {
return fmt.Sprintf("%s,%s", _v2ManifestType, _v2ManifestListType)
}
return fmt.Sprintf("%s,%s,%s,%s",
_v2ManifestType,
_v2ManifestListType,
specs.MediaTypeImageManifest,
specs.MediaTypeImageIndex,
)
}
80 changes: 80 additions & 0 deletions utils/dockerutil/dockerutil_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package dockerutil_test

import (
"bytes"
"testing"

"github.com/docker/distribution/manifest/manifestlist"
Expand Down Expand Up @@ -88,3 +89,82 @@ func TestParseManifestV2List(t *testing.T) {
})
}
}

var testOCIManifestBytes = []byte(`{
"schemaVersion": 2,
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"config": {
"mediaType": "application/vnd.oci.image.config.v1+json",
"size": 985,
"digest": "sha256:1a9ec845ee94c202b2d5da74a24f0ed2058318bfa9879fa541efaecba272e86b"
},
"layers": [
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"size": 153263,
"digest": "sha256:62d8908bee94c202b2d35224a221aaa2058318bfa9879fa541efaecba272331b"
}
]
}`)

var testOCIIndexBytes = []byte(`{
"schemaVersion": 2,
"mediaType": "application/vnd.oci.image.index.v1+json",
"manifests": [
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"size": 985,
"digest": "sha256:1a9ec845ee94c202b2d5da74a24f0ed2058318bfa9879fa541efaecba272e86b",
"platform": {
"architecture": "amd64",
"os": "linux"
}
}
]
}`)

func TestParseManifestOCI(t *testing.T) {
require := require.New(t)

// Success case
manifest, d, err := dockerutil.ParseManifestOCI(testOCIManifestBytes)
require.NoError(err)
mediaType, _, err := manifest.Payload()
require.NoError(err)
require.Equal("application/vnd.oci.image.manifest.v1+json", mediaType)
Comment on lines +130 to +134
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test only asserts that the returned digest uses the sha256 algorithm, which would still pass if the digest value is incorrect. Consider asserting the full digest string matches the expected digest of testOCIManifestBytes (and adding a negative test case to ensure ParseManifestOCI rejects non-OCI media types).

Copilot uses AI. Check for mistakes.
require.Equal("sha256", d.Algo())
require.NotEmpty(d.Hex())

// Failure case: passing a Docker manifest should fail
_, _, err = dockerutil.ParseManifestOCI(testManifestBytes)
require.Error(err)
}

func TestParseManifestOCIIndex(t *testing.T) {
require := require.New(t)

// Success case
manifest, d, err := dockerutil.ParseManifestOCIIndex(testOCIIndexBytes)
require.NoError(err)
mediaType, _, err := manifest.Payload()
require.NoError(err)
require.Equal("application/vnd.oci.image.index.v1+json", mediaType)
Comment on lines +147 to +151
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test only checks the digest algorithm rather than the full digest value, and it doesn't exercise error paths (e.g., passing a manifest instead of an index). Consider asserting the exact digest string for testOCIIndexBytes and adding a failure-case test to confirm ParseManifestOCIIndex rejects non-index inputs.

Copilot uses AI. Check for mistakes.
require.Equal("sha256", d.Algo())
require.NotEmpty(d.Hex())

// Failure case: passing a Docker manifest should fail
_, _, err = dockerutil.ParseManifestOCIIndex(testManifestBytes)
require.Error(err)
}

func TestParseManifestOCIViaParseManifest(t *testing.T) {
require := require.New(t)

_, d, err := dockerutil.ParseManifest(bytes.NewReader(testOCIManifestBytes))
require.NoError(err)
require.Equal("sha256", d.Algo())

_, d, err = dockerutil.ParseManifest(bytes.NewReader(testOCIIndexBytes))
require.NoError(err)
require.Equal("sha256", d.Algo())
}
Loading