Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
b6aada7
feat: add `elastic-package requires` command to manage package depend…
teresaromero May 22, 2026
e682a05
refactor: simplify error handling in dependency resolution
teresaromero May 22, 2026
0f0fb75
refactor: rename SetRequiresDependencyVersion to setRequiresDependenc…
teresaromero May 25, 2026
64bc533
refactor: reorganize requires updates functionality
teresaromero May 25, 2026
18cbcff
refactor: streamline requires command output and dependency sorting
teresaromero May 26, 2026
0d190d5
feat: add tests for content dependency updates in requires updates
teresaromero May 26, 2026
aa5c49b
refactor: simplify version replacement logic in manifest handling
teresaromero May 26, 2026
1e761d1
test: add unit tests for latestRevision function in updates
teresaromero May 26, 2026
8c2f0e6
test: enhance Kibana constraints tests for strict-greater lower bounds
teresaromero May 26, 2026
74861af
refactor: use cobraext.GetProfileFlag in requires update command
teresaromero May 26, 2026
226e2db
refactor: remove requiresBold duplicate, reuse bold from status.go
teresaromero May 26, 2026
99cc538
refactor: remove redundant early return for SkipReason in requires up…
teresaromero May 26, 2026
e192ceb
refactor: reduce complexity in requiresupdates package
teresaromero May 26, 2026
2f1c153
refactor: address remaining requiresupdates review items
teresaromero May 26, 2026
4d81ccd
fix: update no-op message for requires update command
teresaromero May 26, 2026
1e5f9ad
fix: suppress plain-text output after JSON and on skip in requires up…
teresaromero May 26, 2026
9121c05
refactor: replace hardcoded Kibana probe versions with constraint-der…
teresaromero May 26, 2026
4e30654
Merge branch 'main' of github.com:elastic/elastic-package into requir…
teresaromero May 26, 2026
1c4b472
refactor: replace kibana overlap check with subset check
teresaromero May 29, 2026
0f96716
test: complete subset-check test coverage and rename filter function
teresaromero May 29, 2026
59f83d8
refactor: fetch all registry versions in a single request and add --p…
teresaromero May 29, 2026
fa48919
refactor: extract Apply function from Update write loop
teresaromero May 29, 2026
8d5ec20
refactor: split Update into Resolve (resolution-only) and delegate Ap…
teresaromero May 29, 2026
9e68b52
refactor: replace manifest.go YAML helpers with internal/yamledit
teresaromero May 29, 2026
a49e987
feat: add debug logging to requires update command
teresaromero May 29, 2026
d2122f4
docs: fix wording in dependency_management.md requires section
teresaromero May 29, 2026
f9b3ee3
refactor: expose NewJSONFormatter for non-spec callers and reuse in r…
teresaromero May 29, 2026
ee2eebe
test: add table-driven unit tests for fetchAllRevisions registry client
teresaromero May 29, 2026
d3cf953
refactor: convert updates_test.go to table-driven tests
teresaromero May 29, 2026
7ad0939
refactor: clean up flag definitions in cobraext package
teresaromero May 29, 2026
871d7d9
Merge branch 'main' into requires-update-command
teresaromero Jun 2, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -391,7 +391,7 @@ This command combines query capabilities with command execution, allowing you to
The command uses the same query flags as the 'find' command to select packages, then executes the specified subcommand for each matched package.

Allowed subcommands:
build, check, changelog, clean, format, install, lint, test, uninstall, update
build, check, changelog, clean, format, install, lint, requires, test, uninstall, update

### `elastic-package format`

Expand Down Expand Up @@ -506,6 +506,26 @@ _Context: package_

Generate a benchmark report comparing local results against ones from another benchmark run.

### `elastic-package requires`

_Context: package_

Manage requires dependencies for integration packages (requires.input and requires.content in manifest.yml).

Use "requires update" to bump requires.input and requires.content versions from the package registry,
respecting the integration package Kibana version constraint.

### `elastic-package requires update`

_Context: package_

Update requires.input and requires.content pins to the latest registry versions compatible with this package's Kibana constraint.

By default manifest.yml is updated. Use --dry-run to report available bumps without writing the manifest.
Version pins must be exact semver versions (constraints such as ^0.3.0 are not accepted).

When a newer dependency exists but requires a higher Kibana version than this package allows, a warning is printed suggesting to bump conditions.kibana.version on the integration package.

### `elastic-package service`

_Context: package_
Expand Down
1 change: 1 addition & 0 deletions cmd/foreach.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ func getAllowedSubCommands() []string {
"format",
"install",
"lint",
"requires",

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure this will be used on the automated script but i believe is worth enabling it for local use

"test",
"uninstall",
"update",
Expand Down
221 changes: 221 additions & 0 deletions cmd/requires.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,221 @@
// Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
// or more contributor license agreements. Licensed under the Elastic License;
// you may not use this file except in compliance with the Elastic License.

package cmd

import (
"fmt"
"io"
"os"
"path/filepath"
"slices"
"strings"

"github.com/olekukonko/tablewriter"
"github.com/olekukonko/tablewriter/renderer"
"github.com/spf13/cobra"

"github.com/elastic/elastic-package/internal/cobraext"
"github.com/elastic/elastic-package/internal/formatter"
"github.com/elastic/elastic-package/internal/install"
"github.com/elastic/elastic-package/internal/logger"
"github.com/elastic/elastic-package/internal/packages"
"github.com/elastic/elastic-package/internal/registry"
"github.com/elastic/elastic-package/internal/requiresupdates"
"github.com/elastic/elastic-package/internal/stack"
)

const (
requiresLongDescription = `Manage requires dependencies for integration packages (requires.input and requires.content in manifest.yml).

Use "requires update" to bump requires.input and requires.content versions from the package registry,
respecting the integration package Kibana version constraint.`

requiresUpdateLongDescription = `Update requires.input and requires.content pins to the latest registry versions compatible with this package's Kibana constraint.

By default manifest.yml is updated. Use --dry-run to report available bumps without writing the manifest.
Version pins must be exact semver versions (constraints such as ^0.3.0 are not accepted).

When a newer dependency exists but requires a higher Kibana version than this package allows, a warning is printed suggesting to bump conditions.kibana.version on the integration package.`
)

func setupRequiresCommand() *cobraext.Command {
updateCmd := &cobra.Command{
Use: "update",
Short: "Update requires.input and requires.content versions from the registry",
Long: requiresUpdateLongDescription,
Args: cobra.NoArgs,
RunE: requiresUpdateCommandAction,
}
updateCmd.Flags().Bool(cobraext.RequiresDryRunFlagName, false, cobraext.RequiresDryRunFlagDescription)
updateCmd.Flags().String(cobraext.RequiresFormatFlagName, requiresFormatTable, fmt.Sprintf(cobraext.RequiresFormatFlagDescription, strings.Join(requiresFormatChoices, "|")))
updateCmd.Flags().Bool(cobraext.RequiresPrereleaseFlagName, false, cobraext.RequiresPrereleaseFlagDescription)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you considered adding a flag to optionally add the changelog automatically?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, still pending to implement it

cmd := &cobra.Command{
Use: "requires",
Short: "Manage requires dependencies for integration packages",
Long: requiresLongDescription,
}
cmd.AddCommand(updateCmd)
cmd.PersistentFlags().StringP(cobraext.ProfileFlagName, "p", "", fmt.Sprintf(cobraext.ProfileFlagDescription, install.ProfileNameEnvVar))

return cobraext.NewCommand(cmd, cobraext.ContextPackage)
}

const (
requiresFormatTable = "table"
requiresFormatJSON = "json"
)

var requiresFormatChoices = []string{requiresFormatTable, requiresFormatJSON}

func requiresUpdateCommandAction(cmd *cobra.Command, _ []string) error {
dryRun, err := cmd.Flags().GetBool(cobraext.RequiresDryRunFlagName)
if err != nil {
return cobraext.FlagParsingError(err, cobraext.RequiresDryRunFlagName)
}
format, err := cmd.Flags().GetString(cobraext.RequiresFormatFlagName)
if err != nil {
return cobraext.FlagParsingError(err, cobraext.RequiresFormatFlagName)
}
if !slices.Contains(requiresFormatChoices, format) {
return fmt.Errorf("unsupported format %q, supported formats: %s", format, strings.Join(requiresFormatChoices, ", "))
}
prerelease, err := cmd.Flags().GetBool(cobraext.RequiresPrereleaseFlagName)
if err != nil {
return cobraext.FlagParsingError(err, cobraext.RequiresPrereleaseFlagName)
}

packageRoot, err := packages.MustFindPackageRoot()
if err != nil {
return fmt.Errorf("locating package root failed: %w", err)
}

prof, err := cobraext.GetProfileFlag(cmd)
if err != nil {
return err
}

appConfig, err := install.Configuration()
if err != nil {
return fmt.Errorf("loading configuration failed: %w", err)
}

baseURL := stack.PackageRegistryBaseURL(prof, appConfig)
eprClient, err := registry.NewClient(baseURL, stack.RegistryClientOptions(baseURL, prof)...)
if err != nil {
return fmt.Errorf("creating package registry client failed: %w", err)
}
logger.Debugf("using package registry: %s", baseURL)

result, err := requiresupdates.Resolve(requiresupdates.Options{
PackageRoot: packageRoot,
RegistryClient: eprClient,
Prerelease: prerelease,
})
if err != nil {
return err
}

applied := false
hasBumps := slices.ContainsFunc(result.Proposals, func(p requiresupdates.UpdateProposal) bool {
return p.Proposed != ""
})
if !dryRun && hasBumps {
manifestPath := filepath.Join(packageRoot, packages.PackageManifestFile)
manifestBytes, err := os.ReadFile(manifestPath)
if err != nil {
return fmt.Errorf("reading manifest file failed: %w", err)
}
manifestBytes, err = requiresupdates.Apply(manifestBytes, result.Proposals)
if err != nil {
return err
}
logger.Debugf("writing updated manifest: %s", manifestPath)
if err := os.WriteFile(manifestPath, manifestBytes, 0o644); err != nil {
return fmt.Errorf("writing manifest file failed: %w", err)
}
applied = true
}

for _, p := range result.Proposals {
if p.Warning != "" {
logger.Warn(p.Warning)
}
}

if err := printRequiresUpdateResult(result, os.Stdout, format); err != nil {
return err
}

if format == requiresFormatJSON {
return nil
}

if dryRun && hasBumps {
cmd.Println("Dry run: manifest.yml was not modified")
} else if applied {
cmd.Println("Updated manifest.yml")
} else if len(result.Proposals) == 0 && result.SkipReason == "" {
cmd.Println("No dependencies to update")
}

return nil
}

func printRequiresUpdateResult(result *requiresupdates.Result, w io.Writer, format string) error {
if result == nil {
return nil
}
switch format {
case requiresFormatJSON:
data, err := formatter.NewJSONFormatter().Encode(result)
if err != nil {
return err
}
_, err = fmt.Fprintln(w, string(data))
return err
case requiresFormatTable:
if result.Package != "" {
bold.Fprint(w, "Package: ") //nolint:errcheck
fmt.Fprintln(w, result.Package) //nolint:errcheck
}
if result.CodeOwner != "" {
bold.Fprint(w, "Code owner: ") //nolint:errcheck
fmt.Fprintln(w, result.CodeOwner) //nolint:errcheck
}
if result.SkipReason != "" {
fmt.Fprintln(w, result.SkipReason) //nolint:errcheck
return nil
}
if len(result.Proposals) == 0 {
return nil
}
bold.Fprintln(w, "Requires updates:") //nolint:errcheck
table := tablewriter.NewTable(w,
tablewriter.WithRenderer(renderer.NewColorized(defaultColorizedConfig())),
tablewriter.WithConfig(defaultTableConfig),
)
table.Header([]string{"Kind", "Dependency", "Current", "Proposed", "Kibana", "Warning"})
for _, p := range result.Proposals {
proposed := p.Proposed
if proposed == "" {
proposed = "-"
}
if err := table.Append([]string{
string(p.Kind),
p.Package,
p.Current,
proposed,
p.KibanaConstraint,
p.Warning,
}); err != nil {
return fmt.Errorf("populating requires update table: %w", err)
}
}
return table.Render()
default:
return fmt.Errorf("unsupported format %q", format)
}
}
1 change: 1 addition & 0 deletions cmd/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ var commands = []*cobraext.Command{
setupLintCommand(),
setupModifyCommand(),
setupProfilesCommand(),
setupRequiresCommand(),
setupReportsCommand(),
setupServiceCommand(),
setupStackCommand(),
Expand Down
63 changes: 54 additions & 9 deletions docs/howto/dependency_management.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Elastic Packages support two kinds of build-time dependency:

- **Field dependencies** — import field definitions from external schemas (e.g. ECS) using
`_dev/build/build.yml`. Resolved from Git references and cached locally.
- **Package dependencies** — composable (integration) packages can depend on input and content packages
- **Package dependencies** — integration packages with `requires` can depend on input and content packages
declared under `requires` in `manifest.yml`. **Input package** dependencies are resolved
at build time by downloading from the package registry. **Content package** dependencies are
resolved at runtime by Fleet.
Expand Down Expand Up @@ -97,9 +97,9 @@ and use a following field definition:
external: ecs
```

## Composable packages and the package registry
## Integrations with required packages and the package registry

Composable (integration) packages can also depend on input or content packages by declaring them under
Integration packages can depend on input or content packages by declaring them under
`requires` in `manifest.yml`. Depending on the package type, dependencies are resolved
differently: **input package** dependencies are fetched at build time; **content package**
dependencies are resolved at runtime by Fleet.
Expand All @@ -114,7 +114,7 @@ requires:
This type of dependency is resolved at **build time** by downloading the required input package
from the **package registry**. During `elastic-package build`, elastic-package fetches those
packages and updates the built integration: it bundles agent templates (policy and data stream),
merges variable definitions from the input packages into the composable manifest, adds data
merges variable definitions from the input packages into the integration manifest, adds data
stream field definitions where configured, and rewrites `package:` references on inputs and
streams to the concrete input types Fleet needs. Fleet still merges policy-specific values at
policy creation time.
Expand All @@ -127,16 +127,61 @@ package dependencies are fetched from the configured package registry URL
For details on using a local or custom registry when the required input packages are still
under development, see [HOWTO: Use a local or custom package registry](./local_package_registry.md).

### Testing composable packages with source overrides
### Updating `requires` pins from the package registry

When running `elastic-package test` on a composable integration whose required input packages
Integration packages with `requires` pin input and content dependencies in
`manifest.yml`. Use `elastic-package requires update` to bump those pins to the latest
versions published in the package registry that are compatible with this package's
`conditions.kibana.version` constraint.

The command queries the same registry URL used by `elastic-package build`: `stack.epr.base_url`
in the active profile, then `package_registry.base_url` in `~/.elastic-package/config.yml`,
defaulting to `https://epr.elastic.co`. To point the command at a local or custom registry,
see [HOWTO: Use a local or custom package registry](./local_package_registry.md).

By default the command writes updated versions to `manifest.yml`. Use `--dry-run` to preview
bumps without modifying the file.

```bash
# Update requires pins for the package in the current directory
elastic-package requires update

# Preview changes
elastic-package requires update --dry-run

# Machine-readable output for automation (includes package name and owner.github for CI grouping)
elastic-package requires update --dry-run --format json
```

`requires.content` pins are always written as exact semver versions (for example `"0.4.0"`).
Constraint-style pins on content dependencies are normalized to an exact version on update.

JSON output includes `package`, `codeowner` (from `owner.github` in `manifest.yml`), and `proposals`
with each dependency bump. Use `codeowner` to group batch PRs in CI. Packages skipped because they
are not applicable (for example, not an integration or no `requires` block) produce no JSON on stdout;
an info log is written instead.

When a newer dependency revision exists but its `conditions.kibana.version` does not overlap
with this package's `conditions.kibana.version` constraint, the command prints a warning suggesting to bump
`conditions.kibana.version` on the integration package. It does not change that field
automatically.

To refresh many integration packages in a repository (for example from a scheduled CI job):

```bash
elastic-package foreach --type integration requires update
```

### Testing integrations with requires using source overrides

When running `elastic-package test` on an integration with `requires` whose required input packages
are not yet published to the registry, you can point each test runner at a local copy of the
input package using the `requires` key in `_dev/test/config.yml`.

Each entry in the `requires` list uses one of two forms:

- **`source`** — a path to a local package directory or `.zip` file. Relative paths are
resolved relative to the composable package root. The package name is read from the
resolved relative to the integration package root. The package name is read from the
`manifest.yml` at that path.
- **`package` + `version`** — forces a specific version to be fetched from the registry
(useful in CI where the package is already published and you want to pin a version).
Expand All @@ -148,7 +193,7 @@ The `requires` key is supported under any test runner block: `policy`, `system`,
in more than one block, the resolved absolute paths must be identical.

```yaml
# _dev/test/config.yml — composable integration package
# _dev/test/config.yml — integration with requires
policy:
requires:
- source: "../my_input_pkg" # local directory, relative to this package root
Expand Down Expand Up @@ -176,7 +221,7 @@ Some repositories share agent templates using **link files** (files ending in `.
point at shared content). During `elastic-package build`, linked content is copied into the
build output under the **target** path (the link filename without the `.link` suffix).

Composable bundling (`requires.input`) runs **after** linked files are materialized in the
`requires.input` bundling runs **after** linked files are materialized in the
build directory. In `manifest.yml`, always set `template_path` / `template_paths` to those
**materialized** names (for example `owned.hbs`), **not** the stub name (`owned.hbs.link`).
Fleet and the builder resolve templates by the names declared in the manifest; the `.link`
Expand Down
Loading