[Discuss] Remove ModelBooster API

[kube-gopher]

Background

ModelBooster is a high-level API that creates and manages ModelServing, ModelServer, and ModelRoute resources.

Nowadays, users typically create resources directly through ModelServing, while ModelBooster seems to be rarely used. So, maintaining it adds API complexity

Proposal

Consider deprecating and eventually removing the ModelBooster API and its associated implementation.

The recommended deployment model would become:

ModelServing + ModelServer + ModelRoute

Motivation

Removing ModelBooster could:

• Reduce overlap between the high-level API and the underlying APIs.
• Reduce the number of public CRDs users need to understand.
• Remove conversion and cross-resource synchronization logic.
• Simplify the controller manager, webhooks, RBAC, Helm chart, and generated clients.
• Allow ModelServing, ModelServer, and ModelRoute to evolve independently.
• Reduce the testing and maintenance burden.

Potential impact

This would be a breaking API change. Existing users may depend on:

• The ModelBooster CRD
• The ModelBooster controller
• Mutating and validating webhooks
• CLI commands and model templates
• Documentation and examples
• Generated Go clients
• Owner-reference-based lifecycle management

Resources generated by ModelBooster have owner references pointing to the corresponding ModelBooster. Deleting an existing ModelBooster may therefore also delete its generated ModelServing, ModelServer, ModelRoute, and inference pods.

[kube-gopher]

We would like to hear everyone’s views and suggestions on this issue

[avinxshKD]

We would like to hear everyone’s views and suggestions on this issue

Leaning +1. ModelBooster duplicates the lower-level APIs and adds a fair amount of controller, webhook and client maintenance. I wouldn’t remove it in one shot though, deprecate it first, provide a migration path that handles the owner references safely. then remove it in the next major release.

Happy to help on it and review :)

[chacha923]

I think this is a reasonable and needed cleanup. ModelBooster is a thin abstraction over ModelServing/ModelServer/ModelRoute and adds API surface/migration overhead. I support removing it, but since this is a breaking change, please keep a migration path (deprecation notice, docs/samples conversion, and impact updates for CLI/webhooks/clients). Happy to help with migration docs or tests.

[safiya2610]

I also support removing ModelBooster to improve GitOps compatibility and also it will make debugging easier by exposing the underlying resources directly.we can replace this server-side controller with a simple client-side CLI command that scaffolds the necessary YAMLs. Before removal, we just need to provide users a way to strip ownerReferences so their live workloads aren't deleted.

[aeron-gh]

+1 to deprecate first and remove later. two data points from recent work that may help size the cleanp:

• inside the controller manager, the modelbooster controller is the only code that touches the networking CRDs (informers on ModelServers and ModelRoutes plus the create/update/delete handlers). so the networking rules in the controller manager ClusterRole (rbac: allow controller-manager to delete modelservers and modelroutes #1646) and the coupling in ModelBooster controller never starts on a workload only install #1663, where a workload only install leaves that controller waiting on caches that can never sync, both go away with it, and the control plane only shape from docs: highlight Kthena's lightweight and modular architecture #1655 becomes fully clean.
• on the migration side, all 13 built-in kthena create manifest templates emit a ModelBooster today and the quick start is written around it, so the CLI and the quick start would need a ModelServing + ModelServer + ModelRoute equivalent before the deprecation notice lands.

happy to help with either peice

[vanshika2720]

I’m also leaning +1 on removing ModelBooster. It seems to add another layer on top of ModelServing, ModelServer, and ModelRoute, which makes the controller, webhook, RBAC, and client side more complicated than necessary.

I agree that we shouldn’t remove it immediately, though. The main thing I’d be careful about is migration, especially the owner references on the resources created by ModelBooster. We should make sure existing users can move to the lower-level APIs without accidentally deleting their running workloads.

I’d suggest deprecating it first, updating the CLI/templates and documentation to use ModelServing + ModelServer + ModelRoute, and providing a clear migration path for existing resources. Once that’s in place, we can remove ModelBooster in the next major release.

That gives us the cleanup while still giving existing users enough time to migrate safely.

[kube-gopher]

Thank you everyone for sharing your suggestions
If you're interested in this removal, see you at the community meeting this Wednesday

[kube-gopher]

This is the simple description

Background

ModelBooster provides a high-level deployment API that converts a single resource into:

ModelBooster
├── ModelServing
├── ModelServer
└── ModelRoute

ModelBooster itself does not participate in the inference traffic path; it mainly serves as a manifest generation and lifecycle convenience layer.

After deployment, the underlying resources are created and managed by ModelServing, ModelRoute, and ModelServer

Plan

Remove the ModelBooster API and its associated controller, webhook, CRD, RBAC, CLI integration, and related resources.

Deploy directly with Kustomize:

• ModelServing
• ModelServer
• ModelRoute

Each component remains independent, and the controllers are decoupled from each other, each managing its own API

Architecture

ModelBooster no longer participates in deployment or runtime traffic

Keeping ModelBooster

Advantages

• Provides a simple, single‑resource deployment experience.
• Automatically generates the three downstream resources.
• Centralises default value setting and cross‑resource validation.
• Recreates deleted child resources.
• Provides owner‑based garbage collection and simplified status.

Disadvantages

• Couples otherwise independent APIs and controllers.
• Hides the final generated configuration from users and GitOps tools.
• Creates unclear ownership when users directly edit the generated resources.
• Duplicates validation logic, increasing maintenance burden on controllers, webhooks, CRDs, CLI, tests, and documentation.
• Migration is risky because deleting ModelBooster may garbage‑collect its generated resources.

Removing ModelBooster

Advantages

• Deployment is done through ModelServing, ModelServer, and ModelRoute.
• Keeps controllers independent, reducing architectural coupling.
• Natively fits GitOps systems (e.g., Argo CD and Flux).
• Provides immediate access to the full capabilities of the lower‑level APIs.
• Reduces maintenance cost for Kthena’s APIs and controllers.
• Supports reusable deployment patterns via bases and overlays.

Disadvantages

• Users must understand and maintain three resources instead of one.
• Names, selectors, ports, model names, and KV connector settings must be kept consistent.
• No single resource provides a unified deployment status.
• Complex configurations may require tedious patches and overlays.
• Existing ModelBooster deployments require careful migration due to owner references and UID‑based selectors.