What would you like to be added:
Kthena currently rolls each Role independently when RoleRollingUpdate is used. When multiple dependent Roles are updated together, their rollout progress may become unbalanced.
For example, if A depends on B and B depends on C, new A and B may become Ready before new C is Ready. This can cause requests to reach an incomplete new-version dependency chain. Rolling one Role too far ahead may also create idle capacity or overload another Role.
We would like Kthena to support coordinated proportional rolling updates across Roles in the same ServingGroup. Users should be able to configure the participating Roles, the allowed rollout progress difference, and their startup dependencies.
Why is this needed:
PD-disaggregated and multi-stage inference workloads commonly contain dependent Roles with different replica counts and readiness latency. Coordinating their rolling progress can keep new-version capacity approximately proportional and prevent an upstream Role from starting before compatible downstream capacity is Ready.
What would you like to be added:
Kthena currently rolls each Role independently when RoleRollingUpdate is used. When multiple dependent Roles are updated together, their rollout progress may become unbalanced.
For example, if A depends on B and B depends on C, new A and B may become Ready before new C is Ready. This can cause requests to reach an incomplete new-version dependency chain. Rolling one Role too far ahead may also create idle capacity or overload another Role.
We would like Kthena to support coordinated proportional rolling updates across Roles in the same ServingGroup. Users should be able to configure the participating Roles, the allowed rollout progress difference, and their startup dependencies.
Why is this needed:
PD-disaggregated and multi-stage inference workloads commonly contain dependent Roles with different replica counts and readiness latency. Coordinating their rolling progress can keep new-version capacity approximately proportional and prevent an upstream Role from starting before compatible downstream capacity is Ready.