What happened?
We tried to use Medusa with dynamic credentials based on AWS-compatible credentials.
We expected to pass the required config/script into the medusa and medusa-restore containers through the K8ssandraCluster/CassandraDatacenter pod customization fields.
However, custom env, volumeMounts, and related container changes for Medusa are not preserved in the final pod spec. The Medusa containers appear to be rebuilt later by the operator, so user-provided container customizations are lost.
The only working workaround we found is to use the operator’s k8ssandra.io/inject-secret pod annotation, because the secrets webhook injects the Secret volume into medusa and medusa-restore after the Medusa containers are created.
Did you expect to see something different?
Yes.
If a user configures additional volumes, mounts, or env vars for Medusa through the supported pod/container customization fields, those values should be merged into the generated medusa and medusa-restore containers instead of being overwritten.
This is especially important for role-based S3 credentials, where Medusa may need files such as:
[default]
region = example-region-1
credential_process = /bin/sh /home/cassandra/.aws/get-creds.sh
mounted at:
/home/cassandra/.aws/config
How to reproduce it (as minimally and precisely as possible)
Configure Medusa with role-based S3-compatible storage and try to add custom mounts/env for the Medusa containers.
Example relevant config:
medusa:
storageProperties:
storageProvider: s3_compatible
host: s3-compatible.example.com
port: 443
secure: true
region: example-region-1
bucketName: example-bucket
credentialsType: role-based
Then add a custom config/script mount intended for both containers:
cassandra:
containers:
- name: medusa
volumeMounts:
- name: medusa-aws-config
mountPath: /home/cassandra/.aws
- name: medusa-restore
volumeMounts:
- name: medusa-aws-config
mountPath: /home/cassandra/.aws
extraVolumes:
volumes:
- name: medusa-aws-config
secret:
secretName: medusa-aws-config
Expected: the final Cassandra pod has the mount in both medusa and medusa-restore.
Actual: the final pod does not preserve the custom Medusa mounts/env.
Workaround that does work:
cassandra:
datacenters:
- metadata:
pods:
annotations:
k8ssandra.io/inject-secret: |
[{"name":"medusa-aws-config","path":"/home/cassandra/.aws","containers":["medusa","medusa-restore"]}]
Environment
-
K8ssandra Operator version:
v1.31.0
-
cass-operator version:
v1.30.0
-
Kubernetes version information:
Kubernetes v1.34.x
-
Kubernetes cluster kind:
Managed Kubernetes cluster
-
Manifests:
medusa:
storageProperties:
storageProvider: s3_compatible
host: s3-compatible.example.com
port: 443
secure: true
region: example-region-1
bucketName: example-bucket
credentialsType: role-based
apiVersion: v1
kind: Secret
metadata:
name: medusa-aws-config
stringData:
config: |
[default]
region = example-region-1
credential_process = /bin/sh /home/cassandra/.aws/get-creds.sh
get-creds.sh: |
# example script
INFO Medusa is enabled
INFO Reconciling Medusa configMap
INFO Medusa ConfigMap successfully reconciled
INFO Found medusa-restore init container
INFO Mounted per-node configuration
INFO Vector Agent ConfigMap successfully reconciled
Anything else we need to know?
From reading the code, the issue seems to be that Medusa containers are created/updated after generic CassandraDatacenter container customization is merged.
In particular, Medusa reconciliation appears to assign generated container definitions back onto the existing containers, which drops user-provided env and volumeMounts.
The desired behavior would be to merge generated Medusa env/mounts with user-provided env/mounts by name, preserving custom user configuration unless there is an explicit conflict.
What happened?
We tried to use Medusa with dynamic credentials based on AWS-compatible credentials.
We expected to pass the required config/script into the
medusaandmedusa-restorecontainers through the K8ssandraCluster/CassandraDatacenter pod customization fields.However, custom
env,volumeMounts, and related container changes for Medusa are not preserved in the final pod spec. The Medusa containers appear to be rebuilt later by the operator, so user-provided container customizations are lost.The only working workaround we found is to use the operator’s
k8ssandra.io/inject-secretpod annotation, because the secrets webhook injects the Secret volume intomedusaandmedusa-restoreafter the Medusa containers are created.Did you expect to see something different?
Yes.
If a user configures additional volumes, mounts, or env vars for Medusa through the supported pod/container customization fields, those values should be merged into the generated
medusaandmedusa-restorecontainers instead of being overwritten.This is especially important for role-based S3 credentials, where Medusa may need files such as:
mounted at:
How to reproduce it (as minimally and precisely as possible)
Configure Medusa with role-based S3-compatible storage and try to add custom mounts/env for the Medusa containers.
Example relevant config:
Then add a custom config/script mount intended for both containers:
Expected: the final Cassandra pod has the mount in both
medusaandmedusa-restore.Actual: the final pod does not preserve the custom Medusa mounts/env.
Workaround that does work:
Environment
K8ssandra Operator version:
v1.31.0cass-operator version:
v1.30.0Kubernetes version information:
Kubernetes
v1.34.xKubernetes cluster kind:
Managed Kubernetes cluster
Manifests:
Anything else we need to know?
From reading the code, the issue seems to be that Medusa containers are created/updated after generic CassandraDatacenter container customization is merged.
In particular, Medusa reconciliation appears to assign generated container definitions back onto the existing containers, which drops user-provided
envandvolumeMounts.The desired behavior would be to merge generated Medusa env/mounts with user-provided env/mounts by name, preserving custom user configuration unless there is an explicit conflict.