Skip to content

Medusa container injection loses custom env/volumeMounts #1734

Description

@ISemene4kaI

What happened?

We tried to use Medusa with dynamic credentials based on AWS-compatible credentials.

We expected to pass the required config/script into the medusa and medusa-restore containers through the K8ssandraCluster/CassandraDatacenter pod customization fields.

However, custom env, volumeMounts, and related container changes for Medusa are not preserved in the final pod spec. The Medusa containers appear to be rebuilt later by the operator, so user-provided container customizations are lost.

The only working workaround we found is to use the operator’s k8ssandra.io/inject-secret pod annotation, because the secrets webhook injects the Secret volume into medusa and medusa-restore after the Medusa containers are created.

Did you expect to see something different?

Yes.

If a user configures additional volumes, mounts, or env vars for Medusa through the supported pod/container customization fields, those values should be merged into the generated medusa and medusa-restore containers instead of being overwritten.

This is especially important for role-based S3 credentials, where Medusa may need files such as:

[default]
region = example-region-1
credential_process = /bin/sh /home/cassandra/.aws/get-creds.sh

mounted at:

/home/cassandra/.aws/config

How to reproduce it (as minimally and precisely as possible)

Configure Medusa with role-based S3-compatible storage and try to add custom mounts/env for the Medusa containers.

Example relevant config:

medusa:
  storageProperties:
    storageProvider: s3_compatible
    host: s3-compatible.example.com
    port: 443
    secure: true
    region: example-region-1
    bucketName: example-bucket
    credentialsType: role-based

Then add a custom config/script mount intended for both containers:

cassandra:
  containers:
    - name: medusa
      volumeMounts:
        - name: medusa-aws-config
          mountPath: /home/cassandra/.aws
    - name: medusa-restore
      volumeMounts:
        - name: medusa-aws-config
          mountPath: /home/cassandra/.aws
  extraVolumes:
    volumes:
      - name: medusa-aws-config
        secret:
          secretName: medusa-aws-config

Expected: the final Cassandra pod has the mount in both medusa and medusa-restore.

Actual: the final pod does not preserve the custom Medusa mounts/env.

Workaround that does work:

cassandra:
  datacenters:
    - metadata:
        pods:
          annotations:
            k8ssandra.io/inject-secret: |
              [{"name":"medusa-aws-config","path":"/home/cassandra/.aws","containers":["medusa","medusa-restore"]}]

Environment

  • K8ssandra Operator version:

    v1.31.0

  • cass-operator version:

    v1.30.0

  • Kubernetes version information:

    Kubernetes v1.34.x

  • Kubernetes cluster kind:

    Managed Kubernetes cluster

  • Manifests:

medusa:
  storageProperties:
    storageProvider: s3_compatible
    host: s3-compatible.example.com
    port: 443
    secure: true
    region: example-region-1
    bucketName: example-bucket
    credentialsType: role-based
apiVersion: v1
kind: Secret
metadata:
  name: medusa-aws-config
stringData:
  config: |
    [default]
    region = example-region-1
    credential_process = /bin/sh /home/cassandra/.aws/get-creds.sh
  get-creds.sh: |
    # example script
  • K8ssandra Operator Logs:
INFO Medusa is enabled
INFO Reconciling Medusa configMap
INFO Medusa ConfigMap successfully reconciled
INFO Found medusa-restore init container
INFO Mounted per-node configuration
INFO Vector Agent ConfigMap successfully reconciled

Anything else we need to know?

From reading the code, the issue seems to be that Medusa containers are created/updated after generic CassandraDatacenter container customization is merged.

In particular, Medusa reconciliation appears to assign generated container definitions back onto the existing containers, which drops user-provided env and volumeMounts.

The desired behavior would be to merge generated Medusa env/mounts with user-provided env/mounts by name, preserving custom user configuration unless there is an explicit conflict.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions