vmalertmanager: silences not persisted on fresh PVC because pod fsGroup is not defaulted

[aloks1701]

Problem

A VMAlertmanager with persistent storage but no useStrictSecurity and no user-provided spec.securityContext gets a StatefulSet pod with no securityContext at all. The alertmanager process runs as uid 65534 (nobody) in the default images, but a freshly provisioned PersistentVolume is owned by root, so the process cannot write the notification log and silences files under --storage.path. Silences are silently lost across pod restarts, and depending on the CSI driver you get permission-denied on the data directory.

Reproduction

1. Create a VMAlertmanager with spec.storage set (a real StorageClass, fresh PVC) and default security settings (no useStrictSecurity, no spec.securityContext).
2. Create a silence via the UI/API.
3. Delete the pod so it reschedules onto the same PVC.
4. The silence is gone. Inspecting the pod shows no fsGroup, and the alertmanager data dir is root-owned and not group-writable by 65534.

Why fsGroup matters

fsGroup makes the kubelet recursively chown/chmod the mounted volume to the supplemental group so a non-root process can write to it. The operator already defaults fsGroup: 65534 when useStrictSecurity: true (via getDefaultPodSecurityContext), but the default path (addStrictSecuritySettingsToPod) returns nil, so the common default deployment silently has no fsGroup.

Our production context

We run a fleet of VMAlertmanager instances and hit this in production. We currently patch fsGroup: 65534 onto every VMAlertmanager pod securityContext manually to keep silence persistence working.

Relevant code

• internal/controller/operator/factory/build/security.go — addStrictSecuritySettingsToPod returns nil for the default case
• internal/controller/operator/factory/vmalertmanager/statefulset.go — newStsForAlertManager sets no securityContext of its own

The same "no default fsGroup on PVC" pattern also applies to VMSingle, VMStorage/VMSelect (vmcluster), and vmagent statefulStorage, but this issue is scoped to VMAlertmanager since silence loss is the most severe and silent failure mode.

Proposed fix

When a VMAlertmanager has persistent storage and neither useStrictSecurity nor a user spec.securityContext is set, default the pod securityContext to fsGroup: 65534 (with fsGroupChangePolicy: OnRootMismatch where supported). Skip under OpenShift, where the assigned SCC manages fsGroup. Any user-provided securityContext or useStrictSecurity: true takes precedence unchanged.

A PR implementing this (fix + unit tests + changelog) is ready.