We use helm-secrets plugin with SOPS to encrypt values files. Local helm secrets upgrade worked perfectly, but ArgoCD deployments were failing with:
[helm-secrets] Decryption failed
Error: SOPS decryption failed: error getting data key: ...
Root cause: ArgoCD's repo server pod didn't have the AWS KMS permissions needed to decrypt the SOPS-encrypted values. The IAM role attached to ArgoCD's service account didn't include kms:Decrypt.
The fix:
- Added
kms:Decryptpermission to the ArgoCD repo server IAM role:
{
"Effect": "Allow",
"Action": ["kms:Decrypt", "kms:DescribeKey"],
"Resource": "arn:aws:kms:us-east-1:123456789:key/my-sops-key"
}- Also needed to install the
helm-secretsplugin in the ArgoCD repo server container. Added to ArgoCD's Helm values:
repoServer:
volumes:
- name: helm-secrets-tools
emptyDir: {}
initContainers:
- name: install-helm-secrets
image: alpine:latest
command: [sh, -c]
args:
- apk add --no-cache curl && helm plugin install https://github.com/jkroepke/helm-secrets- Added the
.sops.yamlconfig to the repository so SOPS knows which key to use.
Lesson: ArgoCD's repo server needs its own cloud IAM permissions for any external services it accesses during sync. Test this explicitly — local success doesn't mean ArgoCD success.