External Secrets Operator: Stop Committing Sealed Blobs to Git

GitOps wants desired state in Git. Secrets do not belong in Git, even encrypted, if you can avoid it. Encrypted blobs still need a cluster-side key, rotation is awkward, and a leaked SealedSecrets private key decrypts history.

External Secrets Operator (ESO) stores the reference in Git and the value in a real secret manager. The cluster syncs a Kubernetes Secret at runtime.

The objects

• SecretStore / ClusterSecretStore — how to authenticate to Vault, AWS Secrets Manager, GCP Secret Manager, or Azure Key Vault.
• ExternalSecret — which remote keys become which Kubernetes Secret keys.
• Kubernetes Secret — what the Pod actually mounts. ESO owns it.

apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
  name: aws-secretsmanager
spec:
  provider:
    aws:
      service: SecretsManager
      region: us-east-1
      auth:
        jwt:
          serviceAccountRef:
            name: external-secrets
            namespace: external-secrets
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: api
  namespace: prod
spec:
  refreshInterval: 15m
  secretStoreRef:
    name: aws-secretsmanager
    kind: ClusterSecretStore
  target:
    name: api
    creationPolicy: Owner
  data:
    - secretKey: DATABASE_URL
      remoteRef:
        key: prod/api
        property: database_url

The Deployment references secretKeyRef.name: api. Git never sees the URL.

Identity: do not put static keys on the operator

On EKS, give the external-secrets ServiceAccount IRSA (or Pod Identity) with secretsmanager:GetSecretValue on the prefixes it needs. On GKE, Workload Identity. On AKS, Workload ID.

A static access key in the ClusterSecretStore is the same anti-pattern as a static key in GitHub Actions.

Scope the IAM/GCP/Azure role per store if teams must not read each other’s prefixes:

prod store  → arn:...:secret:prod/*
staging     → arn:...:secret:staging/*

Two ClusterSecretStores beat one god-role.

Refresh, rotation, and pods that never restart

refreshInterval: 15m updates the Kubernetes Secret object. It does not restart your process.

Most apps read env vars once at start. After rotation you need one of:

• reloader.stakater.com / native checksum annotation on the Deployment
• app-level watch on the mounted file (/var/secrets/database_url)
• a controlled rollout in the pipeline after the secret version changes

spec:
  template:
    metadata:
      annotations:
        checksum/api-secret: "{{ sha256sum (lookup ... ) }}" # if rendered by Helm

If you use raw manifests, Reloader is less ugly than a handwritten controller.

File mounts update on disk after kubelet sync. Env vars do not. Prefer file mounts for anything you expect to rotate without a rollout.

GitOps order

Install ESO (and its CRDs) before any ExternalSecret. In Argo CD, put CRDs and the operator in an earlier sync wave than the ExternalSecrets.

If the ExternalSecret applies first, you get no matches for kind ExternalSecret or a Secret that never appears. Apps then start with empty env and look like application bugs.

What to put in Git vs the manager

ESO dataFrom.extract can dump an entire JSON secret into a Kubernetes Secret. Convenient, and a great way to put 30 unused keys into every namespace. Prefer explicit data: entries.

Common pitfalls

1. creationPolicy: Owner plus a human-created Secret of the same name. ESO may refuse or overwrite. Pick one owner.
2. ClusterSecretStore in default with a powerful role. Treat stores as prod infrastructure.
3. No network path from the operator to the manager (private Vault, missing VPC endpoint). The ExternalSecret stays SecretSyncedError.
4. Refreshing every 10s. You will rate-limit Secrets Manager and wake up finance.
5. Using ESO to sync TLS that cert-manager already issues. Two controllers fighting one Secret.

Minimal rollout

1. Install ESO in external-secrets with IRSA/Workload Identity.
2. One ClusterSecretStore for staging, one for prod.
3. Migrate one app: create the manager secret, add ExternalSecret, delete the SealedSecret / kubectl secret.
4. Confirm Reloader or file-mount rotation in staging.
5. Ban new SealedSecrets in CI (grep / Kyverno).

For broader GitOps layout, see Kubernetes GitOps best practices.