Secrets Management Platform
Choosing Your Secret Manager
The decision tree is straightforward. Single cloud provider with no advanced requirements? Use the native secret manager. AWS Secrets Manager costs $0.40/secret/month and handles rotation for RDS, Redshift, and DocumentDB natively. GCP Secret Manager is even cheaper at $0.06/10,000 access operations.
Multi-cloud, on-prem, or need dynamic secrets? That's HashiCorp Vault territory. Vault generates short-lived database credentials, PKI certificates, and cloud IAM tokens on demand. The credential exists for minutes, not months. But Vault is operationally heavy. You need to understand unsealing, storage backends (Consul or integrated Raft), audit logging, and HA failover. Budget at least one engineer spending 20% of their time on Vault operations.
HCP Vault (the managed offering) removes the operational burden at roughly $1.58/hr for a production cluster. For most teams, this is worth it.
Dynamic Secrets Rotation
Static secrets are the biggest security risk in most organizations. A database password created two years ago, shared across 12 services, stored in five different places. Dynamic secrets solve this completely. Vault's database secrets engine creates a unique database user with a specific TTL for each requesting application. When the TTL expires, Vault revokes the credential. No shared passwords, no long-lived credentials.
The setup for PostgreSQL takes about 30 minutes. Configure Vault with a connection string and a creation SQL statement. Applications request a credential through the Vault API or a sidecar agent, get a unique username/password valid for one hour, and Vault handles revocation automatically.
Kubernetes Integration Patterns
There are three patterns for getting secrets into Kubernetes pods. Kubernetes native Secrets are the simplest but least secure. They're base64 encoded (not encrypted), stored in etcd, and visible to anyone with RBAC read access to the namespace.
External Secrets Operator (ESO) is the current best practice. ESO watches for ExternalSecret custom resources and syncs secrets from external stores (Vault, AWS Secrets Manager, GCP, Azure Key Vault) into native Kubernetes Secrets. Applications read secrets the normal way through env vars or volume mounts but the actual values live in a proper secret manager.
The Vault Agent Injector uses a mutating webhook to inject a Vault sidecar into pods. The sidecar authenticates to Vault using the pod's service account and writes secrets to a shared volume. This gives you dynamic secrets in Kubernetes but adds a container to every pod.
Secret Sprawl Detection
Secrets leak. Engineers paste database URLs into Slack, commit .env files, or hardcode API keys during debugging. Detection needs to happen at multiple layers: pre-commit hooks (using git-secrets or detect-secrets), CI pipeline scanning (GitGuardian, TruffleHog), and runtime monitoring of logs for patterns that look like credentials.
GitGuardian reports that the average company with 400 developers has over 1,000 leaked secrets in their git history. Start scanning today. TruffleHog's open source version handles git repo scanning and can be integrated into CI in under an hour.
Zero-Trust Secret Delivery
The zero-trust model for secrets means no application gets a secret just because it's running in the right environment. Every secret request requires authentication (who is asking), authorization (are they allowed this secret), and auditing (log every access). Vault's identity-based access ties secret permissions to the application's identity, whether that's a Kubernetes service account, an AWS IAM role, or an OIDC token. This eliminates the "if you're in the VPC, you can read everything" anti-pattern that most organizations start with.
Key Points
- •Dynamic secrets that expire after use eliminate the risk of long-lived credentials sitting in config files for months
- •HashiCorp Vault handles 10,000+ secret reads per second in production clusters but requires dedicated operational expertise to run reliably
- •External Secrets Operator syncs secrets from Vault or cloud secret managers into Kubernetes, keeping your manifests free of secret references
- •Secret sprawl detection tools like TruffleHog and GitGuardian catch credentials that leak into git repos, Slack messages, and CI logs
- •Cloud-native secret managers (AWS Secrets Manager, GCP Secret Manager) are the right default unless you need multi-cloud or advanced dynamic secrets
Common Mistakes
- ✗Storing secrets in Kubernetes native Secrets without encryption at rest, which are just base64 encoded and readable by anyone with cluster access
- ✗Running HashiCorp Vault without understanding unsealing, HA configuration, and backup procedures, leading to outages that lock out all applications
- ✗Rotating secrets in the secret manager but not restarting or reloading the applications that cached the old secret value