<!-- Copyright (c) 2026 defconxt. All rights reserved. -->
<!-- Licensed under AGPL-3.0 — see LICENSE file for details. -->
<!-- NERF is a trademark of defconxt. -->
---
name: implementing-container-secrets-management
description: >-
  Implement secure secrets management for containerized environments using
  vault integration, encrypted runtime injection, secret rotation, and
  least-privilege access patterns to prevent credential exposure.
domain: cybersecurity
subdomain: container-security
tags:
  - secrets-management
  - container-security
  - vault
  - kubernetes-secrets
  - credential-protection
version: "1.0"
author: defconxt
license: AGPL-3.0
metadata:
  mitre-attack: ["T1552", "T1552.001"]
  cwe: ["CWE-798", "CWE-312"]
  tools: ["vault", "kubectl", "sealed-secrets", "external-secrets"]
---

## Overview

Container secrets management prevents credential exposure in images, environment
variables, logs, and orchestrator metadata. Hardcoded secrets in Dockerfiles,
unencrypted Kubernetes Secrets (base64-encoded, not encrypted), and leaked
environment variables are common attack vectors. Proper implementation uses
external secret stores with dynamic injection and automatic rotation.

## Prerequisites

- Tools: `vault`, `kubectl`, `kubeseal`, `helm`, `docker`
- Kubernetes cluster with RBAC enabled
- HashiCorp Vault instance or cloud secret manager
- Container registry access
- Namespace-level permissions for secret management

## Key Concepts

- **Secret sprawl**: Untracked secrets across repos, configs, and images
- **Dynamic secrets**: Short-lived credentials generated on demand
- **Sealed Secrets**: Encrypted Kubernetes secrets safe for Git storage
- **External Secrets Operator**: Syncs secrets from external vaults to K8s
- **Secret rotation**: Automated credential lifecycle management
- **Init container injection**: Secrets fetched at pod startup, not baked in

## Workflow

### Step 1: Audit Existing Secret Exposure

```bash
# Scan container images for hardcoded secrets
docker history --no-trunc myapp:latest | grep -iE 'password|secret|key|token'

# Check for secrets in Dockerfiles
grep -rEi '(PASSWORD|SECRET|API_KEY|TOKEN)\s*=' Dockerfile*

# Scan Kubernetes secrets (base64 decode)
kubectl get secrets -A -o json | jq -r \
  '.items[] | select(.type != "kubernetes.io/service-account-token") |
   "\(.metadata.namespace)/\(.metadata.name): \(.data | keys)"'

# Detect secrets in environment variables
kubectl get pods -A -o json | jq -r \
  '.items[].spec.containers[].env[]? |
   select(.name | test("PASS|SECRET|KEY|TOKEN"; "i")) | .name'
```

### Step 2: Deploy HashiCorp Vault

```bash
# Install Vault via Helm
helm repo add hashicorp https://helm.releases.hashicorp.com
helm install vault hashicorp/vault \
  --set "server.ha.enabled=true" \
  --set "server.ha.replicas=3" \
  -n vault --create-namespace

# Initialize and unseal
kubectl exec -it vault-0 -n vault -- vault operator init \
  -key-shares=5 -key-threshold=3 -format=json > vault-init.json

# Enable Kubernetes auth method
vault auth enable kubernetes
vault write auth/kubernetes/config \
  kubernetes_host="https://kubernetes.default.svc"
```

### Step 3: Configure Vault Secrets Engine

```bash
# Enable KV secrets engine
vault secrets enable -path=secret kv-v2

# Store application secrets
vault kv put secret/myapp/database \
  username="app_user" \
  password="$(openssl rand -base64 32)"

# Create policy for application access
vault policy write myapp-policy - << 'EOF'
path "secret/data/myapp/*" {
  capabilities = ["read"]
}
path "secret/metadata/myapp/*" {
  capabilities = ["list"]
}
EOF

# Create Kubernetes auth role
vault write auth/kubernetes/role/myapp \
  bound_service_account_names=myapp-sa \
  bound_service_account_namespaces=production \
  policies=myapp-policy \
  ttl=1h
```

### Step 4: Implement Vault Agent Sidecar Injection

```yaml
# Pod spec with Vault agent annotations
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
spec:
  template:
    metadata:
      annotations:
        vault.hashicorp.com/agent-inject: "true"
        vault.hashicorp.com/role: "myapp"
        vault.hashicorp.com/agent-inject-secret-db: "secret/data/myapp/database"
        vault.hashicorp.com/agent-inject-template-db: |
          {{- with secret "secret/data/myapp/database" -}}
          export DB_USER="{{ .Data.data.username }}"
          export DB_PASS="{{ .Data.data.password }}"
          {{- end }}
    spec:
      serviceAccountName: myapp-sa
      containers:
        - name: myapp
          image: myapp:latest
          command: ["/bin/sh", "-c", "source /vault/secrets/db && exec /app"]
```

### Step 5: Implement Sealed Secrets for GitOps

```bash
# Install Sealed Secrets controller
helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets
helm install sealed-secrets sealed-secrets/sealed-secrets -n kube-system

# Create and seal a secret
kubectl create secret generic myapp-db \
  --from-literal=password='supersecret' \
  --dry-run=client -o yaml | \
  kubeseal --format yaml > sealed-secret.yaml

# Safe to commit sealed-secret.yaml to Git
cat sealed-secret.yaml
```

### Step 6: Implement Secret Rotation

```bash
# Enable database secrets engine for dynamic credentials
vault secrets enable database
vault write database/config/mydb \
  plugin_name=mysql-database-plugin \
  connection_url="{{username}}:{{password}}@tcp(db:3306)/" \
  allowed_roles="myapp-role" \
  username="vault_admin" \
  password="admin_pass"

# Create role with TTL-based rotation
vault write database/roles/myapp-role \
  db_name=mydb \
  creation_statements="CREATE USER '{{name}}'@'%' IDENTIFIED BY '{{password}}'; \
    GRANT SELECT ON mydb.* TO '{{name}}'@'%';" \
  default_ttl="1h" \
  max_ttl="24h"

# Application requests dynamic credentials
vault read database/creds/myapp-role
```

## Detection Opportunities

| Signal | Source | Description |
|--------|--------|-------------|
| Secret access from unauthorized pod | Vault audit log | Pod without proper role accessing secrets |
| Base64-decoded K8s secrets in logs | SIEM | Secrets leaked to application logs |
| Hardcoded credentials in image layers | Container scanner | Secrets baked into container images |
| Failed secret authentication | Vault logs | Brute force against secret paths |

```yaml
title: Unauthorized Vault Secret Access Attempt
id: f6a7b8c9-d0e1-2345-f012-3456789abcde
status: experimental
description: Detects failed secret access attempts from unauthorized service accounts
logsource:
  product: vault
  category: audit
detection:
  selection:
    type: response
    error|contains: "permission denied"
  condition: selection | count() by auth.metadata.service_account_name > 5
  timeframe: 10m
falsepositives:
  - Newly deployed service account with pending RBAC configuration
level: high
tags:
  - attack.t1552
  - attack.credential_access
```

## Verification

- [ ] No hardcoded secrets in Dockerfiles or container images
- [ ] Kubernetes secrets encrypted at rest (etcd encryption)
- [ ] Vault integration tested with sidecar injection
- [ ] Sealed Secrets deployed for GitOps workflows
- [ ] Dynamic secret rotation configured and tested
- [ ] RBAC policies restrict secret access to authorized workloads

## References

- [HashiCorp Vault Kubernetes Integration](https://developer.hashicorp.com/vault/docs/platform/k8s)
- [Kubernetes Secrets Best Practices](https://kubernetes.io/docs/concepts/configuration/secret/)
- [Sealed Secrets](https://github.com/bitnami-labs/sealed-secrets)
- [MITRE ATT&CK T1552 — Unsecured Credentials](https://attack.mitre.org/techniques/T1552/)
