<!-- Copyright (c) 2026 defconxt. All rights reserved. -->
<!-- Licensed under AGPL-3.0 — see LICENSE file for details. -->
<!-- NERF is a trademark of defconxt. -->
# API Reference — Implementing Security Observability

## OpenTelemetry Collector
```bash
# Validate collector config
otelcol validate --config otel-collector-config.yaml

# Run collector
otelcol --config otel-collector-config.yaml

# Check collector health
curl -s http://localhost:13133/health | jq .
```

## Prometheus — Security Metrics
```bash
# Query auth failure rate
curl -s "http://prometheus:9090/api/v1/query?query=rate(security_auth_failures_total[5m])" | jq .

# Check alert rules
curl -s "http://prometheus:9090/api/v1/rules" | jq '.data.groups[].rules[] | select(.labels.team=="security")'

# Validate alerting rules
promtool check rules security_alerts.yml
```

## Grafana — Dashboard Management
```bash
# Import dashboard
curl -s -X POST "http://grafana:3000/api/dashboards/db" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d @security-dashboard.json

# List dashboards
curl -s "http://grafana:3000/api/search?tag=security" \
  -H "Authorization: Bearer $TOKEN" | jq '.[].title'
```

## Loki — Log Queries
```bash
# Query auth failures in last hour
curl -s "http://loki:3100/loki/api/v1/query_range" \
  --data-urlencode 'query={job="auth"} |= "failed"' \
  --data-urlencode 'start=1h' | jq .

# Count errors by service
curl -s "http://loki:3100/loki/api/v1/query" \
  --data-urlencode 'query=sum by (service)(count_over_time({level="error"}[1h]))' | jq .
```
