feat: phase 6 production hardening + DB-to-DB transfer

Security:
- Graceful shutdown: http.Server + signal.NotifyContext, 10s drain,
  retention sweeper cancelled before pool close
- Scoped API keys (migration 00006): ingest/read/write/admin enforced in
  auth middleware; scope picker + badges in Settings UI; scopeless legacy
  keys act as admin. Read-only keys now get 403 on ingest (verified live)
- nginx: baseline security headers on all responses, CSP scoped to SPA;
  fixed add_header inheritance shadowing
- better-auth rate limit enabled unconditionally (was gated on NODE_ENV,
  which the shipped .env sets to development)
- Mailpit moved to docker-compose.dev.yml; prod compose cannot expose it
- setup.sh generates PRIMORA_ENCRYPTION_KEY

Reliability/ops:
- Retention: per-project retention_{events,audit,webhook}_days
  (migration 00007, 0=off) + hourly RetentionSweeper; edit UI in Projects
- GET /api/v1/metrics Prometheus exposition
- compose: healthchecks + restart policies everywhere, dragonfly pinned
  to v2.0.0, depends_on service_healthy
- backup.sh/restore.sh (pg_dump -Fc + storage volume); restore verified
  against scratch postgres+volume
- .dockerignore: exclude target dirs (build context was 4.7GB)

Correctness/testing:
- DBX transfer endpoint: POST /db-connections/:id/transfer — SQL or Redis
  target, 1000-row bound, server-side creds (verified live pg→dragonfly)
- Pagination audit: ListComponents capped at 500 (auto-registers)
- CI: goose up/reset/up job + compose e2e-smoke job (smoke-e2e.sh
  verified locally: signup, bootstrap, scoped keys, ingest, metrics)
- k6 load sanity script: 6650 reqs, ~217/s, 0 real failures, p95 221ms
  (429s = configured rate limiter working)

Docs: DEPLOYMENT_GUIDE.md, PRODUCTION_READINESS.md; verify script
repaired (stale refs + set -e arithmetic abort). ROADMAP: phase 2 +
phase 6 marked complete.

Authored By: TDvorak <[email protected]>

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

apps/backend/db/queries/api_keys.sql

@@ -4,8 +4,9 @@ INSERT INTO core.api_keys (
   name,
   prefix,
   secret_hash,
+  scopes,
   created_by_user_id
-) VALUES ($1, $2, $3, $4, $5)
+) VALUES ($1, $2, $3, $4, $5, $6)
 RETURNING *;
 
 -- name: ListAPIKeysForProject :many

apps/backend/db/queries/projects.sql

@@ -28,7 +28,10 @@ RETURNING *;
 UPDATE core.projects
 SET slug = $2,
     name = $3,
-    description = $4
+    description = $4,
+    retention_events_days = COALESCE(sqlc.narg('retention_events_days'), retention_events_days),
+    retention_audit_days = COALESCE(sqlc.narg('retention_audit_days'), retention_audit_days),
+    retention_webhook_days = COALESCE(sqlc.narg('retention_webhook_days'), retention_webhook_days)
 WHERE id = $1
 RETURNING *;
 

apps/backend/db/queries/retention.sql

@@ -0,0 +1,33 @@
+-- Per-project retention sweeps. A retention of 0 days disables the sweep for
+-- that stream. Pending webhook deliveries are never swept — they may still be
+-- retrying.
+
+-- name: SweepExpiredEvents :execrows
+DELETE FROM core.events e
+USING core.projects p
+WHERE e.project_id = p.id
+  AND p.retention_events_days > 0
+  AND e.ts < NOW() - make_interval(days => p.retention_events_days);
+
+-- name: SweepExpiredAuditLogs :execrows
+DELETE FROM core.audit_logs a
+USING core.projects p
+WHERE a.project_id = p.id
+  AND p.retention_audit_days > 0
+  AND a.created_at < NOW() - make_interval(days => p.retention_audit_days);
+
+-- name: SweepExpiredOrgAuditLogs :execrows
+-- Org-scoped audit rows (project_id NULL) have no project retention — sweep
+-- them on a fixed 365-day floor so they cannot grow unbounded.
+DELETE FROM core.audit_logs a
+WHERE a.project_id IS NULL
+  AND a.created_at < NOW() - INTERVAL '365 days';
+
+-- name: SweepExpiredWebhookDeliveries :execrows
+DELETE FROM core.webhook_deliveries d
+USING core.webhooks w
+JOIN core.projects p ON p.id = w.project_id
+WHERE d.webhook_id = w.id
+  AND p.retention_webhook_days > 0
+  AND d.status <> 'pending'
+  AND d.created_at < NOW() - make_interval(days => p.retention_webhook_days);

apps/backend/db/queries/telemetry.sql

@@ -11,7 +11,8 @@ WHERE project_id = $1 AND name = $2;
 -- name: ListComponents :many
 SELECT * FROM core.components
 WHERE project_id = $1
-ORDER BY name;
+ORDER BY name
+LIMIT 500;
 
 -- name: DeleteComponent :exec
 DELETE FROM core.components