Primora/ROADMAP.md

Primora Roadmap

Primora is a self-hosted backend platform — the open, ownable alternative to Supabase and Appwrite. One binary deployment gives you auth, Postgres-backed APIs, object storage, document collections, API keys, an audit log, database tooling, observability, and integrations — all behind one dashboard.

Design docs: project_backend.md · project_frontend.md

---

Phase 0 — Foundation (done)

• SolidJS dashboard, dark-first design system, demo mode
• Go + Gin API, sqlc, migrations, local object storage
• Better Auth service (email/password + OAuth), JWT via JWKS
• Docker Compose stack, nginx proxy, OpenAPI-generated client
• MIT license, CONTRIBUTING, SECURITY, rewritten README/QUICK_START

Phase 1 — Surface area & release plumbing ✅

Everything an operator touches that isn't in the box yet.

• primora CLI (TypeScript, apps/cli) — login, org/project context, projects, buckets, objects up/download, keys, audit tail. cac for parsing, @clack/prompts for interactive UX, picocolors for colour. Wraps the generated @primora/api-client directly — one contract, zero duplication. Ships as @primora/cli on npm (npx primora …) and as standalone binaries via bun build --compile in the release pipeline (the opencode pattern).
• @primora/mcp server (apps/mcp) — MCP stdio server exposing primora_* tools (list/create projects, keys, audit query, storage ops) so agents can operate the platform directly. Ported from IMS's apps/mcp.
• Auth users panel — Supabase-style Authentication → Users view backed by the auth service (list users, disable, reset). Requires a small admin read endpoint on the auth service.
• Release pipeline — semver tags, CHANGELOG.md, GitHub Actions: build matrix → GitHub Release + ghcr.io images (primora/backend, primora/auth, primora/frontend, primora/cli).
• Responsive pass — sidebar already collapses; audit every page at 390px for table/card overflow.

Phase 2 — Databases (DBX)

The "any database" story, via DBX — 90+ drivers in a 25 MB MCP server. Primora does not embed DBX; the Go backend keeps @dbx-app/mcp-server as a persistent stdio subprocess and exposes its tools as REST (pattern already proven in IMS's internal/app/dbx.go).

• Connection registry ✅ — per-project saved connections (Postgres, MySQL, SQLite, Redis, MongoDB, …). Credentials stored server-side, never sent to the browser. core.db_connections table + endpoints.
• Databases page ✅ — connection list → database/schema/table browser → column/type/key inspector.
• SQL editor ✅ — query editor with ⌘/Ctrl+Enter, result grid, row detail.
• Redis console ✅ — command input + output history for redis-type connections.
• Visual overview ✅ — schema graph: tables as nodes, foreign keys as edges.
• First-party connections ✅ — the platform's own Postgres and Dragonfly auto-register as connections on every project, so "put Postgres in it and it runs Postgres" works out of the box.
• DB-to-DB links — DBX data transfer between two saved connections (e.g. Postgres → Dragonfly cache warm), plus documented patterns for pairing them.

Phase 3 — Observability (IMS merge) ✅

IMS (~/Desktop/PROG+HTML/IMS) is discontinued and folded in — same stack (Gin + pgx + Postgres), ~2k lines to port.

• Ingest API — POST /api/v1/ingest authenticated by project prm_ API keys (X-Primora-Key or X-API-Key); event types: error, metric, log, heartbeat, event; component auto-registration; single event or ≤500-event batch; permissive CORS so browser SDKs can post directly.
• Issues — error groups by fingerprint (count, first/last seen, severity, stack payload) — the Firebase Crashlytics-shaped view.
• Telemetry views — health overview (totals, activity histogram, component health), issues, events, metric series (avg/p50/p95/max), Integrate tab; live updates over SSE (GET …/telemetry/stream, ?api_key=/?token= for EventSource auth).
• @primora/client SDK — port of @ims/client: queued batching, keepalive flush, installAuto() for browser error capture.
• Agent tooling — primora_list_issues, primora_list_events, primora_telemetry_stats, primora_metric_series in apps/mcp.
• Migration note in IMS repo README → archive, point at Primora.

Phase 4 — Integrations ✅

• Connector framework — core.integrations (migration 00005): type, base URL, AES-256-GCM-encrypted credentials (PRIMORA_ENCRYPTION_KEY), health check (POST …/test). has_credentials only — secrets never serialized.
• Rybbit connector — self-hosted URL + API key; GET …/integrations/:id/analytics normalizes overview/series/top pages/referrers server-side. Integrations page carries the per-connector analytics panel. Read-only, no tracking proxy.
• Webhooks — core.webhooks + core.webhook_deliveries; events issue.created (new fingerprint on ingest), deploy.marker (POST …/deploy-markers), webhook.test. HMAC-SHA256 signature header (X-Primora-Signature), in-process dispatcher with backoff retries, HTTPS for public targets (HTTP allowed for private/self-hosted sinks).

Phase 5 — Distribution ✅

• Desktop app — apps/desktop, a Tauri 2 shell (Linux/Windows/macOS). Thin client: asks for the deployment URL on first run, renders the same SolidJS bundle, keeps a system-tray presence (Show / Change Server / Quit, close-to-tray, single-instance). No embedded stack — self-hosted model preserved.
• Phone — PWA shipped: manifest.webmanifest + Workbox service worker precache the shell (/api, /auth, /mailpit stay NetworkOnly), plus an in-app install banner on beforeinstallprompt. Native shells via Capacitor remain deferred until push notifications are a real requirement.
• Versioning — v0.x minor-per-phase until the API surface stabilizes. v0.5.0 tagged and pushed; the release pipeline now also ships desktop bundles: tag → GitHub Release notes + ghcr.io images + CLI/MCP binaries + desktop installers.
• Docs site — deferred on purpose. README stays canonical; the OpenAPI contract is the machine-readable source. The API still changes every phase — a docs site now would document a moving target. Revisit when v1.0.0 lands.

Follow-ups:

• Verify the macOS/Windows desktop bundles produced by the v0.5.0 CI run (host-verified on Linux only).
• Desktop bundle signing + auto-updater — unsigned artifacts today; add when distribution matures.

Phase 6 — Production hardening (v0.6.0)

The v1 feature surface is essentially complete; what remains is the difference between "works" and "safe to run other people's data". Roughly three-quarters of the distance is travelled — this phase is the rest.

Foundations already in place: liveness/readiness probes with dependency checks, per-key and per-user rate limiting, CORS locked to PUBLIC_URL, AES-256-GCM credential encryption (PRIMORA_ENCRYPTION_KEY enforced in production), audit log on mutations, org/project roles, SMTP + Resend in the auth service, migrations on boot, single-command compose deploy, release pipeline with images + binaries + desktop bundles.

Security

• Graceful shutdown — router.Run() drops in-flight requests on every deploy. Switch to http.Server + signal.NotifyContext with a ~10s drain.
• Security headers — add X-Frame-Options, Referrer-Policy, Permissions-Policy, and a baseline CSP at the nginx/frontend layer.
• TLS story — shipped nginx vhost is HTTP-only. Document a reverse proxy (Caddy/Traefik) in front for production; optionally ship a TLS-ready vhost with cert volume mounts. Don't embed a cert manager.
• Scoped API keys — prm_… keys are full-project-access today. Add a scopes column (ingest, read, write, admin) and enforce in the auth middleware.
• Dev-surface isolation — mailpit is proxied at /mailpit/ in the single compose file. Move dev services to docker-compose.dev.yml so a production deploy cannot expose it.
• Auth brute-force — verify Better Auth's built-in throttling covers login/signup; add IP backoff if it doesn't.
• Setup ergonomics — setup.sh should generate PRIMORA_ENCRYPTION_KEY (openssl rand -hex 32) rather than leaving it to the operator.

Reliability & operations

• Backups + restore runbook — pg_dump of core.*, storage-root archive, cron/timer example, and a documented restore path. The restore must be tested, not just the dump.
• Retention — telemetry events, audit log, and webhook deliveries grow unbounded. Per-project retention settings + a sweeper job.
• Platform metrics export — internal counters exist (surfaced in readiness); expose /metrics for Prometheus scraping.
• Production compose profile — healthchecks on every service, restart policies, pinned versions for dependencies (dragonfly is on latest).
• Upgrade runbook — today it's git pull && compose up -d. Document the safe order (backup → pull → migrate-on-boot → verify) and rollback.

Correctness

• Pagination audit — every list endpoint must paginate; verify the heaviest (audit, events, webhook deliveries) can't return unbounded rows.
• DB-to-DB links — the one open Phase 2 item: DBX transfer between two saved connections.
• External object storage — files live under BACKEND_STORAGE_ROOT on local disk. Acceptable for single-node self-hosting; document the constraint or add an S3-compatible backend behind the storage service.

Testing

• E2E smoke in CI — ci.yml runs unit tests; add a compose-up job exercising health, signup/login, project create, and ingest → issue created. Six Go test files + one frontend file is thin for a platform.
• Migration tests — goose up/down against a scratch database in CI.
• Load sanity — one k6/vegeta script against ingest + a hot read path; publish numbers, don't promise scale.

Distribution

• arm64 images — releases build amd64 only; ARM VPS/Pi can't pull. Add platforms: linux/amd64,linux/arm64 to the buildx steps.
• Carried forward from Phase 5 follow-ups: macOS/Windows bundle verification, signing + auto-updater.
• Optional: SBOM + provenance attestation on release images.

Docs

• Self-host guide — TLS, SMTP, backups, upgrade, and env reference in one place. scripts/verify-production-ready.sh already points at DEPLOYMENT_GUIDE.md / PRODUCTION_READINESS.md, which don't exist — write them or fix the script.

v1.0 bar: every Security item done, a restore test actually run, E2E smoke green in CI, and the upgrade runbook followed once on a real upgrade. Docs site revisited here, not before.

---

What is deliberately out of scope

• Managed/hosted tier — the product is the self-hosted thing.
• Realtime subscriptions, edge functions, generated REST/GraphQL per-table (Supabase PostgREST-style). Revisit after Phase 3 if wanted — Collections already cover the common case.
• Embedding DBX's desktop UI — we use its MCP server as a library, not its app.