# BackendPlan.md ## Mission Build the complete backend for the self-hosted homelab dashboard. The backend owns persistence, OpenAPI, migrations, ordering rules, icon uploads, widget integrations, Docker runtime, and production-grade API behavior. This agent may edit: - `/backend` - `/db` - `/openapi` - root infra/docs needed for backend integration, such as `docker-compose.yml`, `.env.example`, `Makefile`, and `README.md` This agent must not edit `/frontend` except to document commands or contract expectations. ## Product Context The app is a lightweight dashboard for managing self-hosted and external services. It replaces older dashboard-style tools with a modern Vercel/Nothing-inspired interface. v1 has no authentication and assumes trusted LAN/self-hosted use. Core backend responsibilities: - Store service cards and multiple launch URLs per service. - Store groups, collapsed state, and drag/drop ordering. - Store dashboard widgets and widget configuration. - Provide one real third-party widget adapter: Pi-hole. - Store uploaded service icons locally. - Serve a stable OpenAPI contract so frontend can generate types/client. ## Stack - Go 1.22 or newer. - Gin for HTTP routing. - zap for structured logging. - pgx for PostgreSQL access. - PostgreSQL 16. - goose for migrations. - sqlc for typed query generation. - OpenAPI 3.1 in `openapi/openapi.yaml`. - Docker Compose for local/self-hosted runtime. ## Repository Layout Backend implementation should use this structure: ```text backend/ cmd/server/ main.go internal/config/ internal/http/ internal/services/ internal/store/ internal/widgets/ internal/assets/ internal/validation/ internal/testutil/ db/ migrations/ query/ openapi/ openapi.yaml ``` Package intent: - `cmd/server`: process entrypoint, config load, logger, DB pool, router start. - `internal/config`: env parsing, defaults, validation. - `internal/http`: routes, handlers, request/response mapping. - `internal/services`: business rules and ordering transactions. - `internal/store`: sqlc-generated store plus thin transaction helpers. - `internal/widgets`: widget registry, Pi-hole adapter, cache refresh logic. - `internal/assets`: icon upload validation and local file storage. - `internal/validation`: shared URL/name/config validation. ## Runtime Configuration Provide `.env.example` with: ```env APP_ENV=development HTTP_ADDR=:8080 DATABASE_URL=postgres://dash:dash@postgres:5432/dash?sslmode=disable DATA_DIR=/data PUBLIC_BASE_URL=http://localhost:8080 WIDGET_FETCH_TIMEOUT=5s WIDGET_CACHE_TTL=60s MAX_ICON_UPLOAD_BYTES=524288 ALLOWED_ORIGINS=http://localhost:3000 ``` Rules: - Fail fast on invalid required config. - Create icon upload directory under `DATA_DIR/icons`. - Use CORS only for configured origins in development/self-hosted split runtime. - Keep all secrets in env or database, never hardcoded. ## Data Model Use UUID primary keys, `created_at`, `updated_at`, and integer `sort_order` where order matters. Tables: ```text groups id uuid pk name text not null sort_order int not null collapsed boolean not null default false created_at timestamptz not null updated_at timestamptz not null services id uuid pk group_id uuid null references groups(id) on delete set null name text not null icon_url text null icon_asset_id uuid null references asset_files(id) on delete set null sort_order int not null created_at timestamptz not null updated_at timestamptz not null service_urls id uuid pk service_id uuid not null references services(id) on delete cascade label text not null kind text not null check (kind in ('local', 'external', 'custom')) url text not null sort_order int not null is_primary boolean not null default false created_at timestamptz not null updated_at timestamptz not null widget_instances id uuid pk type text not null check (type in ('clock', 'image', 'pihole')) title text not null enabled boolean not null default true sort_order int not null config jsonb not null default '{}' created_at timestamptz not null updated_at timestamptz not null widget_cache widget_id uuid pk references widget_instances(id) on delete cascade status text not null check (status in ('fresh', 'stale', 'error')) data jsonb null error text null fetched_at timestamptz null expires_at timestamptz null updated_at timestamptz not null asset_files id uuid pk original_name text not null stored_name text not null mime_type text not null size_bytes int not null public_path text not null created_at timestamptz not null ``` Indexes: - `groups(sort_order)` - `services(group_id, sort_order)` - `service_urls(service_id, sort_order)` - `widget_instances(enabled, sort_order)` - `asset_files(created_at)` Ordering invariant: - Groups order independently. - Services order inside their current `group_id`; ungrouped services use `group_id = null`. - Widgets order independently. - Reorder endpoints accept full ordered lists and persist in one transaction. ## OpenAPI Contract Backend owns `openapi/openapi.yaml`. Create/update it before handler implementation. Frontend must be able to generate TypeScript client/types from it. Base path: ```text /api/v1 ``` Health: ```text GET /health ``` Required endpoints: ```text GET /api/v1/dashboard GET /api/v1/groups POST /api/v1/groups GET /api/v1/groups/{groupId} PATCH /api/v1/groups/{groupId} DELETE /api/v1/groups/{groupId} GET /api/v1/services POST /api/v1/services GET /api/v1/services/{serviceId} PATCH /api/v1/services/{serviceId} DELETE /api/v1/services/{serviceId} PUT /api/v1/layout POST /api/v1/assets/icons GET /api/v1/widgets POST /api/v1/widgets GET /api/v1/widgets/{widgetId} PATCH /api/v1/widgets/{widgetId} DELETE /api/v1/widgets/{widgetId} GET /api/v1/widgets/{widgetId}/data POST /api/v1/widgets/{widgetId}/refresh ``` Core response schemas: ```yaml Dashboard: groups: Group[] ungroupedServices: Service[] widgets: WidgetInstance[] Group: id: string name: string sortOrder: integer collapsed: boolean services: Service[] createdAt: string updatedAt: string Service: id: string groupId: string | null name: string iconUrl: string | null iconAssetId: string | null sortOrder: integer urls: ServiceUrl[] createdAt: string updatedAt: string ServiceUrl: id: string label: string kind: local | external | custom url: string sortOrder: integer isPrimary: boolean WidgetInstance: id: string type: clock | image | pihole title: string enabled: boolean sortOrder: integer config: object createdAt: string updatedAt: string ``` Error schema: ```yaml ErrorResponse: code: string message: string details: object | null ``` Use stable error codes: - `validation_error` - `not_found` - `conflict` - `upload_too_large` - `unsupported_media_type` - `widget_fetch_failed` - `internal_error` ## API Behavior Dashboard: - `GET /dashboard` returns all enabled/visible dashboard data in render order. - Include services nested under groups. - Include ungrouped services separately. - Include widgets regardless of data freshness; data comes from widget data endpoint. Groups: - Create group with next `sort_order`. - Rename and collapsed state via PATCH. - Delete empty group by default. - If group contains services, return `409 conflict` unless request includes explicit handling: - `?moveServicesToUngrouped=true` moves services to ungrouped. - No hard delete of child services in v1. Services: - Create/update accepts service fields and full URL list. - A service must have at least one URL. - URL labels must be non-empty. - Exactly one primary URL is allowed; if omitted, first URL becomes primary. - Patch replaces URL list atomically for simplicity. - Delete service cascades URLs. Layout: - `PUT /layout` accepts ordered groups, ordered widgets, and ordered services per group/ungrouped. - Validate all referenced IDs exist. - Validate every submitted service appears once. - Persist all ordering in one DB transaction. - Return updated dashboard. Assets: - `POST /assets/icons` accepts multipart field `file`. - Allow PNG, JPEG, WebP, SVG only. - Enforce `MAX_ICON_UPLOAD_BYTES`. - Store file under `DATA_DIR/icons`. - Return `AssetFile` with `publicPath`. - Serve uploaded icons from `/uploads/icons/{storedName}`. Widgets: - Widget config is JSONB but validated by widget type. - `clock` config: `{ "timezones": string[] }`. - `image` config: `{ "imageUrl": string, "linkUrl": string | null }`. - `pihole` config: `{ "baseUrl": string, "apiToken": string }`. - Do not expose `apiToken` in normal widget responses. Return masked/omitted secret fields. - `GET /widgets/{id}/data` returns cache if fresh; refreshes if expired where reasonable. - `POST /widgets/{id}/refresh` forces fetch and updates cache. Pi-hole adapter: - Use HTTP client with timeout from config. - Normalize base URL and call Pi-hole summary endpoint. - Support modern Pi-hole API where possible, but isolate endpoint details behind adapter. - Return compact data useful for cards: blocked count, query count, percent blocked, status, fetchedAt. - On failure, keep stale data if available and return status `stale` with error. - If no data exists, return status `error` with `[ERROR: ...]` compatible message. ## Validation Rules - Names: trim whitespace, 1 to 80 chars. - Labels: trim whitespace, 1 to 40 chars. - URLs: absolute `http` or `https` only for service URLs and widget URLs. - Icon URL: absolute `http`/`https` or uploaded asset reference. - Reject duplicate service URL IDs in update payload. - Reject unknown widget type. - Reject unknown fields only if OpenAPI decoder supports this cleanly; otherwise ignore safely. ## Security Posture v1 has no authentication. Still implement baseline hardening: - No arbitrary file paths from user input. - Uploaded file names must be generated server-side. - SVG uploads must be served as files, not inlined. - Do not log Pi-hole tokens. - CORS restricted to configured origins. - Use request body size limits. - Return generic internal errors to clients. ## Docker and Local Run Provide root `docker-compose.yml` with: - `postgres` - `backend` Frontend service may be added by frontend agent later. Backend compose must still work alone. Backend container: - Multi-stage Dockerfile. - Runs migrations before server start, or provide clear `make migrate-up` used by compose entrypoint. - Exposes `8080`. - Mounts persistent data volume for `/data`. Useful commands: ```bash make backend-dev make backend-test make db-migrate-up make db-migrate-down make openapi-validate make sqlc docker compose up ``` ## Implementation Steps 1. Scaffold Go module and backend package layout. 2. Create Docker Compose, backend Dockerfile, `.env.example`, Makefile targets. 3. Write initial goose migrations for all tables/indexes. 4. Write sqlc config and queries for CRUD, dashboard loading, and ordering. 5. Write `openapi/openapi.yaml` with all schemas/endpoints/errors. 6. Implement config, logger, DB pool, health route. 7. Implement group/service/dashboard APIs. 8. Implement layout transaction. 9. Implement asset upload and static file serving. 10. Implement widget registry, config validation, cache table logic. 11. Implement Pi-hole adapter and refresh/data endpoints. 12. Add backend tests and integration tests. 13. Run full backend validation. ## Test Plan Unit tests: - URL validation accepts `http`/`https`, rejects relative and unsupported schemes. - Service create/update requires at least one URL. - Primary URL fallback selects first URL. - Group delete rejects non-empty group without move flag. - Widget config validation rejects missing Pi-hole token/base URL. - Pi-hole token masking never returns raw token. Integration tests: - Migrations apply cleanly to empty Postgres. - Create group, create service with two URLs, fetch dashboard. - Move service between groups through layout endpoint. - Reorder groups/services/widgets in one transaction. - Delete service cascades URLs. - Upload valid icon, reject oversized/unsupported file. - Widget refresh caches success and preserves stale data on later failure. Smoke tests: ```bash go test ./... go vet ./... make openapi-validate docker compose up --build curl http://localhost:8080/health curl http://localhost:8080/api/v1/dashboard ``` ## Acceptance Criteria - `openapi/openapi.yaml` exists and is valid. - Backend compiles with `go test ./...`. - Migrations run against PostgreSQL 16. - API returns deterministic JSON matching OpenAPI. - Dashboard data persists after container restart. - `docker compose up` boots backend and Postgres cleanly. - Frontend agent can generate client from OpenAPI without manual type fixes. ## Parallel Coordination - Backend can proceed before frontend exists. - Keep breaking OpenAPI changes deliberate and documented. - If contract changes, update `openapi/openapi.yaml` first, then backend handlers/tests. - Do not create frontend types by hand. - Do not move shared contract into frontend.