Dash/BackendPlan.md

BackendPlan.md

Mission

Build the complete backend for the self-hosted homelab dashboard. The backend owns persistence, OpenAPI, migrations, ordering rules, icon uploads, widget integrations, Docker runtime, and production-grade API behavior.

This agent may edit:

• /backend
• /db
• /openapi
• root infra/docs needed for backend integration, such as docker-compose.yml, .env.example, Makefile, and README.md

This agent must not edit /frontend except to document commands or contract expectations.

Product Context

The app is a lightweight dashboard for managing self-hosted and external services. It replaces older dashboard-style tools with a modern Vercel/Nothing-inspired interface. v1 has no authentication and assumes trusted LAN/self-hosted use.

Core backend responsibilities:

• Store service cards and multiple launch URLs per service.
• Store groups, collapsed state, and drag/drop ordering.
• Store dashboard widgets and widget configuration.
• Provide one real third-party widget adapter: Pi-hole.
• Store uploaded service icons locally.
• Serve a stable OpenAPI contract so frontend can generate types/client.

Stack

• Go 1.22 or newer.
• Gin for HTTP routing.
• zap for structured logging.
• pgx for PostgreSQL access.
• PostgreSQL 16.
• goose for migrations.
• sqlc for typed query generation.
• OpenAPI 3.1 in openapi/openapi.yaml.
• Docker Compose for local/self-hosted runtime.

Repository Layout

Backend implementation should use this structure:

backend/
  cmd/server/
    main.go
  internal/config/
  internal/http/
  internal/services/
  internal/store/
  internal/widgets/
  internal/assets/
  internal/validation/
  internal/testutil/
db/
  migrations/
  query/
openapi/
  openapi.yaml

Package intent:

• cmd/server: process entrypoint, config load, logger, DB pool, router start.
• internal/config: env parsing, defaults, validation.
• internal/http: routes, handlers, request/response mapping.
• internal/services: business rules and ordering transactions.
• internal/store: sqlc-generated store plus thin transaction helpers.
• internal/widgets: widget registry, Pi-hole adapter, cache refresh logic.
• internal/assets: icon upload validation and local file storage.
• internal/validation: shared URL/name/config validation.

Runtime Configuration

Provide .env.example with:

APP_ENV=development
HTTP_ADDR=:8080
DATABASE_URL=postgres://dash:dash@postgres:5432/dash?sslmode=disable
DATA_DIR=/data
PUBLIC_BASE_URL=http://localhost:8080
WIDGET_FETCH_TIMEOUT=5s
WIDGET_CACHE_TTL=60s
MAX_ICON_UPLOAD_BYTES=524288
ALLOWED_ORIGINS=http://localhost:3000

Rules:

• Fail fast on invalid required config.
• Create icon upload directory under DATA_DIR/icons.
• Use CORS only for configured origins in development/self-hosted split runtime.
• Keep all secrets in env or database, never hardcoded.

Data Model

Use UUID primary keys, created_at, updated_at, and integer sort_order where order matters.

Tables:

groups
  id uuid pk
  name text not null
  sort_order int not null
  collapsed boolean not null default false
  created_at timestamptz not null
  updated_at timestamptz not null

services
  id uuid pk
  group_id uuid null references groups(id) on delete set null
  name text not null
  icon_url text null
  icon_asset_id uuid null references asset_files(id) on delete set null
  sort_order int not null
  created_at timestamptz not null
  updated_at timestamptz not null

service_urls
  id uuid pk
  service_id uuid not null references services(id) on delete cascade
  label text not null
  kind text not null check (kind in ('local', 'external', 'custom'))
  url text not null
  sort_order int not null
  is_primary boolean not null default false
  created_at timestamptz not null
  updated_at timestamptz not null

widget_instances
  id uuid pk
  type text not null check (type in ('clock', 'image', 'pihole'))
  title text not null
  enabled boolean not null default true
  sort_order int not null
  config jsonb not null default '{}'
  created_at timestamptz not null
  updated_at timestamptz not null

widget_cache
  widget_id uuid pk references widget_instances(id) on delete cascade
  status text not null check (status in ('fresh', 'stale', 'error'))
  data jsonb null
  error text null
  fetched_at timestamptz null
  expires_at timestamptz null
  updated_at timestamptz not null

asset_files
  id uuid pk
  original_name text not null
  stored_name text not null
  mime_type text not null
  size_bytes int not null
  public_path text not null
  created_at timestamptz not null

Indexes:

• groups(sort_order)
• services(group_id, sort_order)
• service_urls(service_id, sort_order)
• widget_instances(enabled, sort_order)
• asset_files(created_at)

Ordering invariant:

• Groups order independently.
• Services order inside their current group_id; ungrouped services use group_id = null.
• Widgets order independently.
• Reorder endpoints accept full ordered lists and persist in one transaction.

OpenAPI Contract

Backend owns openapi/openapi.yaml. Create/update it before handler implementation. Frontend must be able to generate TypeScript client/types from it.

Base path:

/api/v1

Health:

GET /health

Required endpoints:

GET    /api/v1/dashboard

GET    /api/v1/groups
POST   /api/v1/groups
GET    /api/v1/groups/{groupId}
PATCH  /api/v1/groups/{groupId}
DELETE /api/v1/groups/{groupId}

GET    /api/v1/services
POST   /api/v1/services
GET    /api/v1/services/{serviceId}
PATCH  /api/v1/services/{serviceId}
DELETE /api/v1/services/{serviceId}

PUT    /api/v1/layout

POST   /api/v1/assets/icons

GET    /api/v1/widgets
POST   /api/v1/widgets
GET    /api/v1/widgets/{widgetId}
PATCH  /api/v1/widgets/{widgetId}
DELETE /api/v1/widgets/{widgetId}
GET    /api/v1/widgets/{widgetId}/data
POST   /api/v1/widgets/{widgetId}/refresh

Core response schemas:

Dashboard:
  groups: Group[]
  ungroupedServices: Service[]
  widgets: WidgetInstance[]

Group:
  id: string
  name: string
  sortOrder: integer
  collapsed: boolean
  services: Service[]
  createdAt: string
  updatedAt: string

Service:
  id: string
  groupId: string | null
  name: string
  iconUrl: string | null
  iconAssetId: string | null
  sortOrder: integer
  urls: ServiceUrl[]
  createdAt: string
  updatedAt: string

ServiceUrl:
  id: string
  label: string
  kind: local | external | custom
  url: string
  sortOrder: integer
  isPrimary: boolean

WidgetInstance:
  id: string
  type: clock | image | pihole
  title: string
  enabled: boolean
  sortOrder: integer
  config: object
  createdAt: string
  updatedAt: string

Error schema:

ErrorResponse:
  code: string
  message: string
  details: object | null

Use stable error codes:

• validation_error
• not_found
• conflict
• upload_too_large
• unsupported_media_type
• widget_fetch_failed
• internal_error

API Behavior

Dashboard:

• GET /dashboard returns all enabled/visible dashboard data in render order.
• Include services nested under groups.
• Include ungrouped services separately.
• Include widgets regardless of data freshness; data comes from widget data endpoint.

Groups:

• Create group with next sort_order.
• Rename and collapsed state via PATCH.
• Delete empty group by default.
• If group contains services, return 409 conflict unless request includes explicit handling:
  • ?moveServicesToUngrouped=true moves services to ungrouped.
  • No hard delete of child services in v1.

Services:

• Create/update accepts service fields and full URL list.
• A service must have at least one URL.
• URL labels must be non-empty.
• Exactly one primary URL is allowed; if omitted, first URL becomes primary.
• Patch replaces URL list atomically for simplicity.
• Delete service cascades URLs.

Layout:

• PUT /layout accepts ordered groups, ordered widgets, and ordered services per group/ungrouped.
• Validate all referenced IDs exist.
• Validate every submitted service appears once.
• Persist all ordering in one DB transaction.
• Return updated dashboard.

Assets:

• POST /assets/icons accepts multipart field file.
• Allow PNG, JPEG, WebP, SVG only.
• Enforce MAX_ICON_UPLOAD_BYTES.
• Store file under DATA_DIR/icons.
• Return AssetFile with publicPath.
• Serve uploaded icons from /uploads/icons/{storedName}.

Widgets:

• Widget config is JSONB but validated by widget type.
• clock config: { "timezones": string[] }.
• image config: { "imageUrl": string, "linkUrl": string | null }.
• pihole config: { "baseUrl": string, "apiToken": string }.
• Do not expose apiToken in normal widget responses. Return masked/omitted secret fields.
• GET /widgets/{id}/data returns cache if fresh; refreshes if expired where reasonable.
• POST /widgets/{id}/refresh forces fetch and updates cache.

Pi-hole adapter:

• Use HTTP client with timeout from config.
• Normalize base URL and call Pi-hole summary endpoint.
• Support modern Pi-hole API where possible, but isolate endpoint details behind adapter.
• Return compact data useful for cards: blocked count, query count, percent blocked, status, fetchedAt.
• On failure, keep stale data if available and return status stale with error.
• If no data exists, return status error with [ERROR: ...] compatible message.

Validation Rules

• Names: trim whitespace, 1 to 80 chars.
• Labels: trim whitespace, 1 to 40 chars.
• URLs: absolute http or https only for service URLs and widget URLs.
• Icon URL: absolute http/https or uploaded asset reference.
• Reject duplicate service URL IDs in update payload.
• Reject unknown widget type.
• Reject unknown fields only if OpenAPI decoder supports this cleanly; otherwise ignore safely.

Security Posture

v1 has no authentication. Still implement baseline hardening:

• No arbitrary file paths from user input.
• Uploaded file names must be generated server-side.
• SVG uploads must be served as files, not inlined.
• Do not log Pi-hole tokens.
• CORS restricted to configured origins.
• Use request body size limits.
• Return generic internal errors to clients.

Docker and Local Run

Provide root docker-compose.yml with:

• postgres
• backend

Frontend service may be added by frontend agent later. Backend compose must still work alone.

Backend container:

• Multi-stage Dockerfile.
• Runs migrations before server start, or provide clear make migrate-up used by compose entrypoint.
• Exposes 8080.
• Mounts persistent data volume for /data.

Useful commands:

make backend-dev
make backend-test
make db-migrate-up
make db-migrate-down
make openapi-validate
make sqlc
docker compose up

Implementation Steps

1. Scaffold Go module and backend package layout.
2. Create Docker Compose, backend Dockerfile, .env.example, Makefile targets.
3. Write initial goose migrations for all tables/indexes.
4. Write sqlc config and queries for CRUD, dashboard loading, and ordering.
5. Write openapi/openapi.yaml with all schemas/endpoints/errors.
6. Implement config, logger, DB pool, health route.
7. Implement group/service/dashboard APIs.
8. Implement layout transaction.
9. Implement asset upload and static file serving.
10. Implement widget registry, config validation, cache table logic.
11. Implement Pi-hole adapter and refresh/data endpoints.
12. Add backend tests and integration tests.
13. Run full backend validation.

Test Plan

Unit tests:

• URL validation accepts http/https, rejects relative and unsupported schemes.
• Service create/update requires at least one URL.
• Primary URL fallback selects first URL.
• Group delete rejects non-empty group without move flag.
• Widget config validation rejects missing Pi-hole token/base URL.
• Pi-hole token masking never returns raw token.

Integration tests:

• Migrations apply cleanly to empty Postgres.
• Create group, create service with two URLs, fetch dashboard.
• Move service between groups through layout endpoint.
• Reorder groups/services/widgets in one transaction.
• Delete service cascades URLs.
• Upload valid icon, reject oversized/unsupported file.
• Widget refresh caches success and preserves stale data on later failure.

Smoke tests:

go test ./...
go vet ./...
make openapi-validate
docker compose up --build
curl http://localhost:8080/health
curl http://localhost:8080/api/v1/dashboard

Acceptance Criteria

• openapi/openapi.yaml exists and is valid.
• Backend compiles with go test ./....
• Migrations run against PostgreSQL 16.
• API returns deterministic JSON matching OpenAPI.
• Dashboard data persists after container restart.
• docker compose up boots backend and Postgres cleanly.
• Frontend agent can generate client from OpenAPI without manual type fixes.

Parallel Coordination

• Backend can proceed before frontend exists.
• Keep breaking OpenAPI changes deliberate and documented.
• If contract changes, update openapi/openapi.yaml first, then backend handlers/tests.
• Do not create frontend types by hand.
• Do not move shared contract into frontend.