From cec7d92e83e52e74f96fe7a82bb134dfd9da940a Mon Sep 17 00:00:00 2001 From: Tomas Dvorak Date: Thu, 17 Sep 2026 18:37:26 +0200 Subject: [PATCH] docs: sync guides to verified behavior MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rewrites AUTOSCALING.md — it was generic Cloudflare/Swarm advice that predated the platform autoscaler. Now documents the real /scaling API, per-service policy fields, manual scale, and the in-memory caveat. Adds OPERATIONS.md covering cron jobs, exec console, managed databases, preview environments (bookkeeping-only caveat), security scans (heuristic, not Trivy), HA, and node-agent onboarding. Authored By: TDvorak Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- ROADMAP.md | 23 +-- docs/guides/AUTOSCALING.md | 321 +++++-------------------------------- docs/guides/OPERATIONS.md | 92 +++++++++++ 3 files changed, 146 insertions(+), 290 deletions(-) create mode 100644 docs/guides/OPERATIONS.md diff --git a/ROADMAP.md b/ROADMAP.md index a0e0fee..40ad1fc 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -46,13 +46,13 @@ The backend exposes **~95 route registrations**; the frontend consumes | Git providers (PAT add/remove, repo list, branches) | done | done (Settings → Git Providers) | done | | Git repo connect + webhook auto-deploy | done (`/git/repositories/connect`, `/git/webhooks`, `/api/git/webhooks/:id` receiver) | done — provider→repo→branch in Add Service, connect+webhook on create | guides claim it works | | GitHub App install flow | done (`install_url` + connect endpoint) | done — Install button + `installation_id` return handling | done (real-provider e2e pending credentials) | -| Cron jobs (CRUD, trigger, executions) | done — real scheduler + `docker exec` capture | done — Cron tab on service detail | not documented | -| Managed databases (CRUD, actions, backup/restore) | done | done — `/databases` page (conn info, actions, backups) | mentioned in README | -| Preview environments (CRUD, promote, cleanup) | done (bookkeeping only — no real container deploy; URL is a placeholder) | done — Previews section on service detail | not documented | -| Security scans, vulnerabilities, compliance/GDPR reports | done (heuristic scanner — image tags, insecure URLs, build-command smells; no Trivy/Grype binary scanning) | done — `/security` page (scan trigger, findings, metrics, history) | README lists it | +| Cron jobs (CRUD, trigger, executions) | done — real scheduler + `docker exec` capture | done — Cron tab on service detail | OPERATIONS.md | +| Managed databases (CRUD, actions, backup/restore) | done | done — `/databases` page (conn info, actions, backups) | OPERATIONS.md | +| Preview environments (CRUD, promote, cleanup) | done (bookkeeping only — no real container deploy; URL is a placeholder) | done — Previews section on service detail | OPERATIONS.md | +| Security scans, vulnerabilities, compliance/GDPR reports | done (heuristic scanner — image tags, insecure URLs, build-command smells; no Trivy/Grype binary scanning) | done — `/security` page (scan trigger, findings, metrics, history) | OPERATIONS.md | | Audit logs | done | done — `/settings/audit-logs` filterable page | done | -| Autoscaling policies + manual scale | done | done (beta) | per-service Scaling section on service detail; policies in-memory, lost on restart | -| HA / failover policies | done | done (beta) | `/ha` page: manager toggle, failover policies, alerts, health results; state is in-memory | +| Autoscaling policies + manual scale | done | done (beta) — per-service Scaling section; policies in-memory, lost on restart | AUTOSCALING.md | +| HA / failover policies | done | done (beta) | OPERATIONS.md | | Node agents (register additional VPS/LXC/VM hosts, heartbeats) | done | done — token issue/revoke + install command on Usage page | done | | API gateway (merged APwhy): upstream services, API keys, rate limits, ops/traffic analytics | management API done — native handlers under `/api/v1/gateway/*`, schema-corrected; **traffic proxy path still not mounted** (Phase 2) | **missing** | not documented | | Proxmox provider (cluster/nodes/VM/LXC CRUD) | written, **routes never registered** — unreachable | none | not documented | @@ -422,10 +422,13 @@ security scans produce real reports, not empty tables. `.env`, pulls images, runs compose; verify on a clean VPS. - [ ] **Self-upgrade loop**: verify `/system/upgrade/pull` actually swaps the running image; document rollback. -- [ ] **Docs pass**: sync all guides to reality (AUTOSCALING currently - describes a non-existent UI); rewrite getting-started to the canvas - flow; publish OpenAPI at `/docs` inside the app (Docs page exists — - render the spec there). +- [x] **Docs pass (partial)**: `AUTOSCALING.md` rewritten to document the + real `/scaling` API + Scaling section (was generic Cloudflare/Swarm + advice); new `docs/guides/OPERATIONS.md` covers cron, exec console, + managed databases, preview environments, security scans, HA, and + node-agent onboarding — all synced to verified behavior. Remaining: + rewrite getting-started to the canvas flow; publish OpenAPI at + `/docs` inside the app (Docs page exists — render the spec there). - [ ] **Tests**: raise frontend beyond smoke (canvas interactions, template install wizard, git connect); backend table-driven tests for deployments, templates import, webhooks; Playwright e2e suite for the diff --git a/docs/guides/AUTOSCALING.md b/docs/guides/AUTOSCALING.md index c0a7aa7..e91104b 100644 --- a/docs/guides/AUTOSCALING.md +++ b/docs/guides/AUTOSCALING.md @@ -1,288 +1,49 @@ -# Autoscaling with Cloudflare Tunnel +# Autoscaling -## Overview +Containr has a built-in autoscaler that adjusts service replica counts against +CPU/memory targets. Policies and scale events are held in backend memory — +they do not survive a backend restart (tracked on the roadmap). -This document explains how autoscaling works when using Cloudflare Tunnel with the Containr application. +## Per-service policy -## Architecture +Open a service → **Scaling** section. Set: + +| Field | Meaning | +|-------|---------| +| Min replicas | Lower bound, ≥1 | +| Max replicas | Upper bound | +| Target CPU % | Scale up when average CPU exceeds this | +| Target memory % | Scale up when average memory exceeds this | +| Enabled | Whether the autoscaler evaluates this service | + +Creating a policy registers the service with the autoscaler and initializes +its scaling state at `min_replicas`. + +## Manual scale + +The Scaling section also has a replica input + **Scale** button for an +immediate change, clamped to the policy bounds when a policy exists +(1–20 otherwise). Scaling dispatches to the node scheduler — without +registered node agents it fails with `no ready nodes available`. + +## API ``` -Internet → Cloudflare Edge → Cloudflare Tunnel → Traefik → Backend Services +GET /api/v1/scaling/policies/:serviceId +POST /api/v1/scaling/policies {service_id, min_replicas, max_replicas, + target_cpu, target_memory, enabled} +PUT /api/v1/scaling/policies/:serviceId +DELETE /api/v1/scaling/policies/:serviceId (disables, does not remove) +GET /api/v1/scaling/services/:serviceId (current/desired replicas) +POST /api/v1/scaling/services/:serviceId/scale {replicas, reason} +GET /api/v1/scaling/services/:serviceId/history +GET /api/v1/scaling/status | /metrics | /events ``` -## Autoscaling Considerations +## Traffic distribution -### 1. Cloudflare Tunnel Limitations - -**Cloudflare Tunnel itself does NOT provide autoscaling.** It's a secure tunneling service that: -- Creates a persistent connection between your infrastructure and Cloudflare's edge -- Routes traffic through Cloudflare's global network -- Provides DDoS protection and CDN features - -### 2. Where Autoscaling Happens - -Autoscaling must be implemented at different layers: - -#### A. Container Level (Docker Swarm/Kubernetes) -```yaml -# Example with Docker Swarm -backend: - image: containr-backend - deploy: - replicas: 3 - update_config: - parallelism: 1 - delay: 10s - restart_policy: - condition: on-failure -``` - -#### B. Application Level (Load Balancing) -Traefik automatically load balances between multiple backend instances: -```yaml -# Multiple backend containers -backend-1: - # ... backend config - labels: - - "traefik.http.services.backend.loadbalancer.server.port=8080" - -backend-2: - # ... backend config - labels: - - "traefik.http.services.backend.loadbalancer.server.port=8080" -``` - -#### C. Cloud Level (Cloudflare Load Balancer - Paid Feature) -For true autoscaling, you'd need: -- Multiple deployments in different regions -- Cloudflare Load Balancer ($$$/month) -- Health checks and failover - -## Implementation Options - -### Option 1: Docker Swarm (Recommended for Single Host) - -```bash -# Initialize Docker Swarm -docker swarm init - -# Deploy with autoscaling -docker stack deploy -c docker-compose.yml containr -``` - -### Option 2: Kubernetes - -```yaml -# deployment.yaml -apiVersion: apps/v1 -kind: Deployment -metadata: - name: backend -spec: - replicas: 3 - selector: - matchLabels: - app: backend - template: - metadata: - labels: - app: backend - spec: - containers: - - name: backend - image: containr-backend - ports: - - containerPort: 8080 ---- -apiVersion: autoscaling/v2 -kind: HorizontalPodAutoscaler -metadata: - name: backend-hpa -spec: - scaleTargetRef: - apiVersion: apps/v1 - kind: Deployment - name: backend - minReplicas: 2 - maxReplicas: 10 - metrics: - - type: Resource - resource: - name: cpu - target: - type: Utilization - averageUtilization: 70 -``` - -### Option 3: Manual Scaling with Scripts - -```bash -#!/bin/bash -# scale-backend.sh - -scale_up() { - local current=$(docker ps --filter "name=containr-backend" --format "table {{.Names}}" | wc -l) - local target=$((current + 1)) - - echo "Scaling backend to $target instances..." - - for i in $(seq 1 $target); do - docker run -d \ - --name containr-backend-$i \ - --network containr_containr-network \ - -e DATABASE_URL="..." \ - -e REDIS_URL="..." \ - containr-backend - done -} - -scale_down() { - local current=$(docker ps --filter "name=containr-backend" --format "table {{.Names}}" | wc -l) - local target=$((current - 1)) - - if [ $target -lt 1 ]; then - echo "Cannot scale below 1 instance" - exit 1 - fi - - echo "Scaling backend to $target instances..." - docker stop containr-backend-$target - docker rm containr-backend-$target -} - -case "$1" in - up) scale_up ;; - down) scale_down ;; - *) echo "Usage: $0 [up|down]" ;; -esac -``` - -## Monitoring and Metrics - -### Health Checks -All services include health checks: -```yaml -healthcheck: - test: ["CMD", "curl", "-f", "http://localhost:8080/health"] - interval: 30s - timeout: 10s - retries: 3 -``` - -### Metrics Collection -Traefik provides Prometheus metrics: -```yaml -# In docker-compose.yml -command: - - "--metrics.prometheus=true" - - "--metrics.prometheus.addentrypointslabels=true" - - "--metrics.prometheus.addserviceslabels=true" -``` - -### Scaling Triggers -Monitor these metrics for scaling decisions: -- CPU usage (> 70%) -- Memory usage (> 80%) -- Response time (> 500ms) -- Error rate (> 5%) -- Queue depth (if using message queues) - -## Production Recommendations - -### 1. Use Docker Swarm or Kubernetes -- Better orchestration -- Built-in load balancing -- Health management -- Rolling updates - -### 2. Implement Horizontal Pod Autoscaler (HPA) -- Automatic scaling based on metrics -- Min/max replica limits -- Configurable thresholds - -### 3. Use Cloudflare Load Balancer (if budget allows) -- Geographic distribution -- Advanced health checks -- Traffic steering -- DDoS protection - -### 4. Monitoring and Alerting -- Prometheus + Grafana -- Alertmanager -- Log aggregation (ELK stack) - -## Example: Complete Autoscaling Setup - -```yaml -# docker-compose.autoscale.yml -version: '3.8' - -services: - traefik: - image: traefik:v3.2 - command: - - "--api.dashboard=true" - - "--providers.docker=true" - - "--providers.docker.swarmMode=true" - - "--metrics.prometheus=true" - deploy: - replicas: 1 - placement: - constraints: - - node.role == manager - - backend: - image: containr-backend - deploy: - replicas: 3 - update_config: - parallelism: 1 - delay: 10s - restart_policy: - condition: on-failure - labels: - - "traefik.http.services.backend.loadbalancer.server.port=8080" - - "traefik.http.routers.backend.rule=Host(`api.${DOMAIN}`)" - - "traefik.enable=true" - - prometheus: - image: prom/prometheus - deploy: - replicas: 1 - volumes: - - ./prometheus.yml:/etc/prometheus/prometheus.yml - - grafana: - image: grafana/grafana - deploy: - replicas: 1 - environment: - - GF_SECURITY_ADMIN_PASSWORD=admin -``` - -## Summary - -1. **Cloudflare Tunnel ≠ Autoscaling** - It's for secure connectivity -2. **Autoscaling happens at container/orchestration level** -3. **Traefik provides load balancing between instances** -4. **Use Docker Swarm or Kubernetes for production autoscaling** -5. **Monitor metrics and implement HPA for automatic scaling** -6. **Consider Cloudflare Load Balancer for multi-region setups** - -## Quick Start Commands - -```bash -# Start with autoscaling (Docker Swarm) -docker swarm init -docker stack deploy -c docker-compose.autoscale.yml containr - -# Scale manually -docker service scale containr_backend=5 - -# Check status -docker service ls -docker service ps containr_backend - -# View logs -docker service logs containr_backend -``` +Replicas are scheduled across registered node agents. In front of them you +still want a load balancer — Traefik (bundled) discovers containers via +labels, or Cloudflare Tunnel can front the platform for external traffic. +Neither provides autoscaling itself; they distribute traffic to whatever +replicas the autoscaler has placed. diff --git a/docs/guides/OPERATIONS.md b/docs/guides/OPERATIONS.md new file mode 100644 index 0000000..4a9be55 --- /dev/null +++ b/docs/guides/OPERATIONS.md @@ -0,0 +1,92 @@ +# Operations + +Day-2 features for running services: scheduled jobs, ad-hoc exec, managed +databases, preview environments, security scans, and high availability. + +## Cron jobs + +Service detail → **Cron**. Standard 5-field cron expressions (`robfig/cron` +syntax), per-job timezone, enable/disable, manual trigger, and execution +history with captured stdout/stderr. + +- The scheduler ticks every minute and runs due jobs via `docker exec` + (`sh -c `) inside the service's container. +- Executions are retained per-job (`retention` count) and visible in the UI. +- Requires a running container — a stopped service's due job records a + failure, it does not queue. + +``` +GET/POST /api/v1/cron-jobs PUT/DELETE /api/v1/cron-jobs/:id +POST /api/v1/cron-jobs/:id/trigger +GET /api/v1/cron-jobs/:id/executions +``` + +## Exec console + +Service detail → **Console**. One-off `sh -c` commands against the running +container: 30s ceiling, 64KB output cap, exit code + captured output shown +per run. Debugging aid — not an interactive shell. + +``` +POST /api/v1/services/:id/exec {command} +``` + +## Managed databases + +**/databases** page — provision PostgreSQL/Redis/MySQL/MariaDB/MongoDB/ +ClickHouse/Dragonfly as managed containers on the `containr` network. + +- Start/stop/restart actions; connection URL with copy. +- **Backups**: manual snapshot now, or set a cron expression + (`backup_schedule`) — due backups run inside the same scheduler tick as + cron jobs. Archives live in the `containr-db-backups` volume and are + downloadable (`GET /databases/:id/backups/:bid/download`) and restorable. +- **Bind to service**: injects `connection_url` into a service's variables + as a secret (`DATABASE_URL` by default). Redeploy the service afterward — + variables are read at deploy time. + +## Preview environments + +Service detail → **Previews**. Create a preview per branch/PR with a TTL, +promote it to production, or delete it. + +Current limitation: previews are bookkeeping records — they do not build, +deploy, or route a real container. The generated `*.preview.containr.local` +URL is a placeholder. Real preview runtime is tracked on the roadmap. + +## Security scans + +**/security** page — pick a project, run a scan (dependency / configuration +/ comprehensive, optionally scoped to one service). + +The scanner is heuristic config analysis, not Trivy/Grype: it flags +unpinned image tags, `http://` source/public URLs, `npm install` without +`npm ci`, `pip install` without hashes, `curl|sh` build steps, missing +health checks, and similar smells. Findings get severity badges and can be +resolved or ignored; the page also shows a security score, open/resolved +counts, scan history, and compliance status. + +``` +POST /api/v1/security/scans {project_id, service_id?, scan_type} +GET /api/v1/projects/:id/vulnerabilities +PUT /api/v1/vulnerabilities/:id {status: open|resolved|ignored} +GET /api/v1/projects/:id/security/metrics +GET /api/v1/projects/:id/security/history +``` + +## High availability + +**/ha** page — manager enable/disable, node/health-check/alert status +cards, manual failover (with confirmation), per-service failover policies +(project → service picker, strategy, min healthy nodes, max failures), +active alerts with resolve, and health check results. + +HA state is in-memory like autoscaling — policies and alerts reset on +backend restart. + +## Node agents + +**/usage** page → *Connect node*: issue a per-node token (`cagt_…`, shown +once), copy the prefilled install command, revoke tokens when done. Agents +register over `POST /api/agents/register` and heartbeat via +`POST /api/agents/heartbeat`.