TDvorak/Containr
1
0 Fork 0
mirror of https://github.com/Dvorinka/Containr.git synced 2026-10-28 07:41:27 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

docs: sync guides to verified behavior

Browse Source 
Rewrites AUTOSCALING.md — it was generic Cloudflare/Swarm advice that
predated the platform autoscaler. Now documents the real /scaling API,
per-service policy fields, manual scale, and the in-memory caveat.

Adds OPERATIONS.md covering cron jobs, exec console, managed databases,
preview environments (bookkeeping-only caveat), security scans
(heuristic, not Trivy), HA, and node-agent onboarding.

Authored By: TDvorak <[email protected]>

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
Tomas DvorakandDevin committed 2026-09-17 18:37:26 +02:00
1 parent f357c7adee
commit cec7d92e83
3 files changed
+146 -290
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

ROADMAP.md
+13 -10
View File
@@ -46,13 +46,13 @@ The backend exposes **~95 route registrations**; the frontend consumes
| Git providers (PAT add/remove, repo list, branches) | done | done (Settings → Git Providers) | done | | Git providers (PAT add/remove, repo list, branches) | done | done (Settings → Git Providers) | done |
| Git repo connect + webhook auto-deploy | done (`/git/repositories/connect`, `/git/webhooks`, `/api/git/webhooks/:id` receiver) | done — provider→repo→branch in Add Service, connect+webhook on create | guides claim it works | | Git repo connect + webhook auto-deploy | done (`/git/repositories/connect`, `/git/webhooks`, `/api/git/webhooks/:id` receiver) | done — provider→repo→branch in Add Service, connect+webhook on create | guides claim it works |
| GitHub App install flow | done (`install_url` + connect endpoint) | done — Install button + `installation_id` return handling | done (real-provider e2e pending credentials) | | GitHub App install flow | done (`install_url` + connect endpoint) | done — Install button + `installation_id` return handling | done (real-provider e2e pending credentials) |
| Cron jobs (CRUD, trigger, executions) | done — real scheduler + `docker exec` capture | done — Cron tab on service detail | not documented | | Cron jobs (CRUD, trigger, executions) | done — real scheduler + `docker exec` capture | done — Cron tab on service detail | OPERATIONS.md |
| Managed databases (CRUD, actions, backup/restore) | done | done — `/databases` page (conn info, actions, backups) | mentioned in README | | Managed databases (CRUD, actions, backup/restore) | done | done — `/databases` page (conn info, actions, backups) | OPERATIONS.md |
| Preview environments (CRUD, promote, cleanup) | done (bookkeeping only — no real container deploy; URL is a placeholder) | done — Previews section on service detail | not documented | | Preview environments (CRUD, promote, cleanup) | done (bookkeeping only — no real container deploy; URL is a placeholder) | done — Previews section on service detail | OPERATIONS.md |
| Security scans, vulnerabilities, compliance/GDPR reports | done (heuristic scanner — image tags, insecure URLs, build-command smells; no Trivy/Grype binary scanning) | done — `/security` page (scan trigger, findings, metrics, history) | README lists it | | Security scans, vulnerabilities, compliance/GDPR reports | done (heuristic scanner — image tags, insecure URLs, build-command smells; no Trivy/Grype binary scanning) | done — `/security` page (scan trigger, findings, metrics, history) | OPERATIONS.md |
| Audit logs | done | done — `/settings/audit-logs` filterable page | done | | Audit logs | done | done — `/settings/audit-logs` filterable page | done |
| Autoscaling policies + manual scale | done | done (beta) | per-service Scaling section on service detail; policies in-memory, lost on restart | | Autoscaling policies + manual scale | done | done (beta) — per-service Scaling section; policies in-memory, lost on restart | AUTOSCALING.md |
| HA / failover policies | done | done (beta) | `/ha` page: manager toggle, failover policies, alerts, health results; state is in-memory | | HA / failover policies | done | done (beta) | OPERATIONS.md |
| Node agents (register additional VPS/LXC/VM hosts, heartbeats) | done | done — token issue/revoke + install command on Usage page | done | | Node agents (register additional VPS/LXC/VM hosts, heartbeats) | done | done — token issue/revoke + install command on Usage page | done |
| API gateway (merged APwhy): upstream services, API keys, rate limits, ops/traffic analytics | management API done — native handlers under `/api/v1/gateway/*`, schema-corrected; **traffic proxy path still not mounted** (Phase 2) | **missing** | not documented | | API gateway (merged APwhy): upstream services, API keys, rate limits, ops/traffic analytics | management API done — native handlers under `/api/v1/gateway/*`, schema-corrected; **traffic proxy path still not mounted** (Phase 2) | **missing** | not documented |
| Proxmox provider (cluster/nodes/VM/LXC CRUD) | written, **routes never registered** — unreachable | none | not documented | | Proxmox provider (cluster/nodes/VM/LXC CRUD) | written, **routes never registered** — unreachable | none | not documented |
@@ -422,10 +422,13 @@ security scans produce real reports, not empty tables.
`.env`, pulls images, runs compose; verify on a clean VPS. `.env`, pulls images, runs compose; verify on a clean VPS.
- [ ] **Self-upgrade loop**: verify `/system/upgrade/pull` actually swaps the - [ ] **Self-upgrade loop**: verify `/system/upgrade/pull` actually swaps the
running image; document rollback. running image; document rollback.
- [ ] **Docs pass**: sync all guides to reality (AUTOSCALING currently - [x] **Docs pass (partial)**: `AUTOSCALING.md` rewritten to document the
describes a non-existent UI); rewrite getting-started to the canvas real `/scaling` API + Scaling section (was generic Cloudflare/Swarm
flow; publish OpenAPI at `/docs` inside the app (Docs page exists — advice); new `docs/guides/OPERATIONS.md` covers cron, exec console,
render the spec there). managed databases, preview environments, security scans, HA, and
node-agent onboarding — all synced to verified behavior. Remaining:
rewrite getting-started to the canvas flow; publish OpenAPI at
`/docs` inside the app (Docs page exists — render the spec there).
- [ ] **Tests**: raise frontend beyond smoke (canvas interactions, template - [ ] **Tests**: raise frontend beyond smoke (canvas interactions, template
install wizard, git connect); backend table-driven tests for install wizard, git connect); backend table-driven tests for
deployments, templates import, webhooks; Playwright e2e suite for the deployments, templates import, webhooks; Playwright e2e suite for the
docs/guides/AUTOSCALING.md
+41 -280
View File
@@ -1,288 +1,49 @@
# Autoscaling with Cloudflare Tunnel # Autoscaling
## Overview Containr has a built-in autoscaler that adjusts service replica counts against
CPU/memory targets. Policies and scale events are held in backend memory —
they do not survive a backend restart (tracked on the roadmap).
This document explains how autoscaling works when using Cloudflare Tunnel with the Containr application. ## Per-service policy
## Architecture Open a service → **Scaling** section. Set:
| Field | Meaning |
|-------|---------|
| Min replicas | Lower bound, ≥1 |
| Max replicas | Upper bound |
| Target CPU % | Scale up when average CPU exceeds this |
| Target memory % | Scale up when average memory exceeds this |
| Enabled | Whether the autoscaler evaluates this service |
Creating a policy registers the service with the autoscaler and initializes
its scaling state at `min_replicas`.
## Manual scale
The Scaling section also has a replica input + **Scale** button for an
immediate change, clamped to the policy bounds when a policy exists
(1–20 otherwise). Scaling dispatches to the node scheduler — without
registered node agents it fails with `no ready nodes available`.
## API
``` ```
Internet → Cloudflare Edge → Cloudflare Tunnel → Traefik → Backend Services GET /api/v1/scaling/policies/:serviceId
POST /api/v1/scaling/policies {service_id, min_replicas, max_replicas,
target_cpu, target_memory, enabled}
PUT /api/v1/scaling/policies/:serviceId
DELETE /api/v1/scaling/policies/:serviceId (disables, does not remove)
GET /api/v1/scaling/services/:serviceId (current/desired replicas)
POST /api/v1/scaling/services/:serviceId/scale {replicas, reason}
GET /api/v1/scaling/services/:serviceId/history
GET /api/v1/scaling/status | /metrics | /events
``` ```
## Autoscaling Considerations ## Traffic distribution
### 1. Cloudflare Tunnel Limitations Replicas are scheduled across registered node agents. In front of them you
still want a load balancer — Traefik (bundled) discovers containers via
**Cloudflare Tunnel itself does NOT provide autoscaling.** It's a secure tunneling service that: labels, or Cloudflare Tunnel can front the platform for external traffic.
- Creates a persistent connection between your infrastructure and Cloudflare's edge Neither provides autoscaling itself; they distribute traffic to whatever
- Routes traffic through Cloudflare's global network replicas the autoscaler has placed.
- Provides DDoS protection and CDN features
### 2. Where Autoscaling Happens
Autoscaling must be implemented at different layers:
#### A. Container Level (Docker Swarm/Kubernetes)
```yaml
# Example with Docker Swarm
backend:
image: containr-backend
deploy:
replicas: 3
update_config:
parallelism: 1
delay: 10s
restart_policy:
condition: on-failure
```
#### B. Application Level (Load Balancing)
Traefik automatically load balances between multiple backend instances:
```yaml
# Multiple backend containers
backend-1:
# ... backend config
labels:
- "traefik.http.services.backend.loadbalancer.server.port=8080"
backend-2:
# ... backend config
labels:
- "traefik.http.services.backend.loadbalancer.server.port=8080"
```
#### C. Cloud Level (Cloudflare Load Balancer - Paid Feature)
For true autoscaling, you'd need:
- Multiple deployments in different regions
- Cloudflare Load Balancer ($$$/month)
- Health checks and failover
## Implementation Options
### Option 1: Docker Swarm (Recommended for Single Host)
```bash
# Initialize Docker Swarm
docker swarm init
# Deploy with autoscaling
docker stack deploy -c docker-compose.yml containr
```
### Option 2: Kubernetes
```yaml
# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: backend
spec:
replicas: 3
selector:
matchLabels:
app: backend
template:
metadata:
labels:
app: backend
spec:
containers:
- name: backend
image: containr-backend
ports:
- containerPort: 8080
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: backend-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: backend
minReplicas: 2
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
```
### Option 3: Manual Scaling with Scripts
```bash
#!/bin/bash
# scale-backend.sh
scale_up() {
local current=$(docker ps --filter "name=containr-backend" --format "table {{.Names}}" | wc -l)
local target=$((current + 1))
echo "Scaling backend to $target instances..."
for i in $(seq 1 $target); do
docker run -d \
--name containr-backend-$i \
--network containr_containr-network \
-e DATABASE_URL="..." \
-e REDIS_URL="..." \
containr-backend
done
}
scale_down() {
local current=$(docker ps --filter "name=containr-backend" --format "table {{.Names}}" | wc -l)
local target=$((current - 1))
if [ $target -lt 1 ]; then
echo "Cannot scale below 1 instance"
exit 1
fi
echo "Scaling backend to $target instances..."
docker stop containr-backend-$target
docker rm containr-backend-$target
}
case "$1" in
up) scale_up ;;
down) scale_down ;;
*) echo "Usage: $0 [up|down]" ;;
esac
```
## Monitoring and Metrics
### Health Checks
All services include health checks:
```yaml
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
interval: 30s
timeout: 10s
retries: 3
```
### Metrics Collection
Traefik provides Prometheus metrics:
```yaml
# In docker-compose.yml
command:
- "--metrics.prometheus=true"
- "--metrics.prometheus.addentrypointslabels=true"
- "--metrics.prometheus.addserviceslabels=true"
```
### Scaling Triggers
Monitor these metrics for scaling decisions:
- CPU usage (> 70%)
- Memory usage (> 80%)
- Response time (> 500ms)
- Error rate (> 5%)
- Queue depth (if using message queues)
## Production Recommendations
### 1. Use Docker Swarm or Kubernetes
- Better orchestration
- Built-in load balancing
- Health management
- Rolling updates
### 2. Implement Horizontal Pod Autoscaler (HPA)
- Automatic scaling based on metrics
- Min/max replica limits
- Configurable thresholds
### 3. Use Cloudflare Load Balancer (if budget allows)
- Geographic distribution
- Advanced health checks
- Traffic steering
- DDoS protection
### 4. Monitoring and Alerting
- Prometheus + Grafana
- Alertmanager
- Log aggregation (ELK stack)
## Example: Complete Autoscaling Setup
```yaml
# docker-compose.autoscale.yml
version: '3.8'
services:
traefik:
image: traefik:v3.2
command:
- "--api.dashboard=true"
- "--providers.docker=true"
- "--providers.docker.swarmMode=true"
- "--metrics.prometheus=true"
deploy:
replicas: 1
placement:
constraints:
- node.role == manager
backend:
image: containr-backend
deploy:
replicas: 3
update_config:
parallelism: 1
delay: 10s
restart_policy:
condition: on-failure
labels:
- "traefik.http.services.backend.loadbalancer.server.port=8080"
- "traefik.http.routers.backend.rule=Host(`api.${DOMAIN}`)"
- "traefik.enable=true"
prometheus:
image: prom/prometheus
deploy:
replicas: 1
volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml
grafana:
image: grafana/grafana
deploy:
replicas: 1
environment:
- GF_SECURITY_ADMIN_PASSWORD=admin
```
## Summary
1. **Cloudflare Tunnel ≠ Autoscaling** - It's for secure connectivity
2. **Autoscaling happens at container/orchestration level**
3. **Traefik provides load balancing between instances**
4. **Use Docker Swarm or Kubernetes for production autoscaling**
5. **Monitor metrics and implement HPA for automatic scaling**
6. **Consider Cloudflare Load Balancer for multi-region setups**
## Quick Start Commands
```bash
# Start with autoscaling (Docker Swarm)
docker swarm init
docker stack deploy -c docker-compose.autoscale.yml containr
# Scale manually
docker service scale containr_backend=5
# Check status
docker service ls
docker service ps containr_backend
# View logs
docker service logs containr_backend
```
docs/guides/OPERATIONS.md
+92
View File
@@ -0,0 +1,92 @@
# Operations
Day-2 features for running services: scheduled jobs, ad-hoc exec, managed
databases, preview environments, security scans, and high availability.
## Cron jobs
Service detail → **Cron**. Standard 5-field cron expressions (`robfig/cron`
syntax), per-job timezone, enable/disable, manual trigger, and execution
history with captured stdout/stderr.
- The scheduler ticks every minute and runs due jobs via `docker exec`
(`sh -c <command>`) inside the service's container.
- Executions are retained per-job (`retention` count) and visible in the UI.
- Requires a running container — a stopped service's due job records a
failure, it does not queue.
```
GET/POST /api/v1/cron-jobs PUT/DELETE /api/v1/cron-jobs/:id
POST /api/v1/cron-jobs/:id/trigger
GET /api/v1/cron-jobs/:id/executions
```
## Exec console
Service detail → **Console**. One-off `sh -c` commands against the running
container: 30s ceiling, 64KB output cap, exit code + captured output shown
per run. Debugging aid — not an interactive shell.
```
POST /api/v1/services/:id/exec {command}
```
## Managed databases
**/databases** page — provision PostgreSQL/Redis/MySQL/MariaDB/MongoDB/
ClickHouse/Dragonfly as managed containers on the `containr` network.
- Start/stop/restart actions; connection URL with copy.
- **Backups**: manual snapshot now, or set a cron expression
(`backup_schedule`) — due backups run inside the same scheduler tick as
cron jobs. Archives live in the `containr-db-backups` volume and are
downloadable (`GET /databases/:id/backups/:bid/download`) and restorable.
- **Bind to service**: injects `connection_url` into a service's variables
as a secret (`DATABASE_URL` by default). Redeploy the service afterward —
variables are read at deploy time.
## Preview environments
Service detail → **Previews**. Create a preview per branch/PR with a TTL,
promote it to production, or delete it.
Current limitation: previews are bookkeeping records — they do not build,
deploy, or route a real container. The generated `*.preview.containr.local`
URL is a placeholder. Real preview runtime is tracked on the roadmap.
## Security scans
**/security** page — pick a project, run a scan (dependency / configuration
/ comprehensive, optionally scoped to one service).
The scanner is heuristic config analysis, not Trivy/Grype: it flags
unpinned image tags, `http://` source/public URLs, `npm install` without
`npm ci`, `pip install` without hashes, `curl|sh` build steps, missing
health checks, and similar smells. Findings get severity badges and can be
resolved or ignored; the page also shows a security score, open/resolved
counts, scan history, and compliance status.
```
POST /api/v1/security/scans {project_id, service_id?, scan_type}
GET /api/v1/projects/:id/vulnerabilities
PUT /api/v1/vulnerabilities/:id {status: open|resolved|ignored}
GET /api/v1/projects/:id/security/metrics
GET /api/v1/projects/:id/security/history
```
## High availability
**/ha** page — manager enable/disable, node/health-check/alert status
cards, manual failover (with confirmation), per-service failover policies
(project → service picker, strategy, min healthy nodes, max failures),
active alerts with resolve, and health check results.
HA state is in-memory like autoscaling — policies and alerts reset on
backend restart.
## Node agents
**/usage** page → *Connect node*: issue a per-node token (`cagt_…`, shown
once), copy the prefilled install command, revoke tokens when done. Agents
register over `POST /api/agents/register` and heartbeat via
`POST /api/agents/heartbeat`.