mirror of
https://github.com/Dvorinka/Containr.git
synced 2026-10-28 07:41:27 +00:00
docs: sync guides to verified behavior
Rewrites AUTOSCALING.md — it was generic Cloudflare/Swarm advice that predated the platform autoscaler. Now documents the real /scaling API, per-service policy fields, manual scale, and the in-memory caveat. Adds OPERATIONS.md covering cron jobs, exec console, managed databases, preview environments (bookkeeping-only caveat), security scans (heuristic, not Trivy), HA, and node-agent onboarding. Authored By: TDvorak <[email protected]> Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
Tomas DvorakandDevin committed
1 parent
f357c7adee
commit
cec7d92e83
3 files changed
+146
-290
No files matched your search
+13
-10
@@ -46,13 +46,13 @@ The backend exposes **~95 route registrations**; the frontend consumes
|
||||
| Git providers (PAT add/remove, repo list, branches) | done | done (Settings → Git Providers) | done |
|
||||
| Git repo connect + webhook auto-deploy | done (`/git/repositories/connect`, `/git/webhooks`, `/api/git/webhooks/:id` receiver) | done — provider→repo→branch in Add Service, connect+webhook on create | guides claim it works |
|
||||
| GitHub App install flow | done (`install_url` + connect endpoint) | done — Install button + `installation_id` return handling | done (real-provider e2e pending credentials) |
|
||||
| Cron jobs (CRUD, trigger, executions) | done — real scheduler + `docker exec` capture | done — Cron tab on service detail | not documented |
|
||||
| Managed databases (CRUD, actions, backup/restore) | done | done — `/databases` page (conn info, actions, backups) | mentioned in README |
|
||||
| Preview environments (CRUD, promote, cleanup) | done (bookkeeping only — no real container deploy; URL is a placeholder) | done — Previews section on service detail | not documented |
|
||||
| Security scans, vulnerabilities, compliance/GDPR reports | done (heuristic scanner — image tags, insecure URLs, build-command smells; no Trivy/Grype binary scanning) | done — `/security` page (scan trigger, findings, metrics, history) | README lists it |
|
||||
| Cron jobs (CRUD, trigger, executions) | done — real scheduler + `docker exec` capture | done — Cron tab on service detail | OPERATIONS.md |
|
||||
| Managed databases (CRUD, actions, backup/restore) | done | done — `/databases` page (conn info, actions, backups) | OPERATIONS.md |
|
||||
| Preview environments (CRUD, promote, cleanup) | done (bookkeeping only — no real container deploy; URL is a placeholder) | done — Previews section on service detail | OPERATIONS.md |
|
||||
| Security scans, vulnerabilities, compliance/GDPR reports | done (heuristic scanner — image tags, insecure URLs, build-command smells; no Trivy/Grype binary scanning) | done — `/security` page (scan trigger, findings, metrics, history) | OPERATIONS.md |
|
||||
| Audit logs | done | done — `/settings/audit-logs` filterable page | done |
|
||||
| Autoscaling policies + manual scale | done | done (beta) | per-service Scaling section on service detail; policies in-memory, lost on restart |
|
||||
| HA / failover policies | done | done (beta) | `/ha` page: manager toggle, failover policies, alerts, health results; state is in-memory |
|
||||
| Autoscaling policies + manual scale | done | done (beta) — per-service Scaling section; policies in-memory, lost on restart | AUTOSCALING.md |
|
||||
| HA / failover policies | done | done (beta) | OPERATIONS.md |
|
||||
| Node agents (register additional VPS/LXC/VM hosts, heartbeats) | done | done — token issue/revoke + install command on Usage page | done |
|
||||
| API gateway (merged APwhy): upstream services, API keys, rate limits, ops/traffic analytics | management API done — native handlers under `/api/v1/gateway/*`, schema-corrected; **traffic proxy path still not mounted** (Phase 2) | **missing** | not documented |
|
||||
| Proxmox provider (cluster/nodes/VM/LXC CRUD) | written, **routes never registered** — unreachable | none | not documented |
|
||||
@@ -422,10 +422,13 @@ security scans produce real reports, not empty tables.
|
||||
`.env`, pulls images, runs compose; verify on a clean VPS.
|
||||
- [ ] **Self-upgrade loop**: verify `/system/upgrade/pull` actually swaps the
|
||||
running image; document rollback.
|
||||
- [ ] **Docs pass**: sync all guides to reality (AUTOSCALING currently
|
||||
describes a non-existent UI); rewrite getting-started to the canvas
|
||||
flow; publish OpenAPI at `/docs` inside the app (Docs page exists —
|
||||
render the spec there).
|
||||
- [x] **Docs pass (partial)**: `AUTOSCALING.md` rewritten to document the
|
||||
real `/scaling` API + Scaling section (was generic Cloudflare/Swarm
|
||||
advice); new `docs/guides/OPERATIONS.md` covers cron, exec console,
|
||||
managed databases, preview environments, security scans, HA, and
|
||||
node-agent onboarding — all synced to verified behavior. Remaining:
|
||||
rewrite getting-started to the canvas flow; publish OpenAPI at
|
||||
`/docs` inside the app (Docs page exists — render the spec there).
|
||||
- [ ] **Tests**: raise frontend beyond smoke (canvas interactions, template
|
||||
install wizard, git connect); backend table-driven tests for
|
||||
deployments, templates import, webhooks; Playwright e2e suite for the
|
||||
|
||||
+41
-280
@@ -1,288 +1,49 @@
|
||||
# Autoscaling with Cloudflare Tunnel
|
||||
# Autoscaling
|
||||
|
||||
## Overview
|
||||
Containr has a built-in autoscaler that adjusts service replica counts against
|
||||
CPU/memory targets. Policies and scale events are held in backend memory —
|
||||
they do not survive a backend restart (tracked on the roadmap).
|
||||
|
||||
This document explains how autoscaling works when using Cloudflare Tunnel with the Containr application.
|
||||
## Per-service policy
|
||||
|
||||
## Architecture
|
||||
Open a service → **Scaling** section. Set:
|
||||
|
||||
| Field | Meaning |
|
||||
|-------|---------|
|
||||
| Min replicas | Lower bound, ≥1 |
|
||||
| Max replicas | Upper bound |
|
||||
| Target CPU % | Scale up when average CPU exceeds this |
|
||||
| Target memory % | Scale up when average memory exceeds this |
|
||||
| Enabled | Whether the autoscaler evaluates this service |
|
||||
|
||||
Creating a policy registers the service with the autoscaler and initializes
|
||||
its scaling state at `min_replicas`.
|
||||
|
||||
## Manual scale
|
||||
|
||||
The Scaling section also has a replica input + **Scale** button for an
|
||||
immediate change, clamped to the policy bounds when a policy exists
|
||||
(1–20 otherwise). Scaling dispatches to the node scheduler — without
|
||||
registered node agents it fails with `no ready nodes available`.
|
||||
|
||||
## API
|
||||
|
||||
```
|
||||
Internet → Cloudflare Edge → Cloudflare Tunnel → Traefik → Backend Services
|
||||
GET /api/v1/scaling/policies/:serviceId
|
||||
POST /api/v1/scaling/policies {service_id, min_replicas, max_replicas,
|
||||
target_cpu, target_memory, enabled}
|
||||
PUT /api/v1/scaling/policies/:serviceId
|
||||
DELETE /api/v1/scaling/policies/:serviceId (disables, does not remove)
|
||||
GET /api/v1/scaling/services/:serviceId (current/desired replicas)
|
||||
POST /api/v1/scaling/services/:serviceId/scale {replicas, reason}
|
||||
GET /api/v1/scaling/services/:serviceId/history
|
||||
GET /api/v1/scaling/status | /metrics | /events
|
||||
```
|
||||
|
||||
## Autoscaling Considerations
|
||||
## Traffic distribution
|
||||
|
||||
### 1. Cloudflare Tunnel Limitations
|
||||
|
||||
**Cloudflare Tunnel itself does NOT provide autoscaling.** It's a secure tunneling service that:
|
||||
- Creates a persistent connection between your infrastructure and Cloudflare's edge
|
||||
- Routes traffic through Cloudflare's global network
|
||||
- Provides DDoS protection and CDN features
|
||||
|
||||
### 2. Where Autoscaling Happens
|
||||
|
||||
Autoscaling must be implemented at different layers:
|
||||
|
||||
#### A. Container Level (Docker Swarm/Kubernetes)
|
||||
```yaml
|
||||
# Example with Docker Swarm
|
||||
backend:
|
||||
image: containr-backend
|
||||
deploy:
|
||||
replicas: 3
|
||||
update_config:
|
||||
parallelism: 1
|
||||
delay: 10s
|
||||
restart_policy:
|
||||
condition: on-failure
|
||||
```
|
||||
|
||||
#### B. Application Level (Load Balancing)
|
||||
Traefik automatically load balances between multiple backend instances:
|
||||
```yaml
|
||||
# Multiple backend containers
|
||||
backend-1:
|
||||
# ... backend config
|
||||
labels:
|
||||
- "traefik.http.services.backend.loadbalancer.server.port=8080"
|
||||
|
||||
backend-2:
|
||||
# ... backend config
|
||||
labels:
|
||||
- "traefik.http.services.backend.loadbalancer.server.port=8080"
|
||||
```
|
||||
|
||||
#### C. Cloud Level (Cloudflare Load Balancer - Paid Feature)
|
||||
For true autoscaling, you'd need:
|
||||
- Multiple deployments in different regions
|
||||
- Cloudflare Load Balancer ($$$/month)
|
||||
- Health checks and failover
|
||||
|
||||
## Implementation Options
|
||||
|
||||
### Option 1: Docker Swarm (Recommended for Single Host)
|
||||
|
||||
```bash
|
||||
# Initialize Docker Swarm
|
||||
docker swarm init
|
||||
|
||||
# Deploy with autoscaling
|
||||
docker stack deploy -c docker-compose.yml containr
|
||||
```
|
||||
|
||||
### Option 2: Kubernetes
|
||||
|
||||
```yaml
|
||||
# deployment.yaml
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: backend
|
||||
spec:
|
||||
replicas: 3
|
||||
selector:
|
||||
matchLabels:
|
||||
app: backend
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: backend
|
||||
spec:
|
||||
containers:
|
||||
- name: backend
|
||||
image: containr-backend
|
||||
ports:
|
||||
- containerPort: 8080
|
||||
---
|
||||
apiVersion: autoscaling/v2
|
||||
kind: HorizontalPodAutoscaler
|
||||
metadata:
|
||||
name: backend-hpa
|
||||
spec:
|
||||
scaleTargetRef:
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
name: backend
|
||||
minReplicas: 2
|
||||
maxReplicas: 10
|
||||
metrics:
|
||||
- type: Resource
|
||||
resource:
|
||||
name: cpu
|
||||
target:
|
||||
type: Utilization
|
||||
averageUtilization: 70
|
||||
```
|
||||
|
||||
### Option 3: Manual Scaling with Scripts
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# scale-backend.sh
|
||||
|
||||
scale_up() {
|
||||
local current=$(docker ps --filter "name=containr-backend" --format "table {{.Names}}" | wc -l)
|
||||
local target=$((current + 1))
|
||||
|
||||
echo "Scaling backend to $target instances..."
|
||||
|
||||
for i in $(seq 1 $target); do
|
||||
docker run -d \
|
||||
--name containr-backend-$i \
|
||||
--network containr_containr-network \
|
||||
-e DATABASE_URL="..." \
|
||||
-e REDIS_URL="..." \
|
||||
containr-backend
|
||||
done
|
||||
}
|
||||
|
||||
scale_down() {
|
||||
local current=$(docker ps --filter "name=containr-backend" --format "table {{.Names}}" | wc -l)
|
||||
local target=$((current - 1))
|
||||
|
||||
if [ $target -lt 1 ]; then
|
||||
echo "Cannot scale below 1 instance"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Scaling backend to $target instances..."
|
||||
docker stop containr-backend-$target
|
||||
docker rm containr-backend-$target
|
||||
}
|
||||
|
||||
case "$1" in
|
||||
up) scale_up ;;
|
||||
down) scale_down ;;
|
||||
*) echo "Usage: $0 [up|down]" ;;
|
||||
esac
|
||||
```
|
||||
|
||||
## Monitoring and Metrics
|
||||
|
||||
### Health Checks
|
||||
All services include health checks:
|
||||
```yaml
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
```
|
||||
|
||||
### Metrics Collection
|
||||
Traefik provides Prometheus metrics:
|
||||
```yaml
|
||||
# In docker-compose.yml
|
||||
command:
|
||||
- "--metrics.prometheus=true"
|
||||
- "--metrics.prometheus.addentrypointslabels=true"
|
||||
- "--metrics.prometheus.addserviceslabels=true"
|
||||
```
|
||||
|
||||
### Scaling Triggers
|
||||
Monitor these metrics for scaling decisions:
|
||||
- CPU usage (> 70%)
|
||||
- Memory usage (> 80%)
|
||||
- Response time (> 500ms)
|
||||
- Error rate (> 5%)
|
||||
- Queue depth (if using message queues)
|
||||
|
||||
## Production Recommendations
|
||||
|
||||
### 1. Use Docker Swarm or Kubernetes
|
||||
- Better orchestration
|
||||
- Built-in load balancing
|
||||
- Health management
|
||||
- Rolling updates
|
||||
|
||||
### 2. Implement Horizontal Pod Autoscaler (HPA)
|
||||
- Automatic scaling based on metrics
|
||||
- Min/max replica limits
|
||||
- Configurable thresholds
|
||||
|
||||
### 3. Use Cloudflare Load Balancer (if budget allows)
|
||||
- Geographic distribution
|
||||
- Advanced health checks
|
||||
- Traffic steering
|
||||
- DDoS protection
|
||||
|
||||
### 4. Monitoring and Alerting
|
||||
- Prometheus + Grafana
|
||||
- Alertmanager
|
||||
- Log aggregation (ELK stack)
|
||||
|
||||
## Example: Complete Autoscaling Setup
|
||||
|
||||
```yaml
|
||||
# docker-compose.autoscale.yml
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
traefik:
|
||||
image: traefik:v3.2
|
||||
command:
|
||||
- "--api.dashboard=true"
|
||||
- "--providers.docker=true"
|
||||
- "--providers.docker.swarmMode=true"
|
||||
- "--metrics.prometheus=true"
|
||||
deploy:
|
||||
replicas: 1
|
||||
placement:
|
||||
constraints:
|
||||
- node.role == manager
|
||||
|
||||
backend:
|
||||
image: containr-backend
|
||||
deploy:
|
||||
replicas: 3
|
||||
update_config:
|
||||
parallelism: 1
|
||||
delay: 10s
|
||||
restart_policy:
|
||||
condition: on-failure
|
||||
labels:
|
||||
- "traefik.http.services.backend.loadbalancer.server.port=8080"
|
||||
- "traefik.http.routers.backend.rule=Host(`api.${DOMAIN}`)"
|
||||
- "traefik.enable=true"
|
||||
|
||||
prometheus:
|
||||
image: prom/prometheus
|
||||
deploy:
|
||||
replicas: 1
|
||||
volumes:
|
||||
- ./prometheus.yml:/etc/prometheus/prometheus.yml
|
||||
|
||||
grafana:
|
||||
image: grafana/grafana
|
||||
deploy:
|
||||
replicas: 1
|
||||
environment:
|
||||
- GF_SECURITY_ADMIN_PASSWORD=admin
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
1. **Cloudflare Tunnel ≠ Autoscaling** - It's for secure connectivity
|
||||
2. **Autoscaling happens at container/orchestration level**
|
||||
3. **Traefik provides load balancing between instances**
|
||||
4. **Use Docker Swarm or Kubernetes for production autoscaling**
|
||||
5. **Monitor metrics and implement HPA for automatic scaling**
|
||||
6. **Consider Cloudflare Load Balancer for multi-region setups**
|
||||
|
||||
## Quick Start Commands
|
||||
|
||||
```bash
|
||||
# Start with autoscaling (Docker Swarm)
|
||||
docker swarm init
|
||||
docker stack deploy -c docker-compose.autoscale.yml containr
|
||||
|
||||
# Scale manually
|
||||
docker service scale containr_backend=5
|
||||
|
||||
# Check status
|
||||
docker service ls
|
||||
docker service ps containr_backend
|
||||
|
||||
# View logs
|
||||
docker service logs containr_backend
|
||||
```
|
||||
Replicas are scheduled across registered node agents. In front of them you
|
||||
still want a load balancer — Traefik (bundled) discovers containers via
|
||||
labels, or Cloudflare Tunnel can front the platform for external traffic.
|
||||
Neither provides autoscaling itself; they distribute traffic to whatever
|
||||
replicas the autoscaler has placed.
|
||||
@@ -0,0 +1,92 @@
|
||||
# Operations
|
||||
|
||||
Day-2 features for running services: scheduled jobs, ad-hoc exec, managed
|
||||
databases, preview environments, security scans, and high availability.
|
||||
|
||||
## Cron jobs
|
||||
|
||||
Service detail → **Cron**. Standard 5-field cron expressions (`robfig/cron`
|
||||
syntax), per-job timezone, enable/disable, manual trigger, and execution
|
||||
history with captured stdout/stderr.
|
||||
|
||||
- The scheduler ticks every minute and runs due jobs via `docker exec`
|
||||
(`sh -c <command>`) inside the service's container.
|
||||
- Executions are retained per-job (`retention` count) and visible in the UI.
|
||||
- Requires a running container — a stopped service's due job records a
|
||||
failure, it does not queue.
|
||||
|
||||
```
|
||||
GET/POST /api/v1/cron-jobs PUT/DELETE /api/v1/cron-jobs/:id
|
||||
POST /api/v1/cron-jobs/:id/trigger
|
||||
GET /api/v1/cron-jobs/:id/executions
|
||||
```
|
||||
|
||||
## Exec console
|
||||
|
||||
Service detail → **Console**. One-off `sh -c` commands against the running
|
||||
container: 30s ceiling, 64KB output cap, exit code + captured output shown
|
||||
per run. Debugging aid — not an interactive shell.
|
||||
|
||||
```
|
||||
POST /api/v1/services/:id/exec {command}
|
||||
```
|
||||
|
||||
## Managed databases
|
||||
|
||||
**/databases** page — provision PostgreSQL/Redis/MySQL/MariaDB/MongoDB/
|
||||
ClickHouse/Dragonfly as managed containers on the `containr` network.
|
||||
|
||||
- Start/stop/restart actions; connection URL with copy.
|
||||
- **Backups**: manual snapshot now, or set a cron expression
|
||||
(`backup_schedule`) — due backups run inside the same scheduler tick as
|
||||
cron jobs. Archives live in the `containr-db-backups` volume and are
|
||||
downloadable (`GET /databases/:id/backups/:bid/download`) and restorable.
|
||||
- **Bind to service**: injects `connection_url` into a service's variables
|
||||
as a secret (`DATABASE_URL` by default). Redeploy the service afterward —
|
||||
variables are read at deploy time.
|
||||
|
||||
## Preview environments
|
||||
|
||||
Service detail → **Previews**. Create a preview per branch/PR with a TTL,
|
||||
promote it to production, or delete it.
|
||||
|
||||
Current limitation: previews are bookkeeping records — they do not build,
|
||||
deploy, or route a real container. The generated `*.preview.containr.local`
|
||||
URL is a placeholder. Real preview runtime is tracked on the roadmap.
|
||||
|
||||
## Security scans
|
||||
|
||||
**/security** page — pick a project, run a scan (dependency / configuration
|
||||
/ comprehensive, optionally scoped to one service).
|
||||
|
||||
The scanner is heuristic config analysis, not Trivy/Grype: it flags
|
||||
unpinned image tags, `http://` source/public URLs, `npm install` without
|
||||
`npm ci`, `pip install` without hashes, `curl|sh` build steps, missing
|
||||
health checks, and similar smells. Findings get severity badges and can be
|
||||
resolved or ignored; the page also shows a security score, open/resolved
|
||||
counts, scan history, and compliance status.
|
||||
|
||||
```
|
||||
POST /api/v1/security/scans {project_id, service_id?, scan_type}
|
||||
GET /api/v1/projects/:id/vulnerabilities
|
||||
PUT /api/v1/vulnerabilities/:id {status: open|resolved|ignored}
|
||||
GET /api/v1/projects/:id/security/metrics
|
||||
GET /api/v1/projects/:id/security/history
|
||||
```
|
||||
|
||||
## High availability
|
||||
|
||||
**/ha** page — manager enable/disable, node/health-check/alert status
|
||||
cards, manual failover (with confirmation), per-service failover policies
|
||||
(project → service picker, strategy, min healthy nodes, max failures),
|
||||
active alerts with resolve, and health check results.
|
||||
|
||||
HA state is in-memory like autoscaling — policies and alerts reset on
|
||||
backend restart.
|
||||
|
||||
## Node agents
|
||||
|
||||
**/usage** page → *Connect node*: issue a per-node token (`cagt_…`, shown
|
||||
once), copy the prefilled install command, revoke tokens when done. Agents
|
||||
register over `POST /api/agents/register` and heartbeat via
|
||||
`POST /api/agents/heartbeat`.
|
||||
Reference in new issue
Block a user