Tranche 5 — all 24 raw sites in runtime.go. Adds runtime.sql with the
legacy-column COALESCE loader (type/image/command/git_repo fall back to
service_type/image_name/start_command/source_url), env-var reads,
sibling refs, deployed-image concat, and conditional published-port /
replica writes. Status transitions reuse SetServiceStatus.
loadReadableService now shares GetServiceRuntimeWithOwner — the projects
join is safe (FK) and drops a duplicated 24-column select.
TestRuntimeLiveInvariants verifies the COALESCE chain against a service
seeded with only legacy columns, plus refs, port, variables, agent
listing, image concat, and conditional updates on live Postgres.
Generated with [Devin](https://devin.ai)
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
node_agents.default_domain gives each agent a base domain. Services
placed on it (explicit pin or spread) with no explicit service_domains
get <name>.<default_domain> — resolved in serviceDomainNames so edge
routing, wake placeholders, and public_url all see the same names.
Spread services union every eligible node's base domain; explicit
domains always win.
Surfaces: PUT /agents/:id {default_domain} (hostname-validated), CLI
nodes update --domain, MCP arg docs, OpenAPI + regenerated types,
node-detail inline editor.
Also marks the already-shipped backup-import roadmap item.
Generated with [Devin](https://devin.ai)
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
services.traefik_labels (jsonb) holds operator-defined Traefik
middlewares keyed 'middlewares.<name>.<type>[.<field>]', validated
against an allowlist at write time. edgeLabels (extracted from
createReplica) scopes each definition to the service router as
<router>-<name> and auto-attaches it to the router's middleware chain
after builtins. Router/service/provider-level keys are rejected.
Surfaces: create/update API fields, services create/update
--traefik-label (repeatable, key=value), MCP arg docs, OpenAPI +
regenerated types, service-page Access editor (Routing overrides
card + textarea).
Also marks three stale ROADMAP2 items shipped: notification producers
round-out, build-on-node (PR #60), mobile env/domains/scaling (PR #61).
Generated with [Devin](https://devin.ai)
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Operator-set node labels (node_agents.tags) gate which nodes a service
may schedule onto: services.placement_tags requires every listed tag.
auto and spread filter eligible agents by tags; tags are re-evaluated
each reconcile so retagging a node re-spreads replicas. Pin + tags and
local + tags rejected; empty array clears the requirement.
Surfaces: --placement-tags on services create/update, nodes update
--tags, MCP args, OpenAPI, node-detail tag editor, service-detail
placement-tags input.
Generated with [Devin](https://devin.ai)
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
services.spread distributes replicas across every online, schedulable
agent (replica i → agents[i mod n], deterministic). auto placement now
orders by real headroom — memory utilisation, cpu usage, then container
count — instead of count alone.
Lifecycle (start/stop/restart/delete) and GET /runtime are now driven
by container_instances inventory, so remote replicas are visible and
tear down wherever they run — covers spread, pins, and stale pin
leftovers. Spread and explicit pins are mutually exclusive.
Generated with [Devin](https://devin.ai)
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(scheduling): multi-node service placement + remote container lifecycle
services.node_id pins a service to a node agent: explicit id, "auto"
(least-loaded online node), or "local"/empty for the local Docker host.
- DeploymentEngine gains a NodeRunner seam; ReconcileService routes
pinned services to the agent's command queue (create_container with
image/env/command/ports/volumes/labels/resource limits), tracked in
container_instances with deterministic per-replica ids.
- Stop/start/restart/delete fan out to the pinned node — no more leaked
remote containers on service delete.
- Agent create_container is idempotent (rm -f first) and honors restart
policy, labels, memory/cpus, published+ephemeral ports, volumes, env.
- GET /nodes/options exposes id/name/status to authed users; full agent
management stays admin-only.
- CLI services create/update --node, MCP node_id docs, OpenAPI
node_id/node_name + /nodes/options, service-detail Placement section.
Ceilings: remote nodes run registry-pulled images only — git builds,
Traefik domains, and remote sleep detection stay local until
build-on-node and per-node ingress land.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(scheduling): node cordon/drain + schedulable gate
- node_agents.schedulable column; "auto" placement and explicit pins skip
cordoned nodes; already-pinned services keep running (reconcile is
unaffected — cordon is not eviction).
- POST /agents/:id/cordon|uncordon|drain (admin). Drain cordons, removes
every service container via the agent queue, and clears the services'
pins so they redeploy on the local host. Requires the node online —
an offline node can't acknowledge removals.
- CLI nodes cordon|uncordon|drain (drain confirm-gated), MCP
containr_nodes_cordon|uncordon|drain, OpenAPI paths + schedulable
field, node-detail page cordon badge + drain/cordon controls.
- Fix: re-registering an agent no longer resets auto_prune.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(scheduling): remote registry pulls + post-run health check
- Dispatch payload carries the project owner's registry credentials for
the image host; the agent docker-logins for the pull and logs out
after, so private images deploy to remote nodes.
- Completed command payloads get their registry password scrubbed —
secrets don't linger in agent_commands rows.
- Agent verifies the container is still running ~1.5s after docker run;
crash-loops now fail the deployment with the container's last log
lines instead of reporting "running" optimistically.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
GET /services/:id/env-check (read scope) reports, without exposing values:
- unresolved: ${{service.KEY}} refs that don't match a sibling service/env
- empty: variables with blank values
- unreadable: secrets whose ciphertext fails decryption (key rotation)
install.sh now generates SECRETS_KEY explicitly (was falling back to
JWT_SECRET — a rotation would orphan ciphertext). .env.example documents it.
Surfaces: CLI `services env-check`, MCP containr_services_env_check
(90 tools), OpenAPI path, and a warning banner in the service Variables
panel listing each issue.
Verified live: injected a bogus ${{nosuchservice.PORT}} var plus an
unreadable ciphertext row; the endpoint flagged both, ok=false.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
services.builder picks the build strategy per service: auto detects from
repo contents (Dockerfile → railpack → nixpacks), explicit overrides force
a builder, and static builds assets in a node stage then serves them from
an nginx image (static_build_cmd/static_dir, port 80 default).
- cpu_reserve/memory_reserve → Docker CPU shares + MemoryReservation;
cpu/memory stay hard caps (NanoCPUs/Memory). Empty string clears.
- app_settings.default_cpu/default_memory override service-create
fallbacks; capacity_policy=block fails deploys whose requested memory
exceeds node capacity (default warn logs via slog).
- Resource specs are validated — "512mb" fails instead of silently
dropping the limit.
- Fixes: deployments.go hardcoded nixpacks (auto-detect was dead code);
DockerfileBuilder passed the Dockerfile *path* as build-context content;
BuildImage never drained the daemon stream so builds could abort and
errors went unseen — stream is now consumed and errorDetail surfaced.
- CLI: --builder/--cpu-reserve/--memory-reserve/--static-cmd/--static-dir
on create+update; MCP args on services create/update; UI Build &
Resources card; OpenAPI + llms.txt + CLI.md.
Verified live: static build of a real repo → nginx image → deployed;
reserves reach HostConfig (shares=256, MemoryReservation=128Mi);
validation rejects bad specs; builder persists + clears.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
service_domains table holds the full hostname set; services.domain stays
the derived default and is synced on every domain write. Runtime spec
carries all domains into a multi-Host Traefik rule plus access gates.
- GET|POST /services/:id/domains, DELETE + POST /:domain_id/default
- GET /services/:id/domains/check — DNS preflight vs public_ip/base_url
(ok | wrong-target | pending), stamps last_checked_at
- maintenance_mode → Traefik redirectregex to /api/v1/maintenance
(base_url/BASE_URL setting; falls back to noop@internal offline)
- basic_auth [{username,password}] → bcrypt htpasswd on the label;
responses return usernames only
- First domain auto-defaults; domain="" via PUT clears all domains
- CLI: services domains list|add|remove|default|check, update
--maintenance/--basic-auth; MCP: 5 domain tools + update args
- Settings UI: Domains card (CRUD + DNS check badges) + Access card
- Fixes latent C1 bug: PUT without volumes 500'd — nil []byte sent as
bytea, not NULL; COALESCE needs untyped nil + ::jsonb cast
Verified live: domain CRUD + default sync, pending/wrong-target checks,
bcrypt htpasswd and both middlewares on the deployed container's labels.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
services.volumes JSONB [{type: volume|bind, source, target, read_only}]
mapped into every replica's container config at deploy and reconcile
time, hashed into the runtime spec so mount changes trigger
reconciliation. Mounted via existing docker.HostConfig.Mounts.
- RuntimeSpec.Volumes + specHash + mount.Mount mapping in createReplica
- Create/update service accepts volumes; PUT replaces the whole list
- Replica-count reconcile now goes through the deploy queue — the last
raw `go` reconcile left that could race a build
- GET /admin/volumes inventories node volumes with in_use flags;
DELETE /admin/volumes/:name removes unused ones (409 on in-use)
- CLI: services update --volume/--bind/--clear-volumes, volumes
list|delete; MCP: services_update + volumes_list/delete
- Settings UI: Volumes section with mount editor
- Migration normalizes the nullable baseline column to NOT NULL [].
Verified live: deploy applies mounts to the container, docker
auto-creates named volumes, in-use deletion is refused.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
- Webhook-triggered deploys now go through the per-service deploy queue
and default to no_cache builds (dflow convention — prevents stale
layer surprises on push).
- Boot sweep marks deployments interrupted by restart as failed instead
of leaving phantom queued/building rows.
- is_secret environment variables are AES-GCM-encrypted at rest
(enc:v1: prefix); key from SECRETS_KEY or JWT_SECRET fallback.
Existing plaintext secrets encrypt on boot; deploy/exec paths decrypt
transparently. Masked-echo writes preserve stored ciphertext.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
- internal/deployqueue: serialized FIFO per service — concurrent deploys
of one service no longer race (the container-name-conflict failure
mode); different services still run in parallel.
- Deploys and rollbacks enqueue through the queue; a busy service
returns status 'queued' and promotes when the active job finishes.
- POST /deployments/:id/cancel — queued jobs drop immediately
(200 cancelled); active jobs get ctx cancelled so build/reconcile
abort (202 cancelling, settles 'cancelled'); finished deployments
return 409 NOT_CANCELLABLE.
- Redeploy and variables save now route through the same queue so they
can't overlap a build; PUT /services/:id/variables accepts
redeploy:true to apply env changes to running containers.
- no_cache on deployment create plumbs BuildRequest -> BuildOptions for
nixpacks, dockerfile and railpack builders.
- CLI: deployments cancel, deploy --no-cache, variables set/unset
--redeploy. MCP: containr_deployments_cancel; deploy/variables_set
document no_cache/redeploy. UI: Cancel button + queued/cancelled
badges on the deployments table. OpenAPI + types regenerated.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: e2e QA hardening — honest runtime URLs, health, and read-only anon UI
Backend:
- managed DB provisioning: attach DB containers to the platform network and
probe the container IP, fixing the 127.0.0.1 reachability failure inside
the backend container; log provisioning failures
- stop fabricating connection URLs when provisioning produced none
- template deploys now carry port + health_check into service runtime fields
- service health probed over the project network (backend joins it) instead
of an in-container wget that fails on shell-less images
- runtime/public/connection URLs rewrite loopback to the request Host so
links are usable for LAN/remote clients
Frontend:
- service Networking panel shows real runtime URLs/ports/health instead of
hardcoded containr.dev + 8080->443
- anonymous views hide all mutation controls (new project, add service,
group, context menu)
- project cards: remove decorative svcN labels; badge says DOWN only when
zero services run, IDLE when empty, DEGRADED for partial
- mobile projects header no longer clips (fluid search width)
- unknown routes render a not-found page instead of silently redirecting
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: telemetry scope label, chunk split, remove stale vite artifacts
- /system/host now reports scope=host|docker-desktop-vm and the daemon OS;
Projects dashboard labels cards VM.* when the daemon is a Desktop VM so
memory/core numbers aren't mistaken for physical host values
- manualChunks: split @xyflow/react and recharts out of the main bundle;
removes the >1MB chunk warning
- drop tracked vite.config.js/.d.ts build artifacts that shadowed the real
vite.config.ts
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: persist service host port; demo-data scope for telemetry
Public URLs were ephemeral: every restart/redeploy rebound a random host
port, so endpoint links rotted. Persist published_port on the service and
prefer it for replica 0 on create. If the port is taken, fall back to
ephemeral and repersist the new assignment. Post-start the binding is
verified — Docker Desktop accepts a conflicting publish silently, which
previously left services running with no endpoint at all.
demo-data.ts gains the required scope field for HostMonitoring (CI fix).
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Backend:
- tolerate concurrent project-network creation (re-resolve on conflict)
- always publish an ephemeral host port when a port is configured, even
with a domain set — Traefik routes on top when the edge network exists
- public_url now prefers the configured domain, falls back to the first
published port
- service logs resolve replicas by containr.service label and merge all
replica streams — previously queried a nonexistent containr-<id> name
Canvas/detail:
- degraded status renders amber end-to-end (node card, detail badge,
status dot) instead of falling through to "Stopped"
- edges pick handle sides by relative node position — no more backward
loops when the source sits right of the target
- context menu gains Start for stopped services, Visit uses the
published URL when no domain is set, menu clamps to canvas bounds
- node cards show the image tag instead of a redundant hint
- scaling tab falls back to live runtime replica counts when no
autoscaler state exists; manual scale no longer requires it
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Services now reconcile into actual Docker containers instead of
bookkeeping rows:
- Per-project bridge network with service-name DNS aliases and
${{service.KEY}} env expansion (HOST/PORT builtins)
- Replicas, published ports, Traefik domain routing, health checks,
restart policies persisted on the service spec; spec-hash drift
recreates containers on redeploy
- Lifecycle API: start/stop/restart/redeploy, live runtime state,
real image rollback (replaces the fake sleep)
- Manual scale and PUT replicas reconcile live containers; service
and project delete remove their containers and the project network
- Canvas: real URLs, replica badges, env filter, right-click
deploy/restart/stop/delete menu, empty state
- Service detail: real lifecycle buttons, live replica/URL strip,
Networking editor, ${{ }} variable helper
- Fix prebuilt-image pulls racing the tag (drain the pull stream)
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
