feat(scheduling): replica spread + resource-aware auto placement (#56)

services.spread distributes replicas across every online, schedulable
agent (replica i → agents[i mod n], deterministic). auto placement now
orders by real headroom — memory utilisation, cpu usage, then container
count — instead of count alone.

Lifecycle (start/stop/restart/delete) and GET /runtime are now driven
by container_instances inventory, so remote replicas are visible and
tear down wherever they run — covers spread, pins, and stale pin
leftovers. Spread and explicit pins are mutually exclusive.

Generated with [Devin](https://devin.ai)

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

app/frontend/src/features/service/pages/ServiceDetailPage.tsx

@@ -2404,10 +2404,12 @@ export function ServiceDetailPage() {
                 </div>
                 <div className="flex items-center gap-3 text-xs">
                   <select
-                    value={nodeDraft ?? (service.nodeId || 'local')}
+                    value={service.spread ? '__spread__' : (nodeDraft ?? (service.nodeId || 'local'))}
+                    disabled={service.spread}
                     onChange={(e) => setNodeDraft(e.target.value)}
-                    className="px-2 py-1.5 rounded-[var(--radius-md)] bg-[var(--surface-muted)] border border-[var(--border-subtle)] text-sm text-[var(--text-primary)]"
+                    className="px-2 py-1.5 rounded-[var(--radius-md)] bg-[var(--surface-muted)] border border-[var(--border-subtle)] text-sm text-[var(--text-primary)] disabled:opacity-50"
                   >
+                    {service.spread && <option value="__spread__">All nodes (spread)</option>}
                     <option value="local">This server (local Docker)</option>
                     <option value="auto">Auto — least-loaded online node</option>
                     {(nodesQuery.data ?? []).map((n) => (
@@ -2416,6 +2418,17 @@ export function ServiceDetailPage() {
                       </option>
                     ))}
                   </select>
+                  <label className="flex items-center gap-1.5 text-[var(--text-secondary)]">
+                    <input
+                      type="checkbox"
+                      checked={service.spread ?? false}
+                      disabled={updateServiceMutation.isPending || (!!service.nodeId && !service.spread)}
+                      title={service.nodeId && !service.spread ? 'Clear the node pin first (choose "This server")' : undefined}
+                      onChange={(e) => updateServiceMutation.mutate({ spread: e.target.checked })}
+                      className="accent-[var(--accent-primary)]"
+                    />
+                    spread replicas across nodes
+                  </label>
                   {nodeDraft !== null && (
                     <>
                       <button

app/frontend/src/generated/api-types.ts

@@ -10878,6 +10878,8 @@ export interface components {
             node_id?: string;
             /** @description Display name of the pinned node agent (read-only) */
             node_name?: string;
+            /** @description Replicas are distributed across every online, schedulable node agent (replica i → agents[i mod n]). Mutually exclusive with an explicit node_id pin. */
+            spread?: boolean;
             /** @description Computed public URL (published port or domain) */
             public_url?: string;
             /**
@@ -10950,6 +10952,8 @@ export interface components {
             static_dir?: string;
             /** @description Node agent id to pin to, 'auto' = least-loaded online agent; omit/empty = local host */
             node_id?: string;
+            /** @description Distribute replicas across every online, schedulable node agent; mutually exclusive with an explicit node_id pin */
+            spread?: boolean;
         };
         UpdateServiceRequest: {
             /** @description Service name */
@@ -11013,6 +11017,8 @@ export interface components {
             sleep_idle_minutes?: number;
             /** @description Node agent id to pin to, 'auto' = least-loaded online agent, 'local' = run on the Containr host (clears the pin) */
             node_id?: string;
+            /** @description Distribute replicas across every online, schedulable node agent; mutually exclusive with an explicit node_id pin — send 'local' first to clear a pin */
+            spread?: boolean;
         };
         ServiceDomain: {
             /** Format: uuid */

app/frontend/src/generated/docs-snapshot.ts

@@ -40,7 +40,7 @@ export const bundledDocs: BundledDoc[] = [
     "path": "guides/CLI.md",
     "section": "guides",
     "title": "Containr CLI",
-    "markdown": "# Containr CLI\n\nThe CLI controls every part of a Containr instance — the same API surface\nthe web UI uses. Build it with `scripts/build-cli.sh` or\n`go build ./cmd/cli` from `app/backend`.\n\n## Authentication\n\nCreate a personal access token in the web UI under\n**Settings → Personal Access Tokens** (or `containr tokens create`), then:\n\n```bash\ncontainr auth login cnp_...                      # verify + store\ncontainr auth login cnp_... --url https://containr.example.com\ncontainr auth login cnp_... --name production    # named profile\ncontainr auth status                             # show active profile + identity\ncontainr auth logout\n```\n\nResolution order for every command: `--token`/`--api-url` flags >\n`CONTAINR_TOKEN`/`CONTAINR_API_URL` env > active profile >\n`http://localhost:8080/api/v1`. The active profile is `--profile` >\n`CONTAINR_PROFILE` > `current_profile` in `~/.containr.yaml`.\n\n## Output and scripting\n\n- `--json` prints the raw API response — safe to pipe and parse.\n- `--yes` / `-y` skips confirmation on destructive commands (delete,\n  revoke, restore, failover). Non-interactive shells refuse destructive\n  commands without it.\n- Exit codes are stable: `0` ok, `1` error, `2` auth/permission,\n  `3` not found, `4` validation, `5` unreachable.\n\n## Commands\n\n```\nauth login|logout|status         token auth and profiles\ntokens list|create|revoke        personal access tokens\nprojects list|get|create|delete\nservices list|get|create|update|delete|start|stop|restart|redeploy\nservices clone|move              duplicate config or re-parent a service\nservices domains list|add|remove|default|check   multi-domain + DNS preflight\nregistries list|add|update|remove  private-image pull credentials\nvolumes list|delete              docker volume inventory + cleanup (admin)\ndeploy <service-id>              trigger a deployment (--no-cache, --commit-hash, --branch)\ndeployments list|get|logs|rollback|cancel\nlogs <service-id> [--follow]     runtime logs (SSE follow)\nexec <service-id> -- <cmd>       one-off container command\nvariables list|set|unset         env vars (secrets masked; --redeploy applies them)\nvariables shared list|set|unset  project shared vars — referenced as ${{shared.KEY}}\ndatabases list|get|create|update|delete|action|backup|restore|download-backup|import-backup\n                               import-backup <id> <file.tar.gz|-> uploads an archive as a restore\n                               point; ships offsite too when a backup target is assigned\ndatabases list|get|create|update|delete|action|backup|restore|download-backup\n                               create --public exposes the port on all interfaces;\n                               update <id> --public/--public=false toggles it (container recreates, data persists)\ndatabases register <name>      register an external database (--type --host --port --database --username --password --ssl; probed before storing)\ndatabases test-connection      probe a connection — by id, or ad-hoc flags\nbackup-targets list|add|update|remove|test\n                               S3-compatible archive destinations; add/update probe the bucket\n                               first, empty --access-key/--secret-key on update keep stored keys;\n                               update <db-id> --backup-target assigns a target (\"\" clears it)\ncron list|get|create|delete|trigger|executions\ntemplates list|get|create|delete|deploy|plan|import-compose|deploy-graph\n                               deploy --name --var K=V; plan dry-resolves expressions;\n                               import-compose <compose.yml|-> converts a compose file into\n                               a v2 graph config; deploy-graph <project-id> <config.json|->\n                               deploys an ad-hoc service graph\nimport compose <project-id> <compose.yml|->\n                               deploys a docker-compose file straight into a project —\n                               no template row. DB images become managed databases\n                               attached under their service name; --var K=V overrides\n                               template variables\nnodes list|get|delete|tokens     node agents + onboarding tokens (admin)\nnodes add <user@host>            SSH bootstrap — one session installs the agent, then never needed again (--print for manual runs)\nnodes update|prune|commands      rename/auto-prune, bounded docker prune, command history\nmigrate railway                  pull a Railway project (services, vars, domains) via GraphQL and recreate it — --dry-run prints the plan\nmigrate dflow                    import a dflow project (services, vars, volumes, domains) via its Payload REST API — needs --url + --api-key, --dry-run prints the plan\nscaling status|policies|services|scale\nha status|alerts|health|policies|enable|disable|failover\nsecurity scan|vulnerabilities|metrics\ngateway services|keys|anLine truncated
+    "markdown": "# Containr CLI\n\nThe CLI controls every part of a Containr instance — the same API surface\nthe web UI uses. Build it with `scripts/build-cli.sh` or\n`go build ./cmd/cli` from `app/backend`.\n\n## Authentication\n\nCreate a personal access token in the web UI under\n**Settings → Personal Access Tokens** (or `containr tokens create`), then:\n\n```bash\ncontainr auth login cnp_...                      # verify + store\ncontainr auth login cnp_... --url https://containr.example.com\ncontainr auth login cnp_... --name production    # named profile\ncontainr auth status                             # show active profile + identity\ncontainr auth logout\n```\n\nResolution order for every command: `--token`/`--api-url` flags >\n`CONTAINR_TOKEN`/`CONTAINR_API_URL` env > active profile >\n`http://localhost:8080/api/v1`. The active profile is `--profile` >\n`CONTAINR_PROFILE` > `current_profile` in `~/.containr.yaml`.\n\n## Output and scripting\n\n- `--json` prints the raw API response — safe to pipe and parse.\n- `--yes` / `-y` skips confirmation on destructive commands (delete,\n  revoke, restore, failover). Non-interactive shells refuse destructive\n  commands without it.\n- Exit codes are stable: `0` ok, `1` error, `2` auth/permission,\n  `3` not found, `4` validation, `5` unreachable.\n\n## Commands\n\n```\nauth login|logout|status         token auth and profiles\ntokens list|create|revoke        personal access tokens\nprojects list|get|create|delete\nservices list|get|create|update|delete|start|stop|restart|redeploy\nservices clone|move              duplicate config or re-parent a service\nservices domains list|add|remove|default|check   multi-domain + DNS preflight\nregistries list|add|update|remove  private-image pull credentials\nvolumes list|delete              docker volume inventory + cleanup (admin)\ndeploy <service-id>              trigger a deployment (--no-cache, --commit-hash, --branch)\ndeployments list|get|logs|rollback|cancel\nlogs <service-id> [--follow]     runtime logs (SSE follow)\nexec <service-id> -- <cmd>       one-off container command\nvariables list|set|unset         env vars (secrets masked; --redeploy applies them)\nvariables shared list|set|unset  project shared vars — referenced as ${{shared.KEY}}\ndatabases list|get|create|update|delete|action|backup|restore|download-backup|import-backup\n                               import-backup <id> <file.tar.gz|-> uploads an archive as a restore\n                               point; ships offsite too when a backup target is assigned\ndatabases list|get|create|update|delete|action|backup|restore|download-backup\n                               create --public exposes the port on all interfaces;\n                               update <id> --public/--public=false toggles it (container recreates, data persists)\ndatabases register <name>      register an external database (--type --host --port --database --username --password --ssl; probed before storing)\ndatabases test-connection      probe a connection — by id, or ad-hoc flags\nbackup-targets list|add|update|remove|test\n                               S3-compatible archive destinations; add/update probe the bucket\n                               first, empty --access-key/--secret-key on update keep stored keys;\n                               update <db-id> --backup-target assigns a target (\"\" clears it)\ncron list|get|create|delete|trigger|executions\ntemplates list|get|create|delete|deploy|plan|import-compose|deploy-graph\n                               deploy --name --var K=V; plan dry-resolves expressions;\n                               import-compose <compose.yml|-> converts a compose file into\n                               a v2 graph config; deploy-graph <project-id> <config.json|->\n                               deploys an ad-hoc service graph\nimport compose <project-id> <compose.yml|->\n                               deploys a docker-compose file straight into a project —\n                               no template row. DB images become managed databases\n                               attached under their service name; --var K=V overrides\n                               template variables\nnodes list|get|delete|tokens     node agents + onboarding tokens (admin)\nnodes add <user@host>            SSH bootstrap — one session installs the agent, then never needed again (--print for manual runs)\nnodes update|prune|commands      rename/auto-prune, bounded docker prune, command history\nmigrate railway                  pull a Railway project (services, vars, domains) via GraphQL and recreate it — --dry-run prints the plan\nmigrate dflow                    import a dflow project (services, vars, volumes, domains) via its Payload REST API — needs --url + --api-key, --dry-run prints the plan\nscaling status|policies|services|scale\nha status|alerts|health|policies|enable|disable|failover\nsecurity scan|vulnerabilities|metrics\ngateway services|keys|anLine truncated
   },
   {
     "path": "guides/CLOUDFLARE_SETUP.md",

app/frontend/src/lib/api-client.ts

@@ -86,6 +86,7 @@ export type ServiceEntity = {
   sleepIdleMinutes?: number;
   nodeId?: string;
   nodeName?: string;
+  spread?: boolean;
   publicUrl?: string;
 };
 
@@ -696,6 +697,7 @@ function normalizeService(service: RawService): ServiceEntity | null {
     sleepIdleMinutes: service.sleep_idle_minutes,
     nodeId: service.node_id,
     nodeName: service.node_name,
+    spread: service.spread,
     publicUrl: service.public_url,
   };
 }