feat(scheduling): multi-node service placement + remote lifecycle (#54)

* feat(scheduling): multi-node service placement + remote container lifecycle

services.node_id pins a service to a node agent: explicit id, "auto"
(least-loaded online node), or "local"/empty for the local Docker host.

- DeploymentEngine gains a NodeRunner seam; ReconcileService routes
  pinned services to the agent's command queue (create_container with
  image/env/command/ports/volumes/labels/resource limits), tracked in
  container_instances with deterministic per-replica ids.
- Stop/start/restart/delete fan out to the pinned node — no more leaked
  remote containers on service delete.
- Agent create_container is idempotent (rm -f first) and honors restart
  policy, labels, memory/cpus, published+ephemeral ports, volumes, env.
- GET /nodes/options exposes id/name/status to authed users; full agent
  management stays admin-only.
- CLI services create/update --node, MCP node_id docs, OpenAPI
  node_id/node_name + /nodes/options, service-detail Placement section.

Ceilings: remote nodes run registry-pulled images only — git builds,
Traefik domains, and remote sleep detection stay local until
build-on-node and per-node ingress land.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(scheduling): node cordon/drain + schedulable gate

- node_agents.schedulable column; "auto" placement and explicit pins skip
  cordoned nodes; already-pinned services keep running (reconcile is
  unaffected — cordon is not eviction).
- POST /agents/:id/cordon|uncordon|drain (admin). Drain cordons, removes
  every service container via the agent queue, and clears the services'
  pins so they redeploy on the local host. Requires the node online —
  an offline node can't acknowledge removals.
- CLI nodes cordon|uncordon|drain (drain confirm-gated), MCP
  containr_nodes_cordon|uncordon|drain, OpenAPI paths + schedulable
  field, node-detail page cordon badge + drain/cordon controls.
- Fix: re-registering an agent no longer resets auto_prune.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(scheduling): remote registry pulls + post-run health check

- Dispatch payload carries the project owner's registry credentials for
  the image host; the agent docker-logins for the pull and logs out
  after, so private images deploy to remote nodes.
- Completed command payloads get their registry password scrubbed —
  secrets don't linger in agent_commands rows.
- Agent verifies the container is still running ~1.5s after docker run;
  crash-loops now fail the deployment with the container's last log
  lines instead of reporting "running" optimistically.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

ROADMAP2.md

@@ -277,12 +277,31 @@ and wired refs, one click or one CLI call.
   `nodes commands`, MCP `containr_nodes_update|prune|commands`
   (confirm-gated), Usage-page per-node Prune button + auto toggle.
   `cmd/agent/main.go`, `notification_producers.go:scheduleAutoPrune`.
-- [ ] **Multi-node scheduling** — wire `scheduling_rules` (table exists):
-  pin/spread/affinity, prefer-lowest-usage; deploy engine routes to
-  `services.node_id` via agent commands (agent already runs containers).
-- [ ] **Build-on-node** — agents currently `docker run` prebuilt images
-  only; extend agent to pull registry images (build stays central) —
-  private registry creds flow through Phase C.
+- [🟡] **Multi-node scheduling** — phase 1 shipped: `services.node_id`
+  pin (explicit / `auto` = least-loaded online / `local` = clear),
+  `NodeRunner` seam in the deploy engine routes reconcile to the agent's
+  command queue (`create_container` with image/env/cmd/ports/volumes/
+  labels/limits), `container_instances` inventory per (agent, service,
+  replica), remote stop/start/restart/delete fan-out, `GET
+  /nodes/options`, `--node` on services create/update + MCP + OpenAPI +
+  service-detail Placement section. Drain/cordon shipped:
+  `node_agents.schedulable`, `POST /agents/:id/cordon|uncordon|drain`
+  (drain evicts remote containers via the agent queue + unpins services
+  so they redeploy locally), `nodes cordon|uncordon|drain` CLI +
+  MCP (drain confirm-gated) + node-detail page controls; `auto` and
+  explicit pins skip cordoned nodes. Remote nodes run registry-pulled
+  images only — git builds, Traefik domains, and remote sleep are
+  documented ceilings. Remaining: `scheduling_rules` spread/affinity
+  policies, deployment status reported back from agent
+  results, per-node resource-aware `auto` (currently count-based).
+- [🟡] **Build-on-node** — private registry pulls shipped: the dispatch
+  payload carries the project owner's `registries` credentials for the
+  image host; the agent `docker login`s for the pull and logs out after,
+  and command payloads are scrubbed of passwords on completion. Agent
+  also verifies the container is still running ~1.5s after `run` so
+  crash-loops fail the deploy instead of reporting healthy. Remaining:
+  real builds on the node (source checkout + build daemon) — pulls
+  already work.
 - [ ] **IaC provisioning (ROADMAP §2.2, unchanged)** — Terraform/
   OpenTofu, `infra_connections`, Proxmox + AWS modules, + Hetzner module
   (dflow's second provider), cloud-init → auto-enroll.