feat(scheduling): multi-node service placement + remote lifecycle (#54)

* feat(scheduling): multi-node service placement + remote container lifecycle

services.node_id pins a service to a node agent: explicit id, "auto"
(least-loaded online node), or "local"/empty for the local Docker host.

- DeploymentEngine gains a NodeRunner seam; ReconcileService routes
  pinned services to the agent's command queue (create_container with
  image/env/command/ports/volumes/labels/resource limits), tracked in
  container_instances with deterministic per-replica ids.
- Stop/start/restart/delete fan out to the pinned node — no more leaked
  remote containers on service delete.
- Agent create_container is idempotent (rm -f first) and honors restart
  policy, labels, memory/cpus, published+ephemeral ports, volumes, env.
- GET /nodes/options exposes id/name/status to authed users; full agent
  management stays admin-only.
- CLI services create/update --node, MCP node_id docs, OpenAPI
  node_id/node_name + /nodes/options, service-detail Placement section.

Ceilings: remote nodes run registry-pulled images only — git builds,
Traefik domains, and remote sleep detection stay local until
build-on-node and per-node ingress land.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(scheduling): node cordon/drain + schedulable gate

- node_agents.schedulable column; "auto" placement and explicit pins skip
  cordoned nodes; already-pinned services keep running (reconcile is
  unaffected — cordon is not eviction).
- POST /agents/:id/cordon|uncordon|drain (admin). Drain cordons, removes
  every service container via the agent queue, and clears the services'
  pins so they redeploy on the local host. Requires the node online —
  an offline node can't acknowledge removals.
- CLI nodes cordon|uncordon|drain (drain confirm-gated), MCP
  containr_nodes_cordon|uncordon|drain, OpenAPI paths + schedulable
  field, node-detail page cordon badge + drain/cordon controls.
- Fix: re-registering an agent no longer resets auto_prune.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(scheduling): remote registry pulls + post-run health check

- Dispatch payload carries the project owner's registry credentials for
  the image host; the agent docker-logins for the pull and logs out
  after, so private images deploy to remote nodes.
- Completed command payloads get their registry password scrubbed —
  secrets don't linger in agent_commands rows.
- Agent verifies the container is still running ~1.5s after docker run;
  crash-loops now fail the deployment with the container's last log
  lines instead of reporting "running" optimistically.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
Tomáš DvořákandDevin authored and GitHub committed 2026-10-09 16:19:12 +02:00
1 parent 401daf69aa
commit bc93c83cf9
28 files changed
+1565 -48

No files matched your search

+27
View File
@@ -84,9 +84,22 @@ export type ServiceEntity = {
staticDir?: string;
sleepEnabled?: boolean;
sleepIdleMinutes?: number;
nodeId?: string;
nodeName?: string;
publicUrl?: string;
};
export type NodeOption = {
id: string;
name: string;
status: string;
};
export async function listNodeOptions(): Promise<NodeOption[]> {
const payload = await requestJson<{ nodes?: NodeOption[] }>('/nodes/options');
return payload.nodes ?? [];
}
export type ServiceDomain = {
id: string;
service_id: string;
@@ -343,6 +356,7 @@ export type NodeAgentEntity = {
capabilities: AgentCapabilities;
resources: NodeResources;
autoPrune: boolean;
schedulable: boolean;
metadata: Record<string, unknown>;
lastHeartbeat?: string;
createdAt?: string;
@@ -680,6 +694,8 @@ function normalizeService(service: RawService): ServiceEntity | null {
staticDir: service.static_dir,
sleepEnabled: service.sleep_enabled,
sleepIdleMinutes: service.sleep_idle_minutes,
nodeId: service.node_id,
nodeName: service.node_name,
publicUrl: service.public_url,
};
}
@@ -1081,6 +1097,7 @@ function normalizeAgent(agent: RawNodeAgent): NodeAgentEntity | null {
},
lastHeartbeat: agent.last_heartbeat,
autoPrune: agent.auto_prune ?? false,
schedulable: agent.schedulable ?? true,
metadata: (agent.metadata ?? {}) as Record<string, unknown>,
createdAt: agent.created_at,
updatedAt: agent.updated_at,
@@ -1135,6 +1152,16 @@ export async function pruneAgent(id: string, body?: { until?: string; volumes?:
});
}
export async function cordonAgent(id: string, cordoned: boolean): Promise<void> {
await requestJson(`/agents/${id}/${cordoned ? 'cordon' : 'uncordon'}`, { method: 'POST' });
}
export type DrainResult = { services_drained: number; services_unpinned: number };
export async function drainAgent(id: string): Promise<DrainResult> {
return requestJson<DrainResult>(`/agents/${id}/drain`, { method: 'POST' });
}
export async function getAgent(id: string): Promise<NodeAgentEntity | null> {
const payload = await requestJson<{ agent?: RawNodeAgent }>(`/agents/${id}`);
return payload.agent ? normalizeAgent(payload.agent) : null;