feat(databases): backup import — upload .tar.gz as a restore point

D5. POST /databases/:id/backups/import accepts a gzipped tar archive
(multipart `file` field or raw application/gzip body, 2 GiB ceiling),
stages it to a temp file, streams it into the backup volume via the
started holder container, and registers a completed backup row — a
first-class restore point that also ships offsite when a target is
assigned. External databases rejected.

Also fixes `containr databases restore` — it never sent database_id,
which the API requires, so every CLI restore failed validation.

CLI: databases import-backup <id> <file|-> (stdin support via new
client.DoRaw). UI: Import button in the backups row. OpenAPI + docs.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
Tomas DvorakandDevin committed 2026-10-08 23:07:27 +02:00
1 parent 109f79268f
commit 15045b8dd3
11 files changed
+409 -10

No files matched your search

+19 -7
View File
@@ -347,13 +347,25 @@ and wired refs, one click or one CLI call.
## 9b. Phase I — Companion surfaces
- [ ] **Phone app** — read-heavy mobile client over the existing API
(PAT auth already works). Scope v1: project/service list, status,
deploy trigger + log tail, restart/stop, notifications push via
existing notification rows (APNs/FCM relay optional, self-hosted
webhook → ntfy/Gotify as zero-infra alternative). React Native or
native-lean PWA; the API surface is already agent-shaped so the app
is a thin client. Design: dark UI, deploy status as first-class view.
- [ ] **Phone app** — fully-featured companion, not a read-only viewer.
The target: a user on the go can see a problem, understand it, and
fix it without a laptop. PAT + profile auth (multi-instance like the
CLI).
- **Dashboard** — project/service status at a glance, deploy
activity, node health, resource usage (cpu/mem from existing
metrics endpoints).
- **Operate** — deploy, redeploy, restart/stop/start, rollback to a
prior deployment, cancel a running deploy, scale replicas.
- **Observe** — live log tail (SSE), deployment step logs, backup
status, cron execution history.
- **Fix** — env var edit + apply-redeploy, domain add/remove,
maintenance-mode toggle, database backup/restore trigger.
- **Notifications** — push via existing notification rows; zero-infra
path = self-hosted ntfy/Gotify relay (no Firebase dependency),
FCM/APNs optional.
- **Stack** — React Native (Expo) thin client over the agent-shaped
API; dark UI mirroring the web design system; deploy status as the
first-class view. Ships with `app/` workspace in this repo.
- [ ] **Status/offline resilience** — app + CLI degrade gracefully when
the instance is unreachable (cached last-known state, clear banner).
+140
View File
@@ -8,7 +8,9 @@ import (
"errors"
"fmt"
"io"
"log"
"net/http"
"os"
"strings"
"time"
@@ -292,6 +294,144 @@ func (h *DatabaseHandler) DeleteBackupTarget(c *gin.Context) {
c.JSON(http.StatusOK, gin.H{"message": "Backup target deleted"})
}
// ImportBackup accepts a `.tar.gz` archive (multipart `file` field or raw
// body) and stores it as a restore point — the counterpart to offsite
// export, and the entry path for migrators.
func (h *DatabaseHandler) ImportBackup(c *gin.Context) {
userID, ok := requireAuthenticatedUserID(c)
if !ok {
return
}
databaseID := c.Param("id")
userID = h.effectiveUserID(c, userID, databaseID)
row, err := h.queries.GetDatabaseServiceByIDAndUser(c.Request.Context(), sqlcdb.GetDatabaseServiceByIDAndUserParams{
ID: databaseID, UserID: userID,
})
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": "Database not found", "code": "NOT_FOUND"})
return
}
if row.Provider == "external" {
c.JSON(http.StatusBadRequest, gin.H{"error": "backup import applies to managed databases — external restores are not supported yet", "code": "VALIDATION"})
return
}
// Buffer to a temp file — tar headers need the size up front, and a
// streamed upload can't be rewound for the archive write.
src, _, err := openUploadStream(c)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error(), "code": "VALIDATION"})
return
}
defer src.Close()
tmp, err := os.CreateTemp("", "containr-backup-import-*")
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to stage upload"})
return
}
defer func() { tmp.Close(); os.Remove(tmp.Name()) }()
const maxImport = 2 << 30 // 2 GiB ceiling for staged archives
size, err := io.Copy(tmp, http.MaxBytesReader(c.Writer, src, maxImport))
if err != nil {
c.JSON(http.StatusRequestEntityTooLarge, gin.H{"error": "archive too large or unreadable", "code": "VALIDATION"})
return
}
if _, err := tmp.Seek(0, io.SeekStart); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to read staged upload"})
return
}
magic := make([]byte, 2)
if _, err := tmp.Read(magic); err != nil || magic[0] != 0x1f || magic[1] != 0x8b {
c.JSON(http.StatusBadRequest, gin.H{"error": "archive must be gzip-compressed (.tar.gz)", "code": "VALIDATION"})
return
}
if _, err := tmp.Seek(0, io.SeekStart); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to read staged upload"})
return
}
backupID := generateDatabaseBackupID(databaseID)
archivePath := sanitizeBackupArchivePath(managedDatabaseBackupArchivePath(backupID))
if _, err := h.dockerClient.CreateVolume(c.Request.Context(), docker.VolumeConfig{Name: managedDatabaseBackupVolume}); err != nil && !strings.Contains(strings.ToLower(err.Error()), "already exists") {
c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to prepare backup volume"})
return
}
holderID, err := h.createBackupVolumeHolder(c.Request.Context())
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": fmt.Sprintf("Failed to open backup volume: %v", err)})
return
}
defer func() { _ = h.dockerClient.RemoveContainer(context.Background(), holderID, true) }()
pr, pw := io.Pipe()
tw := tar.NewWriter(pw)
go func() {
err := tw.WriteHeader(&tar.Header{Name: archivePath, Mode: 0o644, Size: size})
if err == nil {
_, err = io.Copy(tw, tmp)
}
if cerr := tw.Close(); err == nil {
err = cerr
}
_ = pw.CloseWithError(err)
}()
if err := h.dockerClient.CopyToContainer(c.Request.Context(), holderID, "/backup", pr); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": fmt.Sprintf("Failed to store archive: %v", err)})
return
}
now := sql.NullTime{Time: time.Now(), Valid: true}
if err := h.queries.CreateDatabaseBackup(c.Request.Context(), sqlcdb.CreateDatabaseBackupParams{
ID: backupID,
DatabaseID: databaseID,
Size: humanReadableBytes(size),
Status: "completed",
BackupPath: sql.NullString{String: archivePath, Valid: true},
CreatedAt: now,
}); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to register backup"})
return
}
_ = h.queries.SetDatabaseBackupStatusByID(c.Request.Context(), sqlcdb.SetDatabaseBackupStatusByIDParams{
Status: "completed",
Size: humanReadableBytes(size),
CompletedAt: now,
ID: backupID,
})
// Imported archives ship offsite too when a target is configured —
// the import should be a first-class restore point.
go func() {
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Minute)
defer cancel()
if key, err := h.uploadBackupToTarget(ctx, row, archivePath, backupID); err != nil {
log.Printf("containr: imported backup %s remote upload failed: %v (local archive kept)", backupID, err)
} else if key != "" {
log.Printf("containr: imported backup %s shipped to %s", backupID, key)
}
}()
LogAudit(userID, "database", databaseID, "backup_import", map[string]interface{}{"backup_id": backupID, "size": size})
c.JSON(http.StatusCreated, gin.H{"backup_id": backupID, "size": humanReadableBytes(size), "status": "completed"})
}
// openUploadStream resolves the request body: multipart `file` field when
// present, otherwise the raw body (e.g. CLI pipe of a .tar.gz).
func openUploadStream(c *gin.Context) (io.ReadCloser, string, error) {
if fh, err := c.FormFile("file"); err == nil {
src, err := fh.Open()
return src, fh.Filename, err
}
if c.Request.Body == nil || c.Request.ContentLength == 0 {
return nil, "", errors.New("no archive uploaded — send multipart field 'file' or a raw .tar.gz body")
}
return c.Request.Body, "archive.tar.gz", nil
}
func (h *DatabaseHandler) TestBackupTarget(c *gin.Context) {
userID, ok := requireAuthenticatedUserID(c)
if !ok {
+1
View File
@@ -398,6 +398,7 @@ func SetupRoutes(router *gin.Engine, db *database.DB, redis *database.Redis, cfg
authed.POST("/databases/:id/action", databaseHandler.PerformDatabaseAction)
authed.POST("/databases/:id/backup", databaseHandler.CreateBackup)
authed.POST("/databases/:id/restore", databaseHandler.RestoreBackup)
authed.POST("/databases/:id/backups/import", databaseHandler.ImportBackup)
authed.GET("/databases/:id/backups/:bid/download", databaseHandler.DownloadBackup)
authed.POST("/projects/:id/preview-environments", handleCreatePreviewEnvironment)
@@ -156,6 +156,48 @@ func (c *Client) Do(method, path string, body interface{}) ([]byte, error) {
return data, nil
}
// DoRaw sends a non-JSON request body (file uploads) and returns the response.
func (c *Client) DoRaw(method, path string, body io.Reader, contentType string) ([]byte, error) {
req, err := http.NewRequest(method, c.BaseURL+path, body)
if err != nil {
return nil, err
}
if c.Token != "" {
req.Header.Set("Authorization", "Bearer "+c.Token)
}
req.Header.Set("Content-Type", contentType)
req.Header.Set("Accept", "application/json")
hc := c.HTTPClient
if hc == nil {
hc = &http.Client{Timeout: 5 * time.Minute}
}
resp, err := hc.Do(req)
if err != nil {
return nil, &APIError{Status: 0, Message: fmt.Sprintf("cannot reach %s: %v", c.BaseURL, err), ExitCode: ExitUnavailable}
}
defer resp.Body.Close()
data, err := io.ReadAll(io.LimitReader(resp.Body, 64<<20))
if err != nil {
return nil, fmt.Errorf("failed to read response: %w", err)
}
if resp.StatusCode >= 400 {
msg := strings.TrimSpace(string(data))
var envelope struct {
Error string `json:"error"`
Code string `json:"code"`
}
if json.Unmarshal(data, &envelope) == nil && envelope.Error != "" {
msg = envelope.Error
if envelope.Code != "" {
msg = envelope.Code + ": " + msg
}
}
return nil, &APIError{Status: resp.StatusCode, Message: msg, ExitCode: exitCodeFor(resp.StatusCode)}
}
return data, nil
}
// DoJSON performs a request and unmarshals the response into out.
func (c *Client) DoJSON(method, path string, body interface{}, out interface{}) error {
data, err := c.Do(method, path, body)
+32 -2
View File
@@ -2,6 +2,7 @@ package commands
import (
"fmt"
"io"
"os"
"github.com/spf13/cobra"
@@ -199,7 +200,36 @@ var dbRestoreCmd = &cobra.Command{
if !Confirm(fmt.Sprintf("Restore database %s from backup %s? Current data will be overwritten.", args[0], args[1])) {
return &APIError{Message: "aborted (pass --yes to skip confirmation)", ExitCode: ExitError}
}
data, err := c.Do("POST", "/databases/"+args[0]+"/restore", map[string]string{"backup_id": args[1]})
data, err := c.Do("POST", "/databases/"+args[0]+"/restore", map[string]string{"backup_id": args[1], "database_id": args[0]})
if err != nil {
return err
}
PrintRaw(data)
return nil
},
}
var dbImportBackupCmd = &cobra.Command{
Use: "import-backup <id> <file.tar.gz|->",
Short: "Import a .tar.gz archive as a restore point (use '-' for stdin)",
Args: cobra.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := client()
if err != nil {
return err
}
var src io.ReadCloser
if args[1] == "-" {
src = io.NopCloser(os.Stdin)
} else {
f, err := os.Open(args[1])
if err != nil {
return err
}
src = f
}
defer src.Close()
data, err := c.DoRaw("POST", "/databases/"+args[0]+"/backups/import", src, "application/gzip")
if err != nil {
return err
}
@@ -332,5 +362,5 @@ func init() {
cmd.Flags().String("password", "", "database password (stored encrypted on register)")
cmd.Flags().Bool("ssl", false, "require TLS")
}
DatabasesCmd.AddCommand(dbListCmd, dbGetCmd, dbCreateCmd, dbUpdateCmd, dbRegisterCmd, dbTestConnCmd, dbDeleteCmd, dbActionCmd, dbBackupCmd, dbRestoreCmd, dbDownloadCmd)
DatabasesCmd.AddCommand(dbListCmd, dbGetCmd, dbCreateCmd, dbUpdateCmd, dbRegisterCmd, dbTestConnCmd, dbDeleteCmd, dbActionCmd, dbBackupCmd, dbRestoreCmd, dbImportBackupCmd, dbDownloadCmd)
}
@@ -31,6 +31,7 @@ import {
restoreDatabaseBackup,
createManagedDatabase,
registerExternalDatabase,
importDatabaseBackup,
updateDatabase,
listBackupTargets,
createBackupTarget,
@@ -130,6 +131,7 @@ import {
HeartPulse,
Cloud,
ExternalLink,
Upload,
} from 'lucide-react';
function SecondaryPageHeader({ title, description }: { title: string; description: string }) {
@@ -2120,6 +2122,10 @@ export function DatabasesPage() {
updateDatabase(id, { backup_target_id: targetId }),
onSuccess: invalidate,
});
const importBackupMutation = useMutation({
mutationFn: ({ id, file }: { id: string; file: File }) => importDatabaseBackup(id, file),
onSuccess: invalidate,
});
const testByIdMutation = useMutation({
mutationFn: (id: string) => testDatabaseConnectionByID(id),
onSuccess: (result, id) => setExternalTestResult((prev) => ({ ...prev, [id]: result })),
@@ -2358,6 +2364,24 @@ export function DatabasesPage() {
))}
</select>
)}
<label
className={`flex items-center gap-1.5 px-3 py-1.5 rounded-[var(--radius-md)] text-xs font-medium border border-[var(--border-subtle)] text-[var(--text-secondary)] transition-all ${importBackupMutation.isPending ? 'opacity-50' : 'cursor-pointer hover:text-[var(--text-primary)]'}`}
title="Import a .tar.gz archive as a restore point"
>
<Upload size={11} />
{importBackupMutation.isPending ? 'Importing…' : 'Import'}
<input
type="file"
accept=".tar.gz,application/gzip"
className="hidden"
disabled={importBackupMutation.isPending}
onChange={(e) => {
const f = e.target.files?.[0];
if (f) importBackupMutation.mutate({ id: db.id ?? '', file: f });
e.target.value = '';
}}
/>
</label>
<button
onClick={() => backupMutation.mutate(db.id ?? '')}
disabled={!running || backupMutation.isPending}
+72
View File
@@ -3612,6 +3612,78 @@ export interface paths {
patch?: never;
trace?: never;
};
"/databases/{id}/backups/import": {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
get?: never;
put?: never;
/**
* Import a backup archive
* @description Upload a `.tar.gz` archive (multipart field `file` or a raw application/gzip body) to create a new restore point. The archive lands in the backup volume and, when a backup target is assigned, is also shipped offsite. Managed databases only; 2 GiB ceiling.
*/
post: {
parameters: {
query?: never;
header?: never;
path: {
/** @description Database ID */
id: string;
};
cookie?: never;
};
requestBody: {
content: {
"multipart/form-data": {
/** Format: binary */
file?: string;
};
"application/gzip": string;
};
};
responses: {
/** @description Backup imported */
201: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": {
backup_id?: string;
size?: string;
status?: string;
};
};
};
/** @description Invalid archive */
400: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["Error"];
};
};
/** @description Archive too large */
413: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["Error"];
};
};
};
};
delete?: never;
options?: never;
head?: never;
patch?: never;
trace?: never;
};
"/databases/{id}/backups/{bid}/download": {
parameters: {
query?: never;
+15
View File
@@ -1727,6 +1727,21 @@ export async function restoreDatabaseBackup(id: string, backupId: string): Promi
});
}
export async function importDatabaseBackup(id: string, file: File): Promise<{ backup_id: string; size: string }> {
const form = new FormData();
form.append('file', file);
const resp = await fetch(`${getApiBaseUrl()}/databases/${encodeURIComponent(id)}/backups/import`, {
method: 'POST',
headers: authHeaders(),
body: form,
});
const payload = await resp.json().catch(() => ({}));
if (!resp.ok) {
throw new ApiError(payload.error || 'Import failed', resp.status);
}
return payload;
}
export async function createManagedDatabase(
input: CreateDatabaseInput,
): Promise<{ id: string; status: string }> {
+60
View File
@@ -2388,6 +2388,66 @@ paths:
schema:
$ref: '#/components/schemas/MessageResponse'
/databases/{id}/backups/import:
post:
tags:
- Databases
summary: Import a backup archive
description: >-
Upload a `.tar.gz` archive (multipart field `file` or a raw
application/gzip body) to create a new restore point. The archive
lands in the backup volume and, when a backup target is assigned,
is also shipped offsite. Managed databases only; 2 GiB ceiling.
security:
- bearerAuth: []
parameters:
- name: id
in: path
required: true
description: Database ID
schema:
type: string
requestBody:
required: true
content:
multipart/form-data:
schema:
type: object
properties:
file:
type: string
format: binary
application/gzip:
schema:
type: string
format: binary
responses:
'201':
description: Backup imported
content:
application/json:
schema:
type: object
properties:
backup_id:
type: string
size:
type: string
status:
type: string
'400':
description: Invalid archive
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
'413':
description: Archive too large
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
/databases/{id}/backups/{bid}/download:
get:
tags:
+3 -1
View File
@@ -47,7 +47,9 @@ deployments list|get|logs|rollback|cancel
logs <service-id> [--follow] runtime logs (SSE follow)
exec <service-id> -- <cmd> one-off container command
variables list|set|unset env vars (secrets masked; --redeploy applies them)
databases list|get|create|update|delete|action|backup|restore|download-backup
databases list|get|create|update|delete|action|backup|restore|download-backup|import-backup
import-backup <id> <file.tar.gz|-> uploads an archive as a restore
point; ships offsite too when a backup target is assigned
create --public exposes the port on all interfaces;
update <id> --public/--public=false toggles it (container recreates, data persists)
databases register <name> register an external database (--type --host --port --database --username --password --ssl; probed before storing)
+1
View File
@@ -24,6 +24,7 @@ Base URL: `https://<host>/api/v1`. OpenAPI: `docs/api/openapi.yaml`.
- **External databases**: `POST /databases/register-external` (probes first; `{name,type,host,port,database,username,password,ssl}`), `POST /databases/test-connection` (ad-hoc probe → `{ok,latency_ms,error}`), `POST /databases/{id}/test-connection` (re-probe with stored credentials). Rows carry `provider:"external"` + `external{host,port,database,username,has_password,ssl}`; passwords encrypted at rest, never returned. Lifecycle actions and backups reject external rows — no container exists.
- **Public DB port**: managed databases bind `127.0.0.1` by default; `public_port:true` on create or `PUT /databases/{id} {public_port}` binds `0.0.0.0`. Toggling recreates the container — the data volume persists.
- **Backup targets**: S3-compatible destinations (`/backup-targets` CRUD + `POST /backup-targets/{id}/test`). Endpoints/buckets are probed before storing; keys encrypted at rest (`has_credentials` only). Assign via `PUT /databases/{id} {backup_target_id}` — empty string clears; external databases reject assignment. Managed backups stay local; a copy ships to `[prefix/]databases/<db-id>/<archive>` and rows expose `remote:true`. Restore prefers the local archive and falls back to the remote object. Deleting a target is refused (409) while databases reference it.
- **Backup import**: `POST /databases/{id}/backups/import` — upload a `.tar.gz` (multipart `file` or raw `application/gzip` body, ≤2 GiB) to create a restore point; also ships offsite when a target is assigned. CLI: `databases import-backup <id> <file|->`.
- **Identifiers**: all ids are UUIDs. Tokens: `cnp_` user PATs, `cagt_` agent onboarding tokens.
## Core objects