mirror of
https://github.com/Dvorinka/Primora.git
synced 2026-10-28 07:41:23 +00:00
feat: Presigned object URLs (s3 driver)
POST /buckets/:id/object-presigns mints SigV4 query-signed GET/PUT URLs so clients move bytes directly against S3 — large transfers skip the backend entirely. - storage.Presigner optional interface; S3Store implements, local driver returns a clear error (no external surface to sign). - TTL 60s–1h (default 15m); download requires read role + existing object row (no key probing); upload requires write role. - S3_PUBLIC_ENDPOINT rewrites the client-facing host for internal S3_ENDPOINTs — signatures bind the host, URLs are valid only where minted. - CLI: primora objects:presign <bucket> <key> [--upload] [--ttl]. Verified live against cloudserver: GET fetched the object with no auth headers; PUT pushed 1 MB direct to S3; unsigned list → 403; untracked-key download presign → 404. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
1 parent
a84049b8fc
commit
fd36389328
22 files changed
+446
-7
No files matched your search
@@ -60,6 +60,10 @@ BACKEND_STORAGE_DRIVER=local
|
||||
#S3_SECRET_ACCESS_KEY=
|
||||
#S3_PREFIX=
|
||||
#S3_PATH_STYLE=true
|
||||
# Client-facing S3 endpoint for presigned URLs — set when S3_ENDPOINT is an
|
||||
# internal address (e.g. http://minio:9000 inside compose) that browsers
|
||||
# can't reach. Presigned URLs are valid only at the host they were minted for.
|
||||
#S3_PUBLIC_ENDPOINT=https://objects.example.com
|
||||
|
||||
# Mail defaults — each is overridable in-app (Settings → Instance). In-app
|
||||
# values win over these env vars; deleting the in-app value reverts here.
|
||||
|
||||
@@ -25,6 +25,7 @@ All notable changes to Primora. Format follows [Keep a Changelog](https://keepac
|
||||
- **Email templates + send log** (`core.email_log`, migration 00014) — the mailer renders registered templates (currently `invitation`) and every send — successful or failed — is recorded with template, recipient, subject, status, and error text. `GET /projects/:id/emails` lists recent sends; the Members page shows an "Email log" card under pending invitations.
|
||||
- **S3-compatible object storage backend** — `BACKEND_STORAGE_DRIVER=s3` swaps the local disk store for any S3-compatible API (AWS, MinIO, Garage, R2) via a stdlib-only SigV4 implementation — no AWS SDK dependency. `S3_ENDPOINT`/`S3_REGION`/`S3_BUCKET`/credentials configure it; `S3_PATH_STYLE` (default on) covers MinIO/Garage, off for AWS virtual-hosted. The configured bucket is created on startup if missing; objects stream with computed SHA-256, `storage_path` records `s3://bucket/key`. Verified live end-to-end (bucket → put → get → copy → list → delete).
|
||||
- **Functions** (`core.functions` + `core.function_runs`, migration 00015) — project-scoped JS/TS executed by a runtime binary on the backend host (`bun` or `deno`, per function). The JSON payload arrives on stdin and as `PRIMORA_PAYLOAD`; stdout/stderr are captured per run (capped at 64 KiB) with status, exit code and duration — `FUNCTIONS_RUNTIME`/`FUNCTIONS_TIMEOUT_SECONDS`/`FUNCTIONS_MAX_OUTPUT_BYTES` configure the runner. API: CRUD under `/projects/:id/functions`, `POST …/invoke` (502 + recorded run on failure), `GET …/runs`. Dashboard gains a Functions page: code editor, JSON payload invoke, live result, run history. Code executes with backend privileges — the documented self-hosted trust model.
|
||||
- **Presigned object URLs** — `POST /buckets/:bucketID/object-presigns` mints SigV4 query-signed `GET`/`PUT` URLs (60 s–1 h TTL) so clients transfer large objects directly against S3 without proxying through the backend. Read role for downloads (object must exist — no key probing), write role for uploads. `S3_PUBLIC_ENDPOINT` rewrites the client-facing host for internal `S3_ENDPOINT`s; signatures bind the host so URLs are valid only where they were minted. The local driver returns a clear error — a presigned URL needs a real object API.
|
||||
- **Function triggers** (migration 00016) — three new ways to run a function, all through the same executor and run log. Scheduled jobs accept `function_id` instead of `url` (run status mirrors the exit code); inbound hooks gain `mode: "function"` — the received body becomes the function's stdin payload, acked immediately with the run recorded async; and `event_pattern` on a function (`document.*`, `*`, or an exact type) fires it detached whenever a matching domain event publishes — `{event, project_id, occurred_at, data}` as payload. Runs record trigger source: `manual`/`schedule`/`hook`/`event`. Dashboard: function event-trigger field, job target picker (URL or function), and inbound-hook function mode.
|
||||
|
||||
## [0.7.1] - 2026-09-23
|
||||
|
||||
@@ -46,6 +46,7 @@ docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d
|
||||
| `S3_ENDPOINT` / `S3_REGION` / `S3_BUCKET` | S3 API endpoint (empty endpoint → AWS `s3.<region>.amazonaws.com`), region, bucket | with `s3` |
|
||||
| `S3_ACCESS_KEY_ID` / `S3_SECRET_ACCESS_KEY` | S3 credentials | with `s3` |
|
||||
| `S3_PREFIX` / `S3_PATH_STYLE` | optional key prefix; path-style URLs (default `true`, needed by MinIO/Garage — set `false` for AWS virtual-hosted) | no |
|
||||
| `S3_PUBLIC_ENDPOINT` | client-facing S3 endpoint used for presigned URLs when `S3_ENDPOINT` is internal-only (e.g. `http://minio:9000`); signatures bind the public host | no |
|
||||
| `FUNCTIONS_RUNTIME` / `FUNCTIONS_TIMEOUT_SECONDS` / `FUNCTIONS_MAX_OUTPUT_BYTES` | Functions: runtime binary (`auto` resolves `bun`/`deno` from the backend's PATH — the backend image must ship one, or set this to its path), per-run timeout (default 30 s), captured-output cap (default 64 KiB) | no |
|
||||
| `AUTH_ADMIN_EMAILS` | comma-separated emails promoted to auth admin on boot | no |
|
||||
| `USER_RATE_LIMIT_PER_MINUTE` / `API_KEY_RATE_LIMIT_PER_MINUTE` | per-identity API rate limits (defaults 240 / 600) | no |
|
||||
|
||||
@@ -67,6 +67,7 @@ The important ones:
|
||||
| `USER_RATE_LIMIT_PER_MINUTE` / `API_KEY_RATE_LIMIT_PER_MINUTE` | Throttling defaults — overridable in-app |
|
||||
| `BACKEND_STORAGE_DRIVER` | `local` (default, `BACKEND_STORAGE_ROOT`) or `s3` — S3-compatible object storage |
|
||||
| `S3_ENDPOINT` / `S3_BUCKET` / `S3_ACCESS_KEY_ID` / `S3_SECRET_ACCESS_KEY` | S3 backend config (with `S3_REGION`, `S3_PREFIX`, `S3_PATH_STYLE`) |
|
||||
| `S3_PUBLIC_ENDPOINT` | Client-facing S3 host for presigned URLs when `S3_ENDPOINT` is internal-only |
|
||||
| `FUNCTIONS_RUNTIME` / `FUNCTIONS_TIMEOUT_SECONDS` | JS runtime for Functions — `auto` (default, resolves `bun`/`deno` from PATH) or a binary path; 30 s default timeout |
|
||||
| `VITE_DEMO_MODE` | Enable the client-side demo workspace |
|
||||
|
||||
|
||||
@@ -48,7 +48,7 @@ Supabase, with no hosted tier - your data stays on your hardware.
|
||||
|
||||
- **Organizations & projects** - workspaces with scoped org and project roles.
|
||||
- **Auth** - email/password plus optional GitHub, Google, Discord, and Microsoft OAuth. JWTs verified by the API against JWKS.
|
||||
- **Storage** - S3-style buckets on local disk or any S3-compatible backend (AWS, MinIO, Garage, R2), public or private.
|
||||
- **Storage** - S3-style buckets on local disk or any S3-compatible backend (AWS, MinIO, Garage, R2), public or private. With `s3` the API mints presigned upload/download URLs so large objects skip the backend.
|
||||
- **Collections** - schema-flexible JSON documents in Postgres JSONB, with filter/order queries.
|
||||
- **Functions** - project-scoped JS/TS executed by bun/deno on the host, with per-run logs. Trigger them four ways: manual invoke, cron schedules (`function_id` on a job), inbound hooks (`mode: "function"`), or domain events (`event_pattern` like `document.*`).
|
||||
- **Vault** - AES-256-GCM project secrets plus an Argon2id/XChaCha20 local vault in the CLI and desktop app.
|
||||
|
||||
+6
-4
@@ -365,10 +365,12 @@ Ordered loosely by leverage. None committed; each gets scoped when picked.
|
||||
Runs record trigger source (`manual`/`schedule`/`hook`/`event`).
|
||||
Remaining candidate: isolated runtimes (sidecar/Firecracker) for
|
||||
untrusted code — functions still execute with backend host privileges.
|
||||
- **Phase 10 — Storage backends** — ✅ S3-compatible object storage shipped:
|
||||
`BACKEND_STORAGE_DRIVER=s3` + `S3_*` env vars, stdlib SigV4, verified
|
||||
against a real S3 server. Presigned-URL uploads for large files remain a
|
||||
candidate.
|
||||
- **Phase 10 — Storage backends** — ✅ shipped: `BACKEND_STORAGE_DRIVER=s3`
|
||||
+ `S3_*` env vars, stdlib SigV4, verified against a real S3 server.
|
||||
Presigned upload/download URLs shipped: `POST
|
||||
/buckets/:id/object-presigns` mints SigV4 query-signed GET/PUT URLs
|
||||
(`S3_PUBLIC_ENDPOINT` rewrites the client-facing host); local driver
|
||||
returns a clear error — a presigned URL needs a real object API behind it.
|
||||
- **HA mode** — ✅ shipped: PostgreSQL advisory-lock leadership for the
|
||||
scheduler and alert evaluator; failover verified with two replicas.
|
||||
- **Email surface** — ✅ shipped: templated sends + `core.email_log` +
|
||||
|
||||
@@ -89,6 +89,7 @@ func Bootstrap(ctx context.Context) (*App, error) {
|
||||
Bucket: cfg.S3Bucket,
|
||||
AccessKeyID: cfg.S3AccessKeyID,
|
||||
SecretAccessKey: cfg.S3SecretAccessKey,
|
||||
PublicEndpoint: cfg.S3PublicEndpoint,
|
||||
Prefix: cfg.S3Prefix,
|
||||
PathStyle: cfg.S3PathStyle,
|
||||
})
|
||||
|
||||
@@ -44,6 +44,7 @@ type Config struct {
|
||||
S3Bucket string
|
||||
S3AccessKeyID string
|
||||
S3SecretAccessKey string
|
||||
S3PublicEndpoint string // presigned-URL host when it differs from Endpoint
|
||||
S3Prefix string
|
||||
S3PathStyle bool
|
||||
|
||||
@@ -86,6 +87,7 @@ func Load() (Config, error) {
|
||||
S3Bucket: os.Getenv("S3_BUCKET"),
|
||||
S3AccessKeyID: os.Getenv("S3_ACCESS_KEY_ID"),
|
||||
S3SecretAccessKey: os.Getenv("S3_SECRET_ACCESS_KEY"),
|
||||
S3PublicEndpoint: os.Getenv("S3_PUBLIC_ENDPOINT"),
|
||||
S3Prefix: os.Getenv("S3_PREFIX"),
|
||||
FunctionsRuntime: getenv("FUNCTIONS_RUNTIME", "auto"),
|
||||
FunctionsMaxBytes: 65536,
|
||||
|
||||
@@ -142,6 +142,7 @@ func (h *HTTPHandler) Register(router *gin.Engine) {
|
||||
api.GET("/buckets/:bucketID/objects", h.listObjects)
|
||||
api.POST("/buckets/:bucketID/objects", h.uploadObject)
|
||||
api.POST("/buckets/:bucketID/object-copies", h.copyObject)
|
||||
api.POST("/buckets/:bucketID/object-presigns", h.presignObject)
|
||||
api.GET("/buckets/:bucketID/objects/*objectKey", h.downloadObject)
|
||||
api.PATCH("/buckets/:bucketID/objects/*objectKey", h.updateObject)
|
||||
api.DELETE("/buckets/:bucketID/objects/*objectKey", h.deleteObject)
|
||||
@@ -891,6 +892,39 @@ func (h *HTTPHandler) updateObject(c *gin.Context) {
|
||||
c.Status(http.StatusNoContent)
|
||||
}
|
||||
|
||||
type presignObjectRequest struct {
|
||||
Key string `json:"key" validate:"required"`
|
||||
Op string `json:"op" validate:"omitempty,oneof=download upload"`
|
||||
TTLSeconds int `json:"ttl_seconds" validate:"omitempty,min=60,max=3600"`
|
||||
}
|
||||
|
||||
// presignObject returns a short-lived URL clients call against storage
|
||||
// directly — keeps large transfers off the backend. S3 driver only.
|
||||
func (h *HTTPHandler) presignObject(c *gin.Context) {
|
||||
actor, ok := middleware.RequireActor(c)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
bucketID, ok := parseUUIDParam(c, "bucketID")
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
var body presignObjectRequest
|
||||
if !h.bindAndValidate(c, &body) {
|
||||
return
|
||||
}
|
||||
out, err := h.Platform.PresignObject(c.Request.Context(), actor, bucketID, strings.TrimSpace(body.Key), body.Op, body.TTLSeconds)
|
||||
if err != nil {
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
apperrors.Abort(c, http.StatusNotFound, "not_found", "bucket or object not found")
|
||||
return
|
||||
}
|
||||
h.handleError(c, err)
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusOK, out)
|
||||
}
|
||||
|
||||
func (h *HTTPHandler) deleteObject(c *gin.Context) {
|
||||
actor, ok := middleware.RequireActor(c)
|
||||
if !ok {
|
||||
|
||||
@@ -10,6 +10,7 @@ import (
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
@@ -1418,6 +1419,48 @@ func (s *PlatformService) GetObject(ctx context.Context, actor *models.Actor, bu
|
||||
return object, file, nil
|
||||
}
|
||||
|
||||
// PresignObject mints a short-lived direct-to-storage URL for a single
|
||||
// object. Requires the s3 storage driver — the local store has no external
|
||||
// URL surface to sign against.
|
||||
func (s *PlatformService) PresignObject(ctx context.Context, actor *models.Actor, bucketID uuid.UUID, objectKey, op string, ttlSeconds int) (*storage.PresignedURL, error) {
|
||||
bucket, err := s.repo.Queries().GetBucketByID(ctx, bucketID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
method := ""
|
||||
switch op {
|
||||
case "download", "":
|
||||
method = http.MethodGet
|
||||
if err := s.requireBucketRead(ctx, actor, bucket); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !objectExists(ctx, s, bucketID, objectKey) {
|
||||
return nil, pgx.ErrNoRows
|
||||
}
|
||||
case "upload":
|
||||
method = http.MethodPut
|
||||
if err := s.requireBucketWrite(ctx, actor, bucket); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
default:
|
||||
return nil, fmt.Errorf("invalid op %q — download or upload", op)
|
||||
}
|
||||
p, ok := s.store.(storage.Presigner)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("presigned URLs require BACKEND_STORAGE_DRIVER=s3")
|
||||
}
|
||||
ttl := time.Duration(ttlSeconds) * time.Second
|
||||
return p.Presign(ctx, bucketID.String(), objectKey, method, ttl)
|
||||
}
|
||||
|
||||
func objectExists(ctx context.Context, s *PlatformService, bucketID uuid.UUID, key string) bool {
|
||||
_, err := s.repo.Queries().GetBucketObjectByKey(ctx, db.GetBucketObjectByKeyParams{
|
||||
BucketID: bucketID,
|
||||
ObjectKey: key,
|
||||
})
|
||||
return err == nil
|
||||
}
|
||||
|
||||
func (s *PlatformService) UpdateObject(ctx context.Context, actor *models.Actor, bucketID uuid.UUID, objectKey string, input UpdateObjectInput, requestID string) (db.CoreBucketObject, error) {
|
||||
sourceBucket, err := s.repo.Queries().GetBucketByID(ctx, bucketID)
|
||||
if err != nil {
|
||||
|
||||
@@ -28,7 +28,11 @@ type S3Config struct {
|
||||
SecretAccessKey string
|
||||
Prefix string // optional key prefix, e.g. "primora/"
|
||||
PathStyle bool // bucket/key path style (MinIO, Garage) vs virtual-hosted
|
||||
HTTPClient *http.Client
|
||||
// PublicEndpoint rewrites presigned-URL hosts for clients that can't
|
||||
// reach Endpoint (e.g. minio:9000 inside compose). Signature binds the
|
||||
// public host — the URL is only valid there.
|
||||
PublicEndpoint string
|
||||
HTTPClient *http.Client
|
||||
}
|
||||
|
||||
// S3Store implements Store against an S3-compatible API using stdlib-only
|
||||
@@ -271,6 +275,88 @@ func (s *S3Store) DeleteBucket(bucketID string) error {
|
||||
}
|
||||
}
|
||||
|
||||
// Presign mints a SigV4 query-signed URL for a single GET/PUT on the
|
||||
// object — clients talk to S3 directly instead of proxying bytes through
|
||||
// the backend. TTL is capped at one hour.
|
||||
func (s *S3Store) Presign(ctx context.Context, bucketID, objectKey, method string, ttl time.Duration) (*PresignedURL, error) {
|
||||
if method != http.MethodGet && method != http.MethodPut {
|
||||
return nil, fmt.Errorf("s3 presign: unsupported method %q", method)
|
||||
}
|
||||
key, err := s.objectKey(bucketID, objectKey)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if ttl <= 0 {
|
||||
ttl = 15 * time.Minute
|
||||
}
|
||||
if ttl > time.Hour {
|
||||
ttl = time.Hour
|
||||
}
|
||||
|
||||
now := s.now().UTC()
|
||||
amzDate := now.Format("20060102T150405Z")
|
||||
dateStamp := now.Format("20060102")
|
||||
scope := fmt.Sprintf("%s/%s/s3/aws4_request", dateStamp, s.cfg.Region)
|
||||
|
||||
// Build the unsigned URL — same path/vhost logic as signedRequest, but
|
||||
// on the client-facing endpoint when one is configured.
|
||||
var base *url.URL
|
||||
endpoint := strings.TrimSuffix(s.cfg.PublicEndpoint, "/")
|
||||
if endpoint == "" {
|
||||
endpoint = strings.TrimSuffix(s.cfg.Endpoint, "/")
|
||||
}
|
||||
if s.cfg.PathStyle {
|
||||
u, err := url.Parse(endpoint + "/" + s.cfg.Bucket + "/" + escapeKey(key))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("s3 presign url: %w", err)
|
||||
}
|
||||
base = u
|
||||
} else {
|
||||
u, err := url.Parse(endpoint)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("s3 presign url: %w", err)
|
||||
}
|
||||
u.Host = s.cfg.Bucket + "." + u.Host
|
||||
u.Path = "/" + escapeKey(key)
|
||||
base = u
|
||||
}
|
||||
|
||||
q := base.Query()
|
||||
q.Set("X-Amz-Algorithm", "AWS4-HMAC-SHA256")
|
||||
q.Set("X-Amz-Credential", s.cfg.AccessKeyID+"/"+scope)
|
||||
q.Set("X-Amz-Date", amzDate)
|
||||
q.Set("X-Amz-Expires", fmt.Sprintf("%d", int64(ttl.Seconds())))
|
||||
q.Set("X-Amz-SignedHeaders", "host")
|
||||
|
||||
canonicalURI := base.EscapedPath()
|
||||
if canonicalURI == "" {
|
||||
canonicalURI = "/"
|
||||
}
|
||||
canonicalRequest := strings.Join([]string{
|
||||
method,
|
||||
canonicalURI,
|
||||
canonicalQuery(q),
|
||||
"host:" + base.Host + "\n",
|
||||
"host",
|
||||
"UNSIGNED-PAYLOAD",
|
||||
}, "\n")
|
||||
stringToSign := strings.Join([]string{
|
||||
"AWS4-HMAC-SHA256",
|
||||
amzDate,
|
||||
scope,
|
||||
hexSHA256(canonicalRequest),
|
||||
}, "\n")
|
||||
|
||||
kDate := hmacSHA256([]byte("AWS4"+s.cfg.SecretAccessKey), dateStamp)
|
||||
kRegion := hmacSHA256(kDate, s.cfg.Region)
|
||||
kService := hmacSHA256(kRegion, "s3")
|
||||
kSigning := hmacSHA256(kService, "aws4_request")
|
||||
q.Set("X-Amz-Signature", hex.EncodeToString(hmacSHA256(kSigning, stringToSign)))
|
||||
base.RawQuery = q.Encode()
|
||||
|
||||
return &PresignedURL{URL: base.String(), Method: method, ExpiresAt: now.Add(ttl)}, nil
|
||||
}
|
||||
|
||||
type listResult struct {
|
||||
Contents []struct{ Key string `xml:"Key"` } `xml:"Contents"`
|
||||
NextContinuationToken string `xml:"NextContinuationToken"`
|
||||
|
||||
@@ -6,6 +6,7 @@ import (
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
@@ -214,3 +215,37 @@ func TestCanonicalQueryRFC3986(t *testing.T) {
|
||||
t.Fatalf("canonicalQuery = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestS3StorePresign(t *testing.T) {
|
||||
store, _ := newTestS3(t)
|
||||
ctx := context.Background()
|
||||
|
||||
dl, err := store.Presign(ctx, "bucket-a", "docs/x.txt", http.MethodGet, 10*time.Minute)
|
||||
if err != nil {
|
||||
t.Fatalf("Presign: %v", err)
|
||||
}
|
||||
if dl.Method != http.MethodGet || dl.ExpiresAt.IsZero() {
|
||||
t.Fatalf("bad presign: %+v", dl)
|
||||
}
|
||||
u, err := url.Parse(dl.URL)
|
||||
if err != nil {
|
||||
t.Fatalf("url parse: %v", err)
|
||||
}
|
||||
q := u.Query()
|
||||
for _, k := range []string{"X-Amz-Algorithm", "X-Amz-Credential", "X-Amz-Date", "X-Amz-Expires", "X-Amz-SignedHeaders", "X-Amz-Signature"} {
|
||||
if q.Get(k) == "" {
|
||||
t.Fatalf("missing %s in %s", k, dl.URL)
|
||||
}
|
||||
}
|
||||
if q.Get("X-Amz-Expires") != "600" || q.Get("X-Amz-SignedHeaders") != "host" {
|
||||
t.Fatalf("bad query: %s", dl.URL)
|
||||
}
|
||||
if _, err := store.Presign(ctx, "b", "k", http.MethodDelete, 0); err == nil {
|
||||
t.Fatal("expected method rejection")
|
||||
}
|
||||
// TTL cap.
|
||||
capped, _ := store.Presign(ctx, "b", "k.txt", http.MethodPut, 48*time.Hour)
|
||||
if cu, _ := url.Parse(capped.URL); cu.Query().Get("X-Amz-Expires") != "3600" {
|
||||
t.Fatalf("ttl not capped: %s", capped.URL)
|
||||
}
|
||||
}
|
||||
@@ -3,6 +3,7 @@ package storage
|
||||
import (
|
||||
"context"
|
||||
"io"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Store is the object-storage contract the platform service depends on.
|
||||
@@ -14,3 +15,17 @@ type Store interface {
|
||||
MoveBetweenBuckets(sourceBucketID, destinationBucketID, fromKey, toKey string) (string, error)
|
||||
DeleteBucket(bucketID string) error
|
||||
}
|
||||
|
||||
// PresignedURL is a short-lived direct-to-storage URL for one operation.
|
||||
type PresignedURL struct {
|
||||
URL string `json:"url"`
|
||||
Method string `json:"method"`
|
||||
ExpiresAt time.Time `json:"expires_at"`
|
||||
}
|
||||
|
||||
// Presigner is an optional capability: stores backed by a real object API
|
||||
// (S3) can mint URLs clients call directly. The local driver can't — its
|
||||
// "URL" would have to be served by this backend anyway.
|
||||
type Presigner interface {
|
||||
Presign(ctx context.Context, bucketID, objectKey, method string, ttl time.Duration) (*PresignedURL, error)
|
||||
}
|
||||
@@ -677,6 +677,37 @@ paths:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/BucketObject"
|
||||
/buckets/{bucketID}/object-presigns:
|
||||
post:
|
||||
tags: [Storage]
|
||||
operationId: presignBucketObject
|
||||
description: >-
|
||||
Mint a short-lived URL the client calls against object storage
|
||||
directly — keeps large transfers off the backend. Requires
|
||||
BACKEND_STORAGE_DRIVER=s3; the URL's host comes from
|
||||
S3_PUBLIC_ENDPOINT (or S3_ENDPOINT when unset). Signatures bind the
|
||||
host — a presigned URL is valid only at the endpoint it was minted
|
||||
for.
|
||||
security:
|
||||
- bearerAuth: []
|
||||
- apiKeyAuth: []
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/BucketID"
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/PresignObjectRequest"
|
||||
responses:
|
||||
"200":
|
||||
description: Presigned URL
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/PresignedURL"
|
||||
"404":
|
||||
description: Bucket or object not found (download)
|
||||
/buckets/{bucketID}:
|
||||
patch:
|
||||
tags: [Storage]
|
||||
@@ -3130,6 +3161,36 @@ components:
|
||||
format: uuid
|
||||
nullable: true
|
||||
required: [objectKey, newObjectKey]
|
||||
PresignObjectRequest:
|
||||
type: object
|
||||
properties:
|
||||
key:
|
||||
type: string
|
||||
description: Object key inside the bucket.
|
||||
op:
|
||||
type: string
|
||||
enum: [download, upload]
|
||||
default: download
|
||||
description: download → GET URL (object must exist); upload → PUT URL (creates/overwrites).
|
||||
ttl_seconds:
|
||||
type: integer
|
||||
minimum: 60
|
||||
maximum: 3600
|
||||
default: 900
|
||||
required: [key]
|
||||
PresignedURL:
|
||||
type: object
|
||||
properties:
|
||||
url:
|
||||
type: string
|
||||
description: Fully-signed URL — call it with the returned method, no auth header needed.
|
||||
method:
|
||||
type: string
|
||||
enum: [GET, PUT]
|
||||
expires_at:
|
||||
type: string
|
||||
format: date-time
|
||||
required: [url, method, expires_at]
|
||||
PaginationMetadata:
|
||||
type: object
|
||||
properties:
|
||||
|
||||
@@ -12,6 +12,7 @@ import {
|
||||
cmdObjectsDownload,
|
||||
cmdObjectsList,
|
||||
cmdObjectsRemove,
|
||||
cmdObjectsPresign,
|
||||
cmdObjectsUpload,
|
||||
} from "./commands/objects.js";
|
||||
import { cmdOrgsCreate, cmdOrgsList } from "./commands/orgs.js";
|
||||
@@ -114,6 +115,12 @@ cli
|
||||
.command("objects:rm <bucket> <key>", "Delete an object")
|
||||
.option("--project <id>", "Project override")
|
||||
.action(cmdObjectsRemove);
|
||||
cli
|
||||
.command("objects:presign <bucket> <key>", "Mint a presigned URL (s3 driver)")
|
||||
.option("--project <id>", "Project override")
|
||||
.option("--upload", "PUT URL for direct upload instead of GET download")
|
||||
.option("--ttl <seconds>", "Lifetime 60-3600 (default 900)")
|
||||
.action(cmdObjectsPresign);
|
||||
|
||||
cli
|
||||
.command("keys:list", "List API keys for the current project")
|
||||
|
||||
@@ -2,7 +2,7 @@ import { statSync } from "node:fs";
|
||||
import { readFile, writeFile } from "node:fs/promises";
|
||||
import { basename } from "node:path";
|
||||
|
||||
import { StorageService } from "@primora/api-client";
|
||||
import { PresignObjectRequest, StorageService } from "@primora/api-client";
|
||||
|
||||
import { loadConfig } from "../config.js";
|
||||
import {
|
||||
@@ -133,3 +133,31 @@ export async function cmdObjectsRemove(
|
||||
await StorageService.deleteBucketObject({ bucketId, objectKey });
|
||||
success(`Deleted ${objectKey}`);
|
||||
}
|
||||
|
||||
export async function cmdObjectsPresign(
|
||||
bucketRef: string,
|
||||
objectKey: string,
|
||||
options: { project?: string; upload?: boolean; ttl?: string; json?: boolean },
|
||||
): Promise<void> {
|
||||
const cfg = loadConfig();
|
||||
requireAuth(cfg);
|
||||
configureClient(cfg);
|
||||
|
||||
const bucketId = await bucketIdFor(cfg, bucketRef, options.project);
|
||||
const presigned = await StorageService.presignBucketObject({
|
||||
bucketId,
|
||||
requestBody: {
|
||||
key: objectKey,
|
||||
op: options.upload
|
||||
? PresignObjectRequest.op.UPLOAD
|
||||
: PresignObjectRequest.op.DOWNLOAD,
|
||||
ttl_seconds: options.ttl ? Number(options.ttl) : undefined,
|
||||
},
|
||||
});
|
||||
if (isJson(options)) {
|
||||
printJson(presigned);
|
||||
return;
|
||||
}
|
||||
console.log(presigned.url);
|
||||
console.error(`# ${presigned.method} — expires ${presigned.expires_at}`);
|
||||
}
|
||||
@@ -108,6 +108,8 @@ export type { OrganizationSummary } from './models/OrganizationSummary';
|
||||
export type { PaginationLimit } from './models/PaginationLimit';
|
||||
export type { PaginationMetadata } from './models/PaginationMetadata';
|
||||
export type { PaginationOffset } from './models/PaginationOffset';
|
||||
export { PresignedURL } from './models/PresignedURL';
|
||||
export { PresignObjectRequest } from './models/PresignObjectRequest';
|
||||
export type { Project } from './models/Project';
|
||||
export type { ProjectID } from './models/ProjectID';
|
||||
export { ProjectMember } from './models/ProjectMember';
|
||||
@@ -258,6 +260,8 @@ export { $OrganizationSummary } from './schemas/$OrganizationSummary';
|
||||
export { $PaginationLimit } from './schemas/$PaginationLimit';
|
||||
export { $PaginationMetadata } from './schemas/$PaginationMetadata';
|
||||
export { $PaginationOffset } from './schemas/$PaginationOffset';
|
||||
export { $PresignedURL } from './schemas/$PresignedURL';
|
||||
export { $PresignObjectRequest } from './schemas/$PresignObjectRequest';
|
||||
export { $Project } from './schemas/$Project';
|
||||
export { $ProjectID } from './schemas/$ProjectID';
|
||||
export { $ProjectMember } from './schemas/$ProjectMember';
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
/* generated using openapi-typescript-codegen -- do not edit */
|
||||
/* istanbul ignore file */
|
||||
/* tslint:disable */
|
||||
/* eslint-disable */
|
||||
export type PresignObjectRequest = {
|
||||
/**
|
||||
* Object key inside the bucket.
|
||||
*/
|
||||
key: string;
|
||||
/**
|
||||
* download → GET URL (object must exist); upload → PUT URL (creates/overwrites).
|
||||
*/
|
||||
op?: PresignObjectRequest.op;
|
||||
ttl_seconds?: number;
|
||||
};
|
||||
export namespace PresignObjectRequest {
|
||||
/**
|
||||
* download → GET URL (object must exist); upload → PUT URL (creates/overwrites).
|
||||
*/
|
||||
export enum op {
|
||||
DOWNLOAD = 'download',
|
||||
UPLOAD = 'upload',
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
/* generated using openapi-typescript-codegen -- do not edit */
|
||||
/* istanbul ignore file */
|
||||
/* tslint:disable */
|
||||
/* eslint-disable */
|
||||
export type PresignedURL = {
|
||||
/**
|
||||
* Fully-signed URL — call it with the returned method, no auth header needed.
|
||||
*/
|
||||
url: string;
|
||||
method: PresignedURL.method;
|
||||
expires_at: string;
|
||||
};
|
||||
export namespace PresignedURL {
|
||||
export enum method {
|
||||
GET = 'GET',
|
||||
PUT = 'PUT',
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
/* generated using openapi-typescript-codegen -- do not edit */
|
||||
/* istanbul ignore file */
|
||||
/* tslint:disable */
|
||||
/* eslint-disable */
|
||||
export const $PresignObjectRequest = {
|
||||
properties: {
|
||||
key: {
|
||||
type: 'string',
|
||||
description: `Object key inside the bucket.`,
|
||||
isRequired: true,
|
||||
},
|
||||
op: {
|
||||
type: 'Enum',
|
||||
},
|
||||
ttl_seconds: {
|
||||
type: 'number',
|
||||
maximum: 3600,
|
||||
minimum: 60,
|
||||
},
|
||||
},
|
||||
} as const;
|
||||
@@ -0,0 +1,22 @@
|
||||
/* generated using openapi-typescript-codegen -- do not edit */
|
||||
/* istanbul ignore file */
|
||||
/* tslint:disable */
|
||||
/* eslint-disable */
|
||||
export const $PresignedURL = {
|
||||
properties: {
|
||||
url: {
|
||||
type: 'string',
|
||||
description: `Fully-signed URL — call it with the returned method, no auth header needed.`,
|
||||
isRequired: true,
|
||||
},
|
||||
method: {
|
||||
type: 'Enum',
|
||||
isRequired: true,
|
||||
},
|
||||
expires_at: {
|
||||
type: 'string',
|
||||
isRequired: true,
|
||||
format: 'date-time',
|
||||
},
|
||||
},
|
||||
} as const;
|
||||
@@ -7,6 +7,8 @@ import type { BucketObject } from '../models/BucketObject';
|
||||
import type { BucketObjectListResponse } from '../models/BucketObjectListResponse';
|
||||
import type { CopyBucketObjectRequest } from '../models/CopyBucketObjectRequest';
|
||||
import type { CreateBucketRequest } from '../models/CreateBucketRequest';
|
||||
import type { PresignedURL } from '../models/PresignedURL';
|
||||
import type { PresignObjectRequest } from '../models/PresignObjectRequest';
|
||||
import type { UpdateBucketObjectRequest } from '../models/UpdateBucketObjectRequest';
|
||||
import type { UpdateBucketRequest } from '../models/UpdateBucketRequest';
|
||||
import type { CancelablePromise } from '../core/CancelablePromise';
|
||||
@@ -131,6 +133,31 @@ export class StorageService {
|
||||
mediaType: 'application/json',
|
||||
});
|
||||
}
|
||||
/**
|
||||
* Mint a short-lived URL the client calls against object storage directly — keeps large transfers off the backend. Requires BACKEND_STORAGE_DRIVER=s3; the URL's host comes from S3_PUBLIC_ENDPOINT (or S3_ENDPOINT when unset). Signatures bind the host — a presigned URL is valid only at the endpoint it was minted for.
|
||||
* @returns PresignedURL Presigned URL
|
||||
* @throws ApiError
|
||||
*/
|
||||
public static presignBucketObject({
|
||||
bucketId,
|
||||
requestBody,
|
||||
}: {
|
||||
bucketId: string,
|
||||
requestBody: PresignObjectRequest,
|
||||
}): CancelablePromise<PresignedURL> {
|
||||
return __request(OpenAPI, {
|
||||
method: 'POST',
|
||||
url: '/buckets/{bucketID}/object-presigns',
|
||||
path: {
|
||||
'bucketID': bucketId,
|
||||
},
|
||||
body: requestBody,
|
||||
mediaType: 'application/json',
|
||||
errors: {
|
||||
404: `Bucket or object not found (download)`,
|
||||
},
|
||||
});
|
||||
}
|
||||
/**
|
||||
* @returns void
|
||||
* @throws ApiError
|
||||
|
||||
Reference in new issue
Block a user