TDvorak/SpotifyRecAlg
1
0
Fork 0
mirror of https://github.com/Dvorinka/SpotifyRecAlg.git synced 2026-06-03 20:13:03 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

first commit

Browse Source
This commit is contained in:
Tomas Dvorak
2026-04-13 17:46:58 +02:00
commit 6e8fedf534
234 changed files with 53808 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
apps/backend/.env.example
+13
View File
@@ -0,0 +1,13 @@
APP_ENV=development
APP_VERSION=0.1.0
HTTP_ADDR=:8080
STORE_DRIVER=postgres
DATABASE_URL=postgres://spotify:spotify@localhost:5432/spotifyrec?sslmode=disable
API_KEYS=
SEED_DEMO_DATA=false
REC_CONTENT_WEIGHT=0.44
REC_COLLAB_WEIGHT=0.28
REC_POPULARITY_WEIGHT=0.08
REC_EXPLORATION_WEIGHT=0.20
REC_DIVERSITY_LAMBDA=0.74
apps/backend/Dockerfile
+21
View File
@@ -0,0 +1,21 @@
FROM golang:1.24-bookworm AS build
WORKDIR /src
COPY go.mod go.sum* ./
RUN go mod download
COPY . .
RUN go install github.com/pressly/goose/v3/cmd/goose@v3.24.3
RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/recommendation-api ./cmd/api
FROM gcr.io/distroless/static-debian12:nonroot
WORKDIR /app
COPY --from=build /out/recommendation-api /app/recommendation-api
COPY --from=build /go/bin/goose /app/goose
COPY migrations /app/migrations
COPY docs /app/docs
EXPOSE 8080
ENTRYPOINT ["/app/recommendation-api"]
apps/backend/README.md
+116
View File
@@ -0,0 +1,116 @@
# SpotifyRecAlg Backend
Go recommendation API for music catalogs. It combines the approaches from `project.md` and the included papers:
- content-based exploration over normalized audio features
- weighted Spotify-style audio similarity over fixed feature ranges
- metadata affinity for genre/artist fallback when audio features are missing
- collaborative exploitation using Pearson-style neighborhood scores
- seed-track and manual feature targeting
- explicit user controls for hidden tracks, genres, artists, and explicit content
- popularity dampening, safety penalties, constrained commercial boosts, and diversity reranking
- response explanations so clients can show why a track was recommended
## Authentication Options
**Option 1: Auth-free (default)** - Native Go webplayer client
No Spotify API credentials needed. The backend includes a native webplayer client that generates TOTP tokens (same method as official Web Player) to get anonymous access. No external services required.
```bash
cd apps/backend
STORE_DRIVER=memory SEED_DEMO_DATA=true go run ./cmd/api
```
**Option 2: Official Spotify API** - Set credentials
```bash
export SPOTIFY_CLIENT_ID=...
export SPOTIFY_CLIENT_SECRET=...
cd apps/backend && go run ./cmd/api
```
The backend automatically falls back to the native webplayer client if Spotify credentials are not configured.
## Run Locally
Memory mode, with demo data:
```bash
cd apps/backend
STORE_DRIVER=memory SEED_DEMO_DATA=true go run ./cmd/api
```
Postgres mode:
```bash
docker compose -f infra/docker-compose.yml up postgres -d
cd apps/backend
go install github.com/pressly/goose/v3/cmd/goose@latest
goose -dir migrations postgres "postgres://spotify:spotify@localhost:5432/spotifyrec?sslmode=disable" up
go run ./cmd/api
```
Request recommendations:
```bash
curl -s http://localhost:8080/v1/recommendations \
-H 'Content-Type: application/json' \
-d '{"user_id":"demo-user","limit":5,"mode":"balanced"}'
```
Import one Spotify track (works with unlocker or official API):
```bash
curl -s http://localhost:8080/v1/providers/spotify/import \
-H 'Content-Type: application/json' \
-d '{"source":{"type":"url","value":"https://open.spotify.com/track/3n3Ppam7vgaVa1iaRUc9Lp"}, "market":"US", "enrich_musicbrainz":true, "persist":true}'
```
## API Surface
- `POST /v1/tracks` upsert one track
- `PUT /v1/tracks/batch` upsert up to 1000 tracks
- `POST /v1/interactions` ingest play, skip, like, dislike, save, or hide events
- `POST /v1/recommendations` create explainable ranked recommendations
- `GET /v1/users/{user_id}/taste-profile` inspect the computed profile
- `GET /v1/users/{user_id}/controls` read taste and safety controls
- `PUT /v1/users/{user_id}/controls` update controls
- `POST /v1/providers/spotify/import` import Spotify track, album, playlist, or artist tracks
- `POST /v1/providers/spotify/search` search Spotify tracks with limit capped at 10
- `POST /v1/providers/musicbrainz/enrich` enrich existing tracks by ISRC or title/artist search
- `GET /v1/providers/status` inspect provider configuration, availability, and cache stats
- `GET /healthz` liveness
- `GET /readyz` storage readiness
See `docs/openapi.yaml` for the contract.
## Architecture
The HTTP layer depends on a small storage interface and the recommendation engine depends only on a snapshot provider. That keeps this service wireable to another backend: you can replace Postgres with your own catalog, data lake, event stream, or user service without changing the scorer.
Core scoring:
```text
final =
content_weight * weighted_audio_and_metadata_similarity
+ collaborative_weight * overlap_shrunk_neighbor_score
+ popularity_weight * mode_aware_popularity_fit
+ exploration_weight * target_distance_score
+ constrained_commercial_boost
final *= safety_score * negative_feedback_penalty
```
Candidates are then reranked with a Maximal Marginal Relevance style diversity pass so the top results are not duplicates of the same audio neighborhood.
## Production Notes
- Set `API_KEYS` for backend-to-backend API key protection.
- Set `SPOTIFY_CLIENT_ID` and `SPOTIFY_CLIENT_SECRET` for Spotify client credentials auth, or `SPOTIFY_BEARER_TOKEN` for a short-lived externally managed bearer token.
- Set `SPOTIFY_MARKET` to the default two-letter market, for example `US`.
- Set `MUSICBRAINZ_APP_NAME` and `MUSICBRAINZ_CONTACT`; MusicBrainz requires an identifying User-Agent.
- Set `PROVIDER_CACHE_TTL_HOURS` to control provider payload cache freshness. Expired cache entries may be used as stale fallback when an upstream provider fails.
- Keep user authentication in the parent product and pass stable opaque `user_id` values to this service.
- Run goose migrations before starting Postgres mode.
- Use bulk ingestion for catalog updates and append-only interaction events.
- For large catalogs, replace full snapshots with vector indexes or precomputed candidate sets while keeping the same engine contract.
- When Spotify API credentials are provided, the backend uses the official Web API. Otherwise, it uses the native Go webplayer client which generates TOTP tokens for anonymous access (no user account required).
apps/backend/api
Executable
BIN
View File
Binary file not shown.
apps/backend/cmd/api/main.go
+140
View File
@@ -0,0 +1,140 @@
package main
import (
"context"
"errors"
"log"
"net/http"
"os"
"os/signal"
"syscall"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/config"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/httpapi"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider/musicbrainz"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider/songlink"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider/spotify"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider/webplayer"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/recommendation"
memstore "github.com/tdvorak/spotifyrecalg/apps/backend/internal/storage/memory"
pgstore "github.com/tdvorak/spotifyrecalg/apps/backend/internal/storage/postgres"
"go.uber.org/zap"
)
func main() {
cfg := config.Load()
logger, err := zap.NewProduction()
if cfg.Environment == "development" {
logger, err = zap.NewDevelopment()
}
if err != nil {
log.Fatalf("create logger: %v", err)
}
defer func() { _ = logger.Sync() }()
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
store, cleanup, err := buildStore(ctx, cfg, logger)
if err != nil {
logger.Fatal("initialize storage", zap.Error(err))
}
defer cleanup()
engine := recommendation.NewEngine(recommendation.EngineConfig{
Now: time.Now,
ContentWeight: cfg.ContentWeight,
CollabWeight: cfg.CollaborativeWeight,
PopularityWeight: cfg.PopularityWeight,
ExplorationWeight: cfg.ExplorationWeight,
DiversityLambda: cfg.DiversityLambda,
})
router := httpapi.NewRouter(httpapi.RouterConfig{
Store: store,
Engine: engine,
Provider: buildProviderService(store, cfg),
Logger: logger,
APIKeys: cfg.APIKeys,
Version: cfg.Version,
})
server := &http.Server{
Addr: cfg.HTTPAddr,
Handler: router,
ReadHeaderTimeout: 5 * time.Second,
ReadTimeout: 15 * time.Second,
WriteTimeout: 30 * time.Second,
IdleTimeout: 120 * time.Second,
}
errCh := make(chan error, 1)
go func() {
logger.Info("api listening", zap.String("addr", cfg.HTTPAddr), zap.String("store", cfg.StoreDriver))
errCh <- server.ListenAndServe()
}()
select {
case <-ctx.Done():
shutdownCtx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
defer cancel()
if err := server.Shutdown(shutdownCtx); err != nil {
logger.Error("graceful shutdown failed", zap.Error(err))
}
case err := <-errCh:
if err != nil && !errors.Is(err, http.ErrServerClosed) {
logger.Fatal("server stopped", zap.Error(err))
}
}
}
func buildProviderService(store httpapi.Store, cfg config.Config) *provider.Service {
providerStore, ok := store.(provider.Store)
if !ok {
return nil
}
spotifyClient := spotify.New(spotify.Config{
ClientID: cfg.SpotifyClientID,
ClientSecret: cfg.SpotifyClientSecret,
BearerToken: cfg.SpotifyBearerToken,
Market: cfg.SpotifyMarket,
})
webplayerClient := webplayer.NewClient()
songlinkClient := songlink.NewClient()
musicBrainzClient := musicbrainz.New(musicbrainz.Config{
AppName: cfg.MusicBrainzAppName,
Contact: cfg.MusicBrainzContact,
Version: cfg.Version,
})
return provider.NewService(providerStore, spotifyClient, webplayerClient, songlinkClient, musicBrainzClient, provider.ServiceConfig{
DefaultMarket: cfg.SpotifyMarket,
CacheTTL: cfg.ProviderCacheTTL,
Version: cfg.Version,
})
}
func buildStore(ctx context.Context, cfg config.Config, logger *zap.Logger) (httpapi.Store, func(), error) {
if cfg.StoreDriver == "memory" {
store := memstore.New()
if cfg.SeedDemoData {
memstore.SeedLargeCatalog(store)
}
return store, func() {}, nil
}
pool, err := pgxpool.New(ctx, cfg.DatabaseURL)
if err != nil {
return nil, nil, err
}
if err := pool.Ping(ctx); err != nil {
pool.Close()
return nil, nil, err
}
logger.Info("connected to postgres")
return pgstore.New(pool), pool.Close, nil
}
apps/backend/docs/openapi.yaml
+651
View File
@@ -0,0 +1,651 @@
openapi: 3.1.0
info:
title: SpotifyRecAlg Recommendation API
version: 0.1.0
summary: Explainable hybrid music recommendation API.
license:
name: MIT
identifier: MIT
servers:
- url: https://api.spotifyrec.local
security:
- ApiKeyAuth: []
paths:
/healthz:
get:
summary: Check service liveness.
security: []
tags: [System]
operationId: health
responses:
"200":
description: Service is alive.
"429":
$ref: "#/components/responses/Problem"
/readyz:
get:
summary: Check storage readiness.
security: []
tags: [System]
operationId: ready
responses:
"200":
description: Storage is reachable.
"429":
$ref: "#/components/responses/Problem"
"503":
$ref: "#/components/responses/Problem"
/v1/tracks:
post:
tags: [Catalog]
operationId: upsertTrack
summary: Upsert one catalog track.
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/Track"
responses:
"200":
description: Stored track.
content:
application/json:
schema:
$ref: "#/components/schemas/Track"
"400":
$ref: "#/components/responses/Problem"
"422":
$ref: "#/components/responses/Problem"
/v1/tracks/batch:
put:
tags: [Catalog]
operationId: upsertTracks
summary: Upsert a batch of tracks.
requestBody:
required: true
content:
application/json:
schema:
type: object
required: [tracks]
properties:
tracks:
type: array
maxItems: 1000
items:
$ref: "#/components/schemas/Track"
responses:
"200":
description: Batch stored.
content:
application/json:
schema:
type: object
required: [stored]
properties:
stored:
type: integer
"400":
$ref: "#/components/responses/Problem"
"422":
$ref: "#/components/responses/Problem"
/v1/interactions:
post:
tags: [Events]
operationId: recordInteraction
summary: Append a listening or feedback event.
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/Interaction"
responses:
"202":
description: Event accepted.
"400":
$ref: "#/components/responses/Problem"
"422":
$ref: "#/components/responses/Problem"
/v1/recommendations:
post:
tags: [Recommendations]
operationId: recommend
summary: Get ranked recommendations for a user.
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/RecommendRequest"
examples:
balanced:
value:
user_id: demo-user
limit: 10
mode: balanced
discovery:
value:
user_id: demo-user
limit: 10
seed_track_ids: [trk-neon-dawn]
mode: discovery
exploration_target: 0.35
responses:
"200":
description: Ranked recommendations.
content:
application/json:
schema:
type: object
required: [data, taste_profile, pagination]
properties:
data:
type: array
items:
$ref: "#/components/schemas/Recommendation"
taste_profile:
$ref: "#/components/schemas/TasteProfile"
pagination:
$ref: "#/components/schemas/CursorPage"
"400":
$ref: "#/components/responses/Problem"
"422":
$ref: "#/components/responses/Problem"
/v1/users/{user_id}/taste-profile:
get:
tags: [Users]
operationId: getTasteProfile
summary: Inspect the computed taste profile.
parameters:
- $ref: "#/components/parameters/UserID"
responses:
"200":
description: Taste profile.
content:
application/json:
schema:
$ref: "#/components/schemas/TasteProfile"
"422":
$ref: "#/components/responses/Problem"
/v1/users/{user_id}/controls:
get:
summary: Read user recommendation controls.
tags: [Users]
operationId: getUserControls
parameters:
- $ref: "#/components/parameters/UserID"
responses:
"200":
description: User controls.
content:
application/json:
schema:
$ref: "#/components/schemas/UserControls"
"429":
$ref: "#/components/responses/Problem"
put:
summary: Replace user recommendation controls.
tags: [Users]
operationId: upsertUserControls
parameters:
- $ref: "#/components/parameters/UserID"
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/UserControls"
responses:
"200":
description: Stored controls.
content:
application/json:
schema:
$ref: "#/components/schemas/UserControls"
"400":
$ref: "#/components/responses/Problem"
/v1/providers/spotify/import:
post:
tags: [Providers]
operationId: importSpotify
summary: Import Spotify catalog data into the recommendation catalog.
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/SpotifyImportRequest"
responses:
"200":
description: Import summary.
content:
application/json:
schema:
$ref: "#/components/schemas/ProviderImportResponse"
"400":
$ref: "#/components/responses/Problem"
"422":
$ref: "#/components/responses/Problem"
"502":
$ref: "#/components/responses/Problem"
"503":
$ref: "#/components/responses/Problem"
/v1/providers/spotify/search:
post:
tags: [Providers]
operationId: searchSpotify
summary: Search Spotify tracks and optionally persist mapped results.
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/SpotifySearchRequest"
responses:
"200":
description: Search results.
content:
application/json:
schema:
$ref: "#/components/schemas/SpotifySearchResponse"
"400":
$ref: "#/components/responses/Problem"
"422":
$ref: "#/components/responses/Problem"
"502":
$ref: "#/components/responses/Problem"
"503":
$ref: "#/components/responses/Problem"
/v1/providers/musicbrainz/enrich:
post:
tags: [Providers]
operationId: enrichMusicBrainz
summary: Enrich existing tracks with MusicBrainz recording metadata.
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/MusicBrainzEnrichRequest"
responses:
"200":
description: Enrichment summary.
content:
application/json:
schema:
$ref: "#/components/schemas/MusicBrainzEnrichResponse"
"400":
$ref: "#/components/responses/Problem"
"422":
$ref: "#/components/responses/Problem"
"502":
$ref: "#/components/responses/Problem"
"503":
$ref: "#/components/responses/Problem"
/v1/providers/status:
get:
tags: [Providers]
operationId: getProviderStatus
summary: Report provider configuration, availability, and cache status.
responses:
"200":
description: Provider status.
content:
application/json:
schema:
$ref: "#/components/schemas/ProviderStatusResponse"
"401":
$ref: "#/components/responses/Problem"
"503":
$ref: "#/components/responses/Problem"
components:
securitySchemes:
ApiKeyAuth:
type: apiKey
in: header
name: X-API-Key
description: Optional backend-to-backend API key. Disabled when API_KEYS is empty.
parameters:
UserID:
name: user_id
in: path
required: true
schema:
type: string
minLength: 1
responses:
Problem:
description: RFC 7807 problem response.
content:
application/problem+json:
schema:
$ref: "#/components/schemas/Problem"
schemas:
AudioFeatures:
type: object
required:
- danceability
- energy
- loudness
- speechiness
- acousticness
- instrumentalness
- liveness
- valence
- tempo
- time_signature
properties:
danceability: { type: number, minimum: 0, maximum: 1 }
energy: { type: number, minimum: 0, maximum: 1 }
loudness: { type: number, description: Decibels before service-side normalization. }
speechiness: { type: number, minimum: 0, maximum: 1 }
acousticness: { type: number, minimum: 0, maximum: 1 }
instrumentalness: { type: number, minimum: 0, maximum: 1 }
liveness: { type: number, minimum: 0, maximum: 1 }
valence: { type: number, minimum: 0, maximum: 1 }
tempo: { type: number, minimum: 0 }
time_signature: { type: number, minimum: 0 }
key: { type: number }
mode: { type: number }
Track:
type: object
required: [id, title, artist, popularity, explicit, features]
properties:
id: { type: string }
title: { type: string }
artist: { type: string }
album: { type: string }
genres:
type: array
items: { type: string }
release_date: { type: string }
duration_ms: { type: integer, minimum: 0 }
popularity: { type: number, minimum: 0, maximum: 1 }
explicit: { type: boolean }
features:
$ref: "#/components/schemas/AudioFeatures"
external:
type: object
additionalProperties: { type: string }
commercial_boost: { type: number, minimum: 0, maximum: 1 }
quality_penalty: { type: number, minimum: 0, maximum: 1 }
discovery_allowed: { type: boolean }
Context:
type: object
properties:
locale: { type: string }
device: { type: string }
time_of_day: { type: string }
activity: { type: string }
mood: { type: string }
Interaction:
type: object
required: [user_id, track_id, type]
properties:
user_id: { type: string }
track_id: { type: string }
type:
type: string
enum: [play, skip, like, dislike, save, hide]
weight:
type: number
description: Optional override. Leave zero for default behavior weighting.
occurred_at:
type: string
format: date-time
completed_ms:
type: integer
minimum: 0
context:
$ref: "#/components/schemas/Context"
RecommendRequest:
type: object
required: [user_id]
properties:
user_id: { type: string }
limit: { type: integer, minimum: 1, maximum: 100, default: 20 }
seed_track_ids:
type: array
items: { type: string }
feature_targets:
$ref: "#/components/schemas/AudioFeatures"
context:
$ref: "#/components/schemas/Context"
mode:
type: string
enum: [balanced, comfort, discovery]
default: balanced
exploration_target:
type: number
minimum: 0
maximum: 1
min_popularity: { type: number, minimum: 0, maximum: 1 }
max_popularity: { type: number, minimum: 0, maximum: 1 }
include_explicit: { type: boolean }
excluded_track_ids:
type: array
items: { type: string }
excluded_artist_ids:
type: array
items: { type: string }
excluded_genres:
type: array
items: { type: string }
Recommendation:
type: object
required: [track, score, rank, reason, score_breakdown, explanation]
properties:
track:
$ref: "#/components/schemas/Track"
score: { type: number }
rank: { type: integer }
reason: { type: string }
score_breakdown:
$ref: "#/components/schemas/ScoreBreakdown"
explanation:
type: object
additionalProperties: { type: number }
ScoreBreakdown:
type: object
properties:
content: { type: number }
collaborative: { type: number }
popularity: { type: number }
exploration: { type: number }
diversity: { type: number }
safety: { type: number }
commercial: { type: number }
final: { type: number }
TasteProfile:
type: object
required: [user_id, vector, top_genres, interaction_count, confidence, exploration_readiness, updated_at]
properties:
user_id: { type: string }
vector:
type: array
items: { type: number }
top_genres:
type: object
additionalProperties: { type: number }
interaction_count: { type: integer }
confidence: { type: number }
exploration_readiness: { type: number }
updated_at:
type: string
format: date-time
UserControls:
type: object
properties:
user_id: { type: string }
allow_explicit: { type: boolean, default: true }
excluded_tracks:
type: array
items: { type: string }
excluded_artists:
type: array
items: { type: string }
excluded_genres:
type: array
items: { type: string }
postponed_tracks:
type: array
items: { type: string }
ProviderSource:
type: object
required: [type, value]
properties:
type:
type: string
enum: [track, album, playlist, artist, url]
value:
type: string
SpotifyImportRequest:
type: object
required: [source]
properties:
source:
$ref: "#/components/schemas/ProviderSource"
market:
type: string
minLength: 2
maxLength: 2
default: US
limit:
type: integer
minimum: 1
maximum: 100
default: 100
enrich_musicbrainz:
type: boolean
default: true
persist:
type: boolean
default: true
allow_missing_features:
type: boolean
default: false
SpotifySearchRequest:
type: object
required: [query]
properties:
query:
type: string
minLength: 1
type:
type: string
enum: [track, album, artist, playlist]
default: track
market:
type: string
minLength: 2
maxLength: 2
default: US
limit:
type: integer
minimum: 1
maximum: 10
default: 5
persist:
type: boolean
default: false
enrich_musicbrainz:
type: boolean
default: true
allow_missing_features:
type: boolean
default: false
ProviderImportResponse:
type: object
required: [import_id, imported_tracks, updated_tracks, skipped, warnings]
properties:
import_id: { type: string }
imported_tracks: { type: integer, minimum: 0 }
updated_tracks: { type: integer, minimum: 0 }
skipped: { type: integer, minimum: 0 }
warnings:
type: array
items: { type: string }
SpotifySearchResponse:
type: object
required: [tracks, persisted, skipped, warnings]
properties:
tracks:
type: array
items:
$ref: "#/components/schemas/Track"
persisted: { type: integer, minimum: 0 }
skipped: { type: integer, minimum: 0 }
warnings:
type: array
items: { type: string }
MusicBrainzEnrichRequest:
type: object
required: [track_ids]
properties:
track_ids:
type: array
minItems: 1
items: { type: string }
force:
type: boolean
default: false
MusicBrainzEnrichResponse:
type: object
required: [updated, skipped, warnings]
properties:
updated: { type: integer, minimum: 0 }
skipped: { type: integer, minimum: 0 }
warnings:
type: array
items: { type: string }
ProviderStatusResponse:
type: object
required: [spotify, musicbrainz, cache]
properties:
spotify:
$ref: "#/components/schemas/ProviderStatus"
musicbrainz:
$ref: "#/components/schemas/ProviderStatus"
cache:
$ref: "#/components/schemas/ProviderCacheStats"
ProviderStatus:
type: object
required: [configured, available, checked_at]
properties:
configured: { type: boolean }
token_mode:
type: string
enum: [client_credentials, static_bearer, user_agent, unconfigured]
available: { type: boolean }
last_error: { type: string }
checked_at:
type: string
format: date-time
ProviderCacheStats:
type: object
required: [entries, fresh_entries, stale_entries]
properties:
entries: { type: integer, minimum: 0 }
fresh_entries: { type: integer, minimum: 0 }
stale_entries: { type: integer, minimum: 0 }
CursorPage:
type: object
required: [has_more]
properties:
next_cursor:
type:
- string
- "null"
has_more:
type: boolean
Problem:
type: object
required: [type, title, status]
properties:
type: { type: string, format: uri }
title: { type: string }
status: { type: integer }
detail: { type: string }
instance: { type: string }
apps/backend/go.mod
+47
View File
@@ -0,0 +1,47 @@
module github.com/tdvorak/spotifyrecalg/apps/backend
go 1.24
require (
github.com/gin-gonic/gin v1.11.0
github.com/jackc/pgx/v5 v5.7.6
go.uber.org/zap v1.27.0
)
require (
github.com/bytedance/sonic v1.14.0 // indirect
github.com/bytedance/sonic/loader v0.3.0 // indirect
github.com/cloudwego/base64x v0.1.6 // indirect
github.com/gabriel-vasile/mimetype v1.4.8 // indirect
github.com/gin-contrib/sse v1.1.0 // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.27.0 // indirect
github.com/goccy/go-json v0.10.2 // indirect
github.com/goccy/go-yaml v1.18.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421 // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
github.com/quic-go/qpack v0.5.1 // indirect
github.com/quic-go/quic-go v0.54.0 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/ugorji/go/codec v1.3.0 // indirect
go.uber.org/mock v0.5.0 // indirect
go.uber.org/multierr v1.10.0 // indirect
golang.org/x/arch v0.20.0 // indirect
golang.org/x/crypto v0.40.0 // indirect
golang.org/x/mod v0.25.0 // indirect
golang.org/x/net v0.42.0 // indirect
golang.org/x/sync v0.16.0 // indirect
golang.org/x/sys v0.35.0 // indirect
golang.org/x/text v0.27.0 // indirect
golang.org/x/tools v0.34.0 // indirect
google.golang.org/protobuf v1.36.9 // indirect
)
apps/backend/go.sum
+103
View File
@@ -0,0 +1,103 @@
github.com/bytedance/sonic v1.14.0 h1:/OfKt8HFw0kh2rj8N0F6C/qPGRESq0BbaNZgcNXXzQQ=
github.com/bytedance/sonic v1.14.0/go.mod h1:WoEbx8WTcFJfzCe0hbmyTGrfjt8PzNEBdxlNUO24NhA=
github.com/bytedance/sonic/loader v0.3.0 h1:dskwH8edlzNMctoruo8FPTJDF3vLtDT0sXZwvZJyqeA=
github.com/bytedance/sonic/loader v0.3.0/go.mod h1:N8A3vUdtUebEY2/VQC0MyhYeKUFosQU6FxH2JmUe6VI=
github.com/cloudwego/base64x v0.1.6 h1:t11wG9AECkCDk5fMSoxmufanudBtJ+/HemLstXDLI2M=
github.com/cloudwego/base64x v0.1.6/go.mod h1:OFcloc187FXDaYHvrNIjxSe8ncn0OOM8gEHfghB2IPU=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/gabriel-vasile/mimetype v1.4.8 h1:FfZ3gj38NjllZIeJAmMhr+qKL8Wu+nOoI3GqacKw1NM=
github.com/gabriel-vasile/mimetype v1.4.8/go.mod h1:ByKUIKGjh1ODkGM1asKUbQZOLGrPjydw3hYPU2YU9t8=
github.com/gin-contrib/sse v1.1.0 h1:n0w2GMuUpWDVp7qSpvze6fAu9iRxJY4Hmj6AmBOU05w=
github.com/gin-contrib/sse v1.1.0/go.mod h1:hxRZ5gVpWMT7Z0B0gSNYqqsSCNIJMjzvm6fqCz9vjwM=
github.com/gin-gonic/gin v1.11.0 h1:OW/6PLjyusp2PPXtyxKHU0RbX6I/l28FTdDlae5ueWk=
github.com/gin-gonic/gin v1.11.0/go.mod h1:+iq/FyxlGzII0KHiBGjuNn4UNENUlKbGlNmc+W50Dls=
github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s=
github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY=
github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY=
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
github.com/go-playground/validator/v10 v10.27.0 h1:w8+XrWVMhGkxOaaowyKH35gFydVHOvC0/uWoy2Fzwn4=
github.com/go-playground/validator/v10 v10.27.0/go.mod h1:I5QpIEbmr8On7W0TktmJAumgzX4CA1XNl4ZmDuVHKKo=
github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU=
github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
github.com/goccy/go-yaml v1.18.0 h1:8W7wMFS12Pcas7KU+VVkaiCng+kG8QiFeFwzFb+rwuw=
github.com/goccy/go-yaml v1.18.0/go.mod h1:XBurs7gK8ATbW4ZPGKgcbrY1Br56PdM69F7LkFRi1kA=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgx/v5 v5.7.6 h1:rWQc5FwZSPX58r1OQmkuaNicxdmExaEz5A2DO2hUuTk=
github.com/jackc/pgx/v5 v5.7.6/go.mod h1:aruU7o91Tc2q2cFp5h4uP3f6ztExVpyVv88Xl/8Vl8M=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421 h1:ZqeYNhU3OHLH3mGKHDcjJRFFRrJa6eAM5H+CtDdOsPc=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/quic-go/qpack v0.5.1 h1:giqksBPnT/HDtZ6VhtFKgoLOWmlyo9Ei6u9PqzIMbhI=
github.com/quic-go/qpack v0.5.1/go.mod h1:+PC4XFrEskIVkcLzpEkbLqq1uCoxPhQuvK5rH1ZgaEg=
github.com/quic-go/quic-go v0.54.0 h1:6s1YB9QotYI6Ospeiguknbp2Znb/jZYjZLRXn9kMQBg=
github.com/quic-go/quic-go v0.54.0/go.mod h1:e68ZEaCdyviluZmy44P6Iey98v/Wfz6HCjQEm+l8zTY=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
github.com/ugorji/go/codec v1.3.0 h1:Qd2W2sQawAfG8XSvzwhBeoGq71zXOC/Q1E9y/wUcsUA=
github.com/ugorji/go/codec v1.3.0/go.mod h1:pRBVtBSKl77K30Bv8R2P+cLSGaTtex6fsA2Wjqmfxj4=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.uber.org/mock v0.5.0 h1:KAMbZvZPyBPWgD14IrIQ38QCyjwpvVVV6K/bHl1IwQU=
go.uber.org/mock v0.5.0/go.mod h1:ge71pBPLYDk7QIi1LupWxdAykm7KIEFchiOqd6z7qMM=
go.uber.org/multierr v1.10.0 h1:S0h4aNzvfcFsC3dRF1jLoaov7oRaKqRGC/pUEJ2yvPQ=
go.uber.org/multierr v1.10.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8=
go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
golang.org/x/arch v0.20.0 h1:dx1zTU0MAE98U+TQ8BLl7XsJbgze2WnNKF/8tGp/Q6c=
golang.org/x/arch v0.20.0/go.mod h1:bdwinDaKcfZUGpH09BB7ZmOfhalA8lQdzl62l8gGWsk=
golang.org/x/crypto v0.40.0 h1:r4x+VvoG5Fm+eJcxMaY8CQM7Lb0l1lsmjGBQ6s8BfKM=
golang.org/x/crypto v0.40.0/go.mod h1:Qr1vMER5WyS2dfPHAlsOj01wgLbsyWtFn/aY+5+ZdxY=
golang.org/x/mod v0.25.0 h1:n7a+ZbQKQA/Ysbyb0/6IbB1H/X41mKgbhfv7AfG/44w=
golang.org/x/mod v0.25.0/go.mod h1:IXM97Txy2VM4PJ3gI61r1YEk/gAj6zAHN3AdZt6S9Ww=
golang.org/x/net v0.42.0 h1:jzkYrhi3YQWD6MLBJcsklgQsoAcw89EcZbJw8Z614hs=
golang.org/x/net v0.42.0/go.mod h1:FF1RA5d3u7nAYA4z2TkclSCKh68eSXtiFwcWQpPXdt8=
golang.org/x/sync v0.16.0 h1:ycBJEhp9p4vXvUZNszeOq0kGTPghopOL8q0fq3vstxw=
golang.org/x/sync v0.16.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.35.0 h1:vz1N37gP5bs89s7He8XuIYXpyY0+QlsKmzipCbUtyxI=
golang.org/x/sys v0.35.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/text v0.27.0 h1:4fGWRpyh641NLlecmyl4LOe6yDdfaYNrGb2zdfo4JV4=
golang.org/x/text v0.27.0/go.mod h1:1D28KMCvyooCX9hBiosv5Tz/+YLxj0j7XhWjpSUF7CU=
golang.org/x/tools v0.34.0 h1:qIpSLOxeCYGg9TrcJokLBG4KFA6d795g0xkBkiESGlo=
golang.org/x/tools v0.34.0/go.mod h1:pAP9OwEaY1CAW3HOmg3hLZC5Z0CCmzjAF2UQMSqNARg=
google.golang.org/protobuf v1.36.9 h1:w2gp2mA27hUeUzj9Ex9FBjsBm40zfaDtEWow293U7Iw=
google.golang.org/protobuf v1.36.9/go.mod h1:fuxRtAxBytpl4zzqUh6/eyUujkJdNiuEkXntxiD/uRU=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
apps/backend/internal/config/config.go
+115
View File
@@ -0,0 +1,115 @@
package config
import (
"os"
"strconv"
"strings"
"time"
)
type Config struct {
Environment string
Version string
HTTPAddr string
StoreDriver string
DatabaseURL string
APIKeys []string
SeedDemoData bool
ContentWeight float64
CollaborativeWeight float64
PopularityWeight float64
ExplorationWeight float64
DiversityLambda float64
SpotifyClientID string
SpotifyClientSecret string
SpotifyBearerToken string
SpotifyMarket string
UnlockerURL string
MusicBrainzAppName string
MusicBrainzContact string
ProviderCacheTTL time.Duration
}
func Load() Config {
return Config{
Environment: env("APP_ENV", "development"),
Version: env("APP_VERSION", "0.1.0"),
HTTPAddr: env("HTTP_ADDR", ":8080"),
StoreDriver: env("STORE_DRIVER", "postgres"),
DatabaseURL: env("DATABASE_URL", "postgres://spotify:spotify@localhost:5432/spotifyrec?sslmode=disable"),
APIKeys: csv(env("API_KEYS", "")),
SeedDemoData: boolEnv("SEED_DEMO_DATA", false),
ContentWeight: floatEnv("REC_CONTENT_WEIGHT", 0.44),
CollaborativeWeight: floatEnv("REC_COLLAB_WEIGHT", 0.28),
PopularityWeight: floatEnv("REC_POPULARITY_WEIGHT", 0.08),
ExplorationWeight: floatEnv("REC_EXPLORATION_WEIGHT", 0.20),
DiversityLambda: floatEnv("REC_DIVERSITY_LAMBDA", 0.74),
SpotifyClientID: env("SPOTIFY_CLIENT_ID", ""),
SpotifyClientSecret: env("SPOTIFY_CLIENT_SECRET", ""),
SpotifyBearerToken: env("SPOTIFY_BEARER_TOKEN", ""),
SpotifyMarket: env("SPOTIFY_MARKET", "US"),
MusicBrainzAppName: env("MUSICBRAINZ_APP_NAME", "SpotifyRecAlg"),
MusicBrainzContact: env("MUSICBRAINZ_CONTACT", ""),
ProviderCacheTTL: time.Duration(intEnv("PROVIDER_CACHE_TTL_HOURS", 24)) * time.Hour,
UnlockerURL: env("UNLOCKER_URL", "http://localhost:5000"),
}
}
func env(key, fallback string) string {
value := strings.TrimSpace(os.Getenv(key))
if value == "" {
return fallback
}
return value
}
func csv(value string) []string {
if strings.TrimSpace(value) == "" {
return nil
}
parts := strings.Split(value, ",")
out := make([]string, 0, len(parts))
for _, part := range parts {
part = strings.TrimSpace(part)
if part != "" {
out = append(out, part)
}
}
return out
}
func boolEnv(key string, fallback bool) bool {
raw := strings.TrimSpace(os.Getenv(key))
if raw == "" {
return fallback
}
value, err := strconv.ParseBool(raw)
if err != nil {
return fallback
}
return value
}
func floatEnv(key string, fallback float64) float64 {
raw := strings.TrimSpace(os.Getenv(key))
if raw == "" {
return fallback
}
value, err := strconv.ParseFloat(raw, 64)
if err != nil {
return fallback
}
return value
}
func intEnv(key string, fallback int) int {
raw := strings.TrimSpace(os.Getenv(key))
if raw == "" {
return fallback
}
value, err := strconv.Atoi(raw)
if err != nil {
return fallback
}
return value
}
apps/backend/internal/httpapi/middleware.go
+100
View File
@@ -0,0 +1,100 @@
package httpapi
import (
"crypto/rand"
"encoding/hex"
"net/http"
"slices"
"time"
"github.com/gin-gonic/gin"
"go.uber.org/zap"
)
const requestIDKey = "request_id"
func requestID() gin.HandlerFunc {
return func(c *gin.Context) {
id := c.GetHeader("X-Request-ID")
if id == "" {
id = newRequestID()
}
c.Set(requestIDKey, id)
c.Header("X-Request-ID", id)
c.Next()
}
}
func accessLog(logger *zap.Logger) gin.HandlerFunc {
return func(c *gin.Context) {
start := time.Now()
c.Next()
logger.Info("http request",
zap.String("method", c.Request.Method),
zap.String("path", c.FullPath()),
zap.Int("status", c.Writer.Status()),
zap.Duration("duration", time.Since(start)),
zap.String("request_id", requestIDFromContext(c)),
)
}
}
func recovery(logger *zap.Logger) gin.HandlerFunc {
return gin.CustomRecovery(func(c *gin.Context, recovered any) {
logger.Error("panic recovered", zap.Any("panic", recovered), zap.String("request_id", requestIDFromContext(c)))
problem(c, http.StatusInternalServerError, "https://spotify-rec.local/errors/internal", "Internal server error", "The server encountered an unexpected error.")
})
}
func apiKeyAuth(keys []string) gin.HandlerFunc {
return func(c *gin.Context) {
if len(keys) == 0 || c.Request.URL.Path == "/healthz" || c.Request.URL.Path == "/readyz" {
c.Next()
return
}
key := c.GetHeader("X-API-Key")
if !slices.Contains(keys, key) {
problem(c, http.StatusUnauthorized, "https://spotify-rec.local/errors/unauthorized", "Unauthorized", "A valid X-API-Key header is required.")
c.Abort()
return
}
c.Next()
}
}
func newRequestID() string {
var b [16]byte
if _, err := rand.Read(b[:]); err != nil {
return time.Now().UTC().Format("20060102150405.000000000")
}
return hex.EncodeToString(b[:])
}
func requestIDFromContext(c *gin.Context) string {
value, ok := c.Get(requestIDKey)
if !ok {
return ""
}
id, _ := value.(string)
return id
}
func cors() gin.HandlerFunc {
return func(c *gin.Context) {
origin := c.GetHeader("Origin")
if origin == "" {
origin = "*"
}
c.Header("Access-Control-Allow-Origin", origin)
c.Header("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE, OPTIONS")
c.Header("Access-Control-Allow-Headers", "Content-Type, X-API-Key, X-Request-ID")
c.Header("Access-Control-Allow-Credentials", "true")
c.Header("Access-Control-Max-Age", "86400")
if c.Request.Method == "OPTIONS" {
c.AbortWithStatus(http.StatusNoContent)
return
}
c.Next()
}
}
apps/backend/internal/httpapi/problem.go
+33
View File
@@ -0,0 +1,33 @@
package httpapi
import (
"net/http"
"github.com/gin-gonic/gin"
)
type Problem struct {
Type string `json:"type"`
Title string `json:"title"`
Status int `json:"status"`
Detail string `json:"detail,omitempty"`
Instance string `json:"instance,omitempty"`
}
func problem(c *gin.Context, status int, problemType, title, detail string) {
if c.Writer.Written() {
return
}
c.Header("Content-Type", "application/problem+json")
c.JSON(status, Problem{
Type: problemType,
Title: title,
Status: status,
Detail: detail,
Instance: c.Request.URL.Path,
})
}
func notFound(c *gin.Context) {
problem(c, http.StatusNotFound, "https://spotify-rec.local/errors/not-found", "Not found", "The requested resource does not exist.")
}
apps/backend/internal/httpapi/provider_test.go
+87
View File
@@ -0,0 +1,87 @@
package httpapi
import (
"bytes"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider/musicbrainz"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider/songlink"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider/spotify"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider/webplayer"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/recommendation"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/storage/memory"
"go.uber.org/zap"
)
func TestSpotifyImportEndpoint(t *testing.T) {
store := memory.New()
spotifyServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
switch r.URL.Path {
case "/v1/tracks/imported":
_ = json.NewEncoder(w).Encode(map[string]any{
"id": "imported",
"name": "Imported",
"artists": []map[string]string{{"name": "Artist"}},
"album": map[string]any{"name": "Album", "release_date": "2025-01-01"},
"duration_ms": 180000,
"popularity": 60,
"external_ids": map[string]string{"isrc": "USRC17607839"},
})
case "/v1/audio-features/imported":
_ = json.NewEncoder(w).Encode(map[string]any{
"danceability": 0.6, "energy": 0.7, "loudness": -6, "speechiness": 0.05,
"acousticness": 0.2, "instrumentalness": 0, "liveness": 0.1, "valence": 0.5,
"tempo": 110, "time_signature": 4, "key": 5, "mode": 1,
})
default:
http.NotFound(w, r)
}
}))
defer spotifyServer.Close()
service := provider.NewService(store,
spotify.New(spotify.Config{BearerToken: "secret-token", APIBaseURL: spotifyServer.URL + "/v1"}),
webplayer.NewClient(),
songlink.NewClient(),
musicbrainz.New(musicbrainz.Config{AppName: "SpotifyRecAlg", Contact: "test@example.com", BaseURL: "http://127.0.0.1:1", MinDelay: time.Nanosecond}),
provider.ServiceConfig{DefaultMarket: "US", CacheTTL: time.Hour},
)
router := NewRouter(RouterConfig{
Store: store,
Engine: recommendation.NewEngine(recommendation.EngineConfig{}),
Provider: service,
Logger: zap.NewNop(),
})
body := bytes.NewBufferString(`{"source":{"type":"url","value":"https://open.spotify.com/track/imported"},"market":"US","persist":true}`)
req := httptest.NewRequest(http.MethodPost, "/v1/providers/spotify/import", body)
rec := httptest.NewRecorder()
router.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d body=%s", rec.Code, rec.Body.String())
}
var resp provider.ImportResponse
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode response: %v", err)
}
if resp.ImportedTracks != 1 {
t.Fatalf("imported tracks = %d, want 1", resp.ImportedTracks)
}
req = httptest.NewRequest(http.MethodGet, "/v1/providers/status", nil)
rec = httptest.NewRecorder()
router.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status endpoint = %d body=%s", rec.Code, rec.Body.String())
}
if bytes.Contains(rec.Body.Bytes(), []byte("secret-token")) {
t.Fatal("status response leaked bearer token")
}
}
apps/backend/internal/httpapi/router.go
+307
View File
@@ -0,0 +1,307 @@
package httpapi
import (
"context"
"errors"
"net/http"
"strconv"
"strings"
"github.com/gin-gonic/gin"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/recommendation"
"go.uber.org/zap"
)
type Store interface {
recommendation.SnapshotProvider
Ping(ctx context.Context) error
UpsertTrack(ctx context.Context, track recommendation.Track) error
UpsertTracks(ctx context.Context, tracks []recommendation.Track) error
RecordInteraction(ctx context.Context, interaction recommendation.Interaction) error
GetControls(ctx context.Context, userID string) (recommendation.UserControls, error)
UpsertControls(ctx context.Context, controls recommendation.UserControls) error
}
type RouterConfig struct {
Store Store
Engine *recommendation.Engine
Provider *provider.Service
Logger *zap.Logger
APIKeys []string
Version string
}
func NewRouter(cfg RouterConfig) http.Handler {
gin.SetMode(gin.ReleaseMode)
router := gin.New()
router.Use(recovery(cfg.Logger), cors(), requestID(), accessLog(cfg.Logger), apiKeyAuth(cfg.APIKeys))
handler := handler{
store: cfg.Store,
engine: cfg.Engine,
provider: cfg.Provider,
logger: cfg.Logger,
version: cfg.Version,
}
router.GET("/healthz", handler.health)
router.GET("/readyz", handler.ready)
v1 := router.Group("/v1")
v1.GET("/openapi.yaml", handler.openapi)
v1.POST("/tracks", handler.upsertTrack)
v1.PUT("/tracks/batch", handler.upsertTracks)
v1.POST("/interactions", handler.recordInteraction)
v1.POST("/recommendations", handler.recommend)
v1.GET("/users/:user_id/taste-profile", handler.tasteProfile)
v1.GET("/users/:user_id/controls", handler.getControls)
v1.PUT("/users/:user_id/controls", handler.upsertControls)
v1.POST("/providers/spotify/import", handler.importSpotify)
v1.POST("/providers/spotify/search", handler.searchSpotify)
v1.POST("/providers/musicbrainz/enrich", handler.enrichMusicBrainz)
v1.GET("/providers/status", handler.providerStatus)
return router
}
type handler struct {
store Store
engine *recommendation.Engine
provider *provider.Service
logger *zap.Logger
version string
}
func (h handler) health(c *gin.Context) {
c.JSON(http.StatusOK, gin.H{"status": "ok", "version": h.version})
}
func (h handler) ready(c *gin.Context) {
if err := h.store.Ping(c.Request.Context()); err != nil {
problem(c, http.StatusServiceUnavailable, "https://spotify-rec.local/errors/storage-unavailable", "Storage unavailable", err.Error())
return
}
c.JSON(http.StatusOK, gin.H{"status": "ready"})
}
func (h handler) openapi(c *gin.Context) {
c.File("docs/openapi.yaml")
}
func (h handler) upsertTrack(c *gin.Context) {
var req recommendation.Track
if err := c.ShouldBindJSON(&req); err != nil {
problem(c, http.StatusBadRequest, "https://spotify-rec.local/errors/invalid-json", "Invalid JSON", err.Error())
return
}
if err := recommendation.ValidateTrack(req); err != nil {
problem(c, http.StatusUnprocessableEntity, "https://spotify-rec.local/errors/validation", "Validation failed", err.Error())
return
}
if err := h.store.UpsertTrack(c.Request.Context(), req); err != nil {
h.logger.Error("upsert track", zap.Error(err))
problem(c, http.StatusInternalServerError, "https://spotify-rec.local/errors/storage-write", "Storage write failed", "Track could not be stored.")
return
}
c.JSON(http.StatusOK, req)
}
func (h handler) upsertTracks(c *gin.Context) {
var req struct {
Tracks []recommendation.Track `json:"tracks" binding:"required"`
}
if err := c.ShouldBindJSON(&req); err != nil {
problem(c, http.StatusBadRequest, "https://spotify-rec.local/errors/invalid-json", "Invalid JSON", err.Error())
return
}
if len(req.Tracks) == 0 {
problem(c, http.StatusUnprocessableEntity, "https://spotify-rec.local/errors/validation", "Validation failed", "tracks must contain at least one item")
return
}
if len(req.Tracks) > 1000 {
problem(c, http.StatusUnprocessableEntity, "https://spotify-rec.local/errors/validation", "Validation failed", "batch limit is 1000 tracks")
return
}
for i, track := range req.Tracks {
if err := recommendation.ValidateTrack(track); err != nil {
problem(c, http.StatusUnprocessableEntity, "https://spotify-rec.local/errors/validation", "Validation failed", "tracks["+strconv.Itoa(i)+"]: "+err.Error())
return
}
}
if err := h.store.UpsertTracks(c.Request.Context(), req.Tracks); err != nil {
h.logger.Error("upsert tracks", zap.Error(err))
problem(c, http.StatusInternalServerError, "https://spotify-rec.local/errors/storage-write", "Storage write failed", "Tracks could not be stored.")
return
}
c.JSON(http.StatusOK, gin.H{"stored": len(req.Tracks)})
}
func (h handler) recordInteraction(c *gin.Context) {
var req recommendation.Interaction
if err := c.ShouldBindJSON(&req); err != nil {
problem(c, http.StatusBadRequest, "https://spotify-rec.local/errors/invalid-json", "Invalid JSON", err.Error())
return
}
if strings.TrimSpace(req.UserID) == "" || strings.TrimSpace(req.TrackID) == "" || strings.TrimSpace(string(req.Type)) == "" {
problem(c, http.StatusUnprocessableEntity, "https://spotify-rec.local/errors/validation", "Validation failed", "user_id, track_id, and type are required")
return
}
if err := h.store.RecordInteraction(c.Request.Context(), req); err != nil {
h.logger.Error("record interaction", zap.Error(err))
problem(c, http.StatusInternalServerError, "https://spotify-rec.local/errors/storage-write", "Storage write failed", "Interaction could not be stored.")
return
}
c.JSON(http.StatusAccepted, gin.H{"accepted": true})
}
func (h handler) recommend(c *gin.Context) {
var req recommendation.RecommendRequest
if err := c.ShouldBindJSON(&req); err != nil {
problem(c, http.StatusBadRequest, "https://spotify-rec.local/errors/invalid-json", "Invalid JSON", err.Error())
return
}
recs, profile, err := h.engine.Recommend(c.Request.Context(), h.store, req)
if err != nil {
switch {
case errors.Is(err, context.Canceled):
return
case strings.Contains(err.Error(), "required"), strings.Contains(err.Error(), "empty"):
problem(c, http.StatusUnprocessableEntity, "https://spotify-rec.local/errors/validation", "Validation failed", err.Error())
default:
h.logger.Error("recommend", zap.Error(err))
problem(c, http.StatusInternalServerError, "https://spotify-rec.local/errors/recommendation-failed", "Recommendation failed", "The recommendation engine could not complete the request.")
}
return
}
c.JSON(http.StatusOK, gin.H{
"data": recs,
"taste_profile": profile,
"pagination": gin.H{"next_cursor": nil, "has_more": false},
})
}
func (h handler) tasteProfile(c *gin.Context) {
userID := c.Param("user_id")
profile, err := h.engine.TasteProfile(c.Request.Context(), h.store, userID)
if err != nil {
problem(c, http.StatusUnprocessableEntity, "https://spotify-rec.local/errors/profile-unavailable", "Taste profile unavailable", err.Error())
return
}
c.JSON(http.StatusOK, profile)
}
func (h handler) getControls(c *gin.Context) {
controls, err := h.store.GetControls(c.Request.Context(), c.Param("user_id"))
if err != nil {
h.logger.Error("get controls", zap.Error(err))
problem(c, http.StatusInternalServerError, "https://spotify-rec.local/errors/storage-read", "Storage read failed", "Controls could not be loaded.")
return
}
c.JSON(http.StatusOK, controls)
}
func (h handler) upsertControls(c *gin.Context) {
var req recommendation.UserControls
if err := c.ShouldBindJSON(&req); err != nil {
problem(c, http.StatusBadRequest, "https://spotify-rec.local/errors/invalid-json", "Invalid JSON", err.Error())
return
}
req.UserID = c.Param("user_id")
if strings.TrimSpace(req.UserID) == "" {
problem(c, http.StatusUnprocessableEntity, "https://spotify-rec.local/errors/validation", "Validation failed", "user_id is required")
return
}
if err := h.store.UpsertControls(c.Request.Context(), req); err != nil {
h.logger.Error("upsert controls", zap.Error(err))
problem(c, http.StatusInternalServerError, "https://spotify-rec.local/errors/storage-write", "Storage write failed", "Controls could not be stored.")
return
}
c.JSON(http.StatusOK, req)
}
func (h handler) importSpotify(c *gin.Context) {
service, ok := h.providerService(c)
if !ok {
return
}
var req provider.ImportRequest
if err := c.ShouldBindJSON(&req); err != nil {
problem(c, http.StatusBadRequest, "https://spotify-rec.local/errors/invalid-json", "Invalid JSON", err.Error())
return
}
resp, err := service.ImportSpotify(c.Request.Context(), req)
if err != nil {
h.providerProblem(c, err)
return
}
c.JSON(http.StatusOK, resp)
}
func (h handler) searchSpotify(c *gin.Context) {
service, ok := h.providerService(c)
if !ok {
return
}
var req provider.SearchRequest
if err := c.ShouldBindJSON(&req); err != nil {
problem(c, http.StatusBadRequest, "https://spotify-rec.local/errors/invalid-json", "Invalid JSON", err.Error())
return
}
resp, err := service.SearchSpotify(c.Request.Context(), req)
if err != nil {
h.providerProblem(c, err)
return
}
c.JSON(http.StatusOK, resp)
}
func (h handler) enrichMusicBrainz(c *gin.Context) {
service, ok := h.providerService(c)
if !ok {
return
}
var req provider.EnrichRequest
if err := c.ShouldBindJSON(&req); err != nil {
problem(c, http.StatusBadRequest, "https://spotify-rec.local/errors/invalid-json", "Invalid JSON", err.Error())
return
}
resp, err := service.EnrichMusicBrainz(c.Request.Context(), req)
if err != nil {
h.providerProblem(c, err)
return
}
c.JSON(http.StatusOK, resp)
}
func (h handler) providerStatus(c *gin.Context) {
service, ok := h.providerService(c)
if !ok {
return
}
c.JSON(http.StatusOK, service.Status(c.Request.Context()))
}
func (h handler) providerService(c *gin.Context) (*provider.Service, bool) {
if h.provider == nil {
problem(c, http.StatusServiceUnavailable, "https://spotify-rec.local/errors/provider-unavailable", "Provider unavailable", "Provider imports are not configured for this storage backend.")
return nil, false
}
return h.provider, true
}
func (h handler) providerProblem(c *gin.Context, err error) {
if errors.Is(err, context.Canceled) {
return
}
message := err.Error()
switch {
case strings.Contains(message, "not configured"), strings.Contains(message, "credentials"):
problem(c, http.StatusServiceUnavailable, "https://spotify-rec.local/errors/provider-not-configured", "Provider not configured", message)
case strings.Contains(message, "required"), strings.Contains(message, "unsupported"), strings.Contains(message, "must be"):
problem(c, http.StatusUnprocessableEntity, "https://spotify-rec.local/errors/provider-validation", "Validation failed", message)
default:
h.logger.Error("provider request", zap.Error(err))
problem(c, http.StatusBadGateway, "https://spotify-rec.local/errors/provider-request-failed", "Provider request failed", "The upstream provider request could not be completed.")
}
}
apps/backend/internal/httpapi/router_test.go
+64
View File
@@ -0,0 +1,64 @@
package httpapi
import (
"bytes"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/gin-gonic/gin"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/recommendation"
memstore "github.com/tdvorak/spotifyrecalg/apps/backend/internal/storage/memory"
"go.uber.org/zap"
)
func TestRecommendationEndpoint(t *testing.T) {
gin.SetMode(gin.TestMode)
store := memstore.New()
memstore.SeedDemo(store)
engine := recommendation.NewEngine(recommendation.EngineConfig{
Now: func() time.Time { return time.Date(2026, 4, 13, 12, 0, 0, 0, time.UTC) },
ContentWeight: 0.44,
CollabWeight: 0.28,
PopularityWeight: 0.08,
ExplorationWeight: 0.20,
DiversityLambda: 0.74,
})
router := NewRouter(RouterConfig{
Store: store,
Engine: engine,
Logger: zap.NewNop(),
Version: "test",
})
body := bytes.NewBufferString(`{"user_id":"demo-user","limit":3,"mode":"balanced"}`)
req := httptest.NewRequest(http.MethodPost, "/v1/recommendations", body)
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
router.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200, got %d: %s", rec.Code, rec.Body.String())
}
if !bytes.Contains(rec.Body.Bytes(), []byte(`"taste_profile"`)) {
t.Fatalf("expected taste profile in response: %s", rec.Body.String())
}
}
func TestAPIKeyMiddleware(t *testing.T) {
router := NewRouter(RouterConfig{
Store: memstore.New(),
Engine: recommendation.NewEngine(recommendation.EngineConfig{}),
Logger: zap.NewNop(),
APIKeys: []string{"secret"},
Version: "test",
})
req := httptest.NewRequest(http.MethodPost, "/v1/recommendations", bytes.NewBufferString(`{}`))
rec := httptest.NewRecorder()
router.ServeHTTP(rec, req)
if rec.Code != http.StatusUnauthorized {
t.Fatalf("expected 401, got %d", rec.Code)
}
}
apps/backend/internal/provider/mapper.go
+105
View File
@@ -0,0 +1,105 @@
package provider
import (
"strings"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider/musicbrainz"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider/spotify"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/recommendation"
)
func mapSpotifyTrack(track spotify.Track, features spotify.AudioFeatures, mb musicbrainz.Recording, missingFeatures bool) recommendation.Track {
artist := ""
if len(track.Artists) > 0 {
artist = track.Artists[0].Name
}
spotifyURL := "https://open.spotify.com/track/" + track.ID
external := map[string]string{
"source": ProviderSpotify,
"spotify_id": track.ID,
"spotify": spotifyURL,
"spotify_url": spotifyURL,
}
if value := strings.TrimSpace(track.ExternalURLs["spotify"]); value != "" {
external["spotify"] = value
external["spotify_url"] = value
}
if isrc := strings.ToUpper(strings.TrimSpace(track.ExternalIDs["isrc"])); isrc != "" {
external["isrc"] = isrc
}
if len(track.Album.Images) > 0 && track.Album.Images[0].URL != "" {
external["image_url"] = track.Album.Images[0].URL
external["spotify_image_url"] = track.Album.Images[0].URL
}
if missingFeatures {
external["features_missing"] = "true"
}
if mb.ID != "" {
external["musicbrainz_recording_id"] = mb.ID
}
if mb.ArtistID != "" {
external["musicbrainz_artist_id"] = mb.ArtistID
}
if mb.ISRC != "" && external["isrc"] == "" {
external["isrc"] = mb.ISRC
}
genres := mergeStrings(nil, mb.Genres...)
genres = mergeStrings(genres, mb.Tags...)
return recommendation.Track{
ID: "spotify:track:" + track.ID,
Title: track.Name,
Artist: artist,
Album: track.Album.Name,
Genres: genres,
ReleaseDate: track.Album.ReleaseDate,
DurationMS: track.DurationMS,
Popularity: clamp01(float64(track.Popularity) / 100),
Explicit: track.Explicit,
Features: recommendation.AudioFeatures{
Danceability: features.Danceability,
Energy: features.Energy,
Loudness: features.Loudness,
Speechiness: features.Speechiness,
Acousticness: features.Acousticness,
Instrumentalness: features.Instrumentalness,
Liveness: features.Liveness,
Valence: features.Valence,
Tempo: features.Tempo,
TimeSignature: features.TimeSignature,
Key: features.Key,
Mode: features.Mode,
},
External: external,
DiscoveryAllowed: true,
}
}
func mergeStrings(values []string, next ...string) []string {
seen := make(map[string]struct{}, len(values)+len(next))
out := make([]string, 0, len(values)+len(next))
for _, value := range append(values, next...) {
value = strings.ToLower(strings.TrimSpace(value))
if value == "" {
continue
}
if _, ok := seen[value]; ok {
continue
}
seen[value] = struct{}{}
out = append(out, value)
}
return out
}
func clamp01(value float64) float64 {
if value < 0 {
return 0
}
if value > 1 {
return 1
}
return value
}
apps/backend/internal/provider/musicbrainz/client.go
+271
View File
@@ -0,0 +1,271 @@
package musicbrainz
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"strings"
"sync"
"time"
)
const (
defaultBaseURL = "https://musicbrainz.org/ws/2"
defaultTimeout = 10 * time.Second
)
type Config struct {
AppName string
Contact string
Version string
BaseURL string
HTTPClient *http.Client
Timeout time.Duration
MinDelay time.Duration
}
type Client struct {
appName string
contact string
version string
baseURL string
httpClient *http.Client
minDelay time.Duration
mu sync.Mutex
lastCall time.Time
lastError string
}
type Recording struct {
ID string
Title string
Artist string
ArtistID string
ISRC string
Genres []string
Tags []string
}
func New(cfg Config) *Client {
timeout := cfg.Timeout
if timeout <= 0 {
timeout = defaultTimeout
}
httpClient := cfg.HTTPClient
if httpClient == nil {
httpClient = &http.Client{Timeout: timeout}
}
baseURL := strings.TrimRight(cfg.BaseURL, "/")
if baseURL == "" {
baseURL = defaultBaseURL
}
minDelay := cfg.MinDelay
if minDelay <= 0 {
minDelay = time.Second
}
version := strings.TrimSpace(cfg.Version)
if version == "" {
version = "0.1.0"
}
return &Client{
appName: strings.TrimSpace(cfg.AppName),
contact: strings.TrimSpace(cfg.Contact),
version: version,
baseURL: baseURL,
httpClient: httpClient,
minDelay: minDelay,
}
}
func (c *Client) Configured() bool {
return c.appName != "" && c.contact != ""
}
func (c *Client) LastError() string {
c.mu.Lock()
defer c.mu.Unlock()
return c.lastError
}
func (c *Client) LookupByISRC(ctx context.Context, isrc string) (Recording, []byte, error) {
isrc = strings.ToUpper(strings.TrimSpace(isrc))
if isrc == "" {
return Recording{}, nil, errors.New("isrc is required")
}
params := url.Values{}
params.Set("fmt", "json")
params.Set("inc", "artist-credits+isrcs+tags")
payload, err := c.get(ctx, "/isrc/"+url.PathEscape(isrc), params)
if err != nil {
return Recording{}, payload, err
}
recording, err := parseISRCRecording(payload, isrc)
return recording, payload, err
}
func (c *Client) SearchRecording(ctx context.Context, title, artist string) (Recording, []byte, error) {
title = strings.TrimSpace(title)
artist = strings.TrimSpace(artist)
if title == "" {
return Recording{}, nil, errors.New("title is required")
}
query := `recording:"` + escapeQuery(title) + `"`
if artist != "" {
query += ` AND artist:"` + escapeQuery(artist) + `"`
}
params := url.Values{}
params.Set("fmt", "json")
params.Set("query", query)
params.Set("limit", "1")
payload, err := c.get(ctx, "/recording", params)
if err != nil {
return Recording{}, payload, err
}
recording, err := parseSearchRecording(payload)
return recording, payload, err
}
func (c *Client) get(ctx context.Context, path string, params url.Values) ([]byte, error) {
if !c.Configured() {
err := errors.New("musicbrainz app name and contact are required")
c.setLastError(err.Error())
return nil, err
}
if err := c.wait(ctx); err != nil {
return nil, err
}
endpoint := c.baseURL + path
if encoded := params.Encode(); encoded != "" {
endpoint += "?" + encoded
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
if err != nil {
return nil, err
}
req.Header.Set("Accept", "application/json")
req.Header.Set("User-Agent", c.userAgent())
resp, err := c.httpClient.Do(req)
if err != nil {
c.setLastError(err.Error())
return nil, err
}
defer resp.Body.Close()
payload, err := io.ReadAll(io.LimitReader(resp.Body, 4<<20))
if err != nil {
c.setLastError(err.Error())
return payload, err
}
if resp.StatusCode < 200 || resp.StatusCode > 299 {
err := fmt.Errorf("musicbrainz request failed with status %d", resp.StatusCode)
c.setLastError(err.Error())
return payload, err
}
c.setLastError("")
return payload, nil
}
func (c *Client) wait(ctx context.Context) error {
c.mu.Lock()
defer c.mu.Unlock()
wait := c.minDelay - time.Since(c.lastCall)
if wait > 0 {
timer := time.NewTimer(wait)
c.mu.Unlock()
select {
case <-ctx.Done():
timer.Stop()
c.mu.Lock()
return ctx.Err()
case <-timer.C:
}
c.mu.Lock()
}
c.lastCall = time.Now()
return nil
}
func (c *Client) userAgent() string {
return fmt.Sprintf("%s/%s (%s)", c.appName, c.version, c.contact)
}
func (c *Client) setLastError(message string) {
c.mu.Lock()
defer c.mu.Unlock()
c.lastError = message
}
func parseISRCRecording(payload []byte, isrc string) (Recording, error) {
var decoded struct {
Recordings []recordingJSON `json:"recordings"`
}
if err := json.Unmarshal(payload, &decoded); err != nil {
return Recording{}, fmt.Errorf("decode musicbrainz isrc: %w", err)
}
if len(decoded.Recordings) == 0 {
return Recording{}, errors.New("musicbrainz isrc lookup returned no recordings")
}
return decoded.Recordings[0].toRecording(isrc), nil
}
func parseSearchRecording(payload []byte) (Recording, error) {
var decoded struct {
Recordings []recordingJSON `json:"recordings"`
}
if err := json.Unmarshal(payload, &decoded); err != nil {
return Recording{}, fmt.Errorf("decode musicbrainz recording search: %w", err)
}
if len(decoded.Recordings) == 0 {
return Recording{}, errors.New("musicbrainz recording search returned no matches")
}
return decoded.Recordings[0].toRecording(""), nil
}
type recordingJSON struct {
ID string `json:"id"`
Title string `json:"title"`
ArtistCredit []struct {
Artist struct {
ID string `json:"id"`
Name string `json:"name"`
} `json:"artist"`
} `json:"artist-credit"`
ISRCs []string `json:"isrcs"`
Tags []struct {
Name string `json:"name"`
} `json:"tags"`
Genres []struct {
Name string `json:"name"`
} `json:"genres"`
}
func (r recordingJSON) toRecording(fallbackISRC string) Recording {
out := Recording{ID: r.ID, Title: r.Title, ISRC: fallbackISRC}
if len(r.ArtistCredit) > 0 {
out.Artist = r.ArtistCredit[0].Artist.Name
out.ArtistID = r.ArtistCredit[0].Artist.ID
}
if out.ISRC == "" && len(r.ISRCs) > 0 {
out.ISRC = strings.ToUpper(r.ISRCs[0])
}
for _, genre := range r.Genres {
if genre.Name != "" {
out.Genres = append(out.Genres, genre.Name)
}
}
for _, tag := range r.Tags {
if tag.Name != "" {
out.Tags = append(out.Tags, tag.Name)
}
}
return out
}
func escapeQuery(value string) string {
return strings.ReplaceAll(value, `"`, `\"`)
}
apps/backend/internal/provider/service.go
+912
View File
@@ -0,0 +1,912 @@
package provider
import (
"context"
"crypto/rand"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"strings"
"time"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider/musicbrainz"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider/songlink"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider/spotify"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider/urlparser"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider/webplayer"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/recommendation"
)
type ServiceConfig struct {
DefaultMarket string
CacheTTL time.Duration
Version string
}
type Service struct {
store Store
spotify *spotify.Client
webplayer *webplayer.Client
songlink *songlink.Client
urlparser *urlparser.Parser
musicbrainz *musicbrainz.Client
defaultMarket string
cacheTTL time.Duration
now func() time.Time
}
func NewService(store Store, spotifyClient *spotify.Client, webplayerClient *webplayer.Client, songlinkClient *songlink.Client, musicBrainzClient *musicbrainz.Client, cfg ServiceConfig) *Service {
cacheTTL := cfg.CacheTTL
if cacheTTL <= 0 {
cacheTTL = 24 * time.Hour
}
return &Service{
store: store,
spotify: spotifyClient,
webplayer: webplayerClient,
songlink: songlinkClient,
urlparser: urlparser.NewParser(),
musicbrainz: musicBrainzClient,
defaultMarket: strings.ToUpper(strings.TrimSpace(cfg.DefaultMarket)),
cacheTTL: cacheTTL,
now: func() time.Time { return time.Now().UTC() },
}
}
func (s *Service) ImportSpotify(ctx context.Context, req ImportRequest) (ImportResponse, error) {
// Try official Spotify API first (more reliable, has audio features)
if s.spotify != nil && s.spotify.Configured() {
return s.importFromOfficialAPI(ctx, req)
}
// Fall back to native webplayer client (auth-free, no API keys needed)
if s.webplayer != nil && s.webplayer.Configured() {
return s.importFromWebPlayer(ctx, req)
}
return ImportResponse{}, spotify.ErrNotConfigured
}
func (s *Service) importFromOfficialAPI(ctx context.Context, req ImportRequest) (ImportResponse, error) {
persist := true
if req.Persist != nil {
persist = *req.Persist
}
limit := capLimit(req.Limit, 100)
market := s.market(req.Market)
parsed, sourceWarnings, err := s.resolveSpotifySource(ctx, req.Source)
if err != nil {
return ImportResponse{}, err
}
job := ImportJob{
ID: newID("import"),
Provider: ProviderSpotify,
SourceType: parsed.Type,
SourceValue: parsed.ID,
Market: market,
Status: "running",
StartedAt: s.now(),
}
if persist {
if err := s.store.CreateImportJob(ctx, job); err != nil {
return ImportResponse{}, err
}
}
tracks, skipped, warnings, err := s.importSpotifyTracks(ctx, parsed, market, limit, boolDefault(req.EnrichMusicBrainz, true), req.AllowMissingFields)
warnings = append(sourceWarnings, warnings...)
if err != nil {
job.Status = "failed"
job.Warnings = append(warnings, err.Error())
job.FinishedAt = s.now()
if persist {
_ = s.store.FinishImportJob(ctx, job)
}
return ImportResponse{}, err
}
imported, updated := 0, 0
if persist && len(tracks) > 0 {
existingIDs := make([]string, 0, len(tracks))
for _, track := range tracks {
existingIDs = append(existingIDs, track.ID)
}
existing, err := s.store.GetTracksByIDs(ctx, existingIDs)
if err != nil {
return ImportResponse{}, err
}
existingSet := make(map[string]struct{}, len(existing))
for _, track := range existing {
existingSet[track.ID] = struct{}{}
}
for _, track := range tracks {
if _, ok := existingSet[track.ID]; ok {
updated++
} else {
imported++
}
}
if err := s.store.UpsertTracks(ctx, tracks); err != nil {
return ImportResponse{}, err
}
if err := s.upsertTrackEnrichments(ctx, tracks); err != nil {
return ImportResponse{}, err
}
}
job.Status = "succeeded"
job.ImportedTracks = imported
job.UpdatedTracks = updated
job.Skipped = skipped
job.Warnings = warnings
job.FinishedAt = s.now()
if persist {
if err := s.store.FinishImportJob(ctx, job); err != nil {
return ImportResponse{}, err
}
}
return ImportResponse{
ImportID: job.ID,
ImportedTracks: imported,
UpdatedTracks: updated,
Skipped: skipped,
Warnings: warnings,
}, nil
}
func (s *Service) resolveSpotifySource(ctx context.Context, source Source) (spotify.ParsedSource, []string, error) {
_ = ctx
parsed, err := spotify.ParseSource(source.Type, source.Value)
if err == nil {
return parsed, nil, nil
}
if strings.ToLower(strings.TrimSpace(source.Type)) != "url" {
return spotify.ParsedSource{}, nil, err
}
parsedURL := s.urlparser.ParseURL(source.Value)
if parsedURL == nil || parsedURL.Service == urlparser.Spotify {
return spotify.ParsedSource{}, nil, err
}
if s.songlink == nil || !s.songlink.Configured() {
return spotify.ParsedSource{}, nil, err
}
links, linkErr := s.songlink.GetLinks(parsedURL.URL)
if linkErr != nil {
return spotify.ParsedSource{}, nil, fmt.Errorf("could not resolve %s URL to Spotify: %w", parsedURL.Service, linkErr)
}
if strings.TrimSpace(links.SpotifyID) == "" {
return spotify.ParsedSource{}, nil, fmt.Errorf("could not resolve %s URL to a Spotify track", parsedURL.Service)
}
spotifyID := strings.TrimSpace(links.SpotifyID)
return spotify.ParsedSource{
Type: "track",
ID: spotifyID,
URL: "https://open.spotify.com/track/" + spotifyID,
}, []string{"resolved " + string(parsedURL.Service) + " URL to Spotify via Song.link"}, nil
}
func (s *Service) SearchSpotify(ctx context.Context, req SearchRequest) (SearchResponse, error) {
if s.spotify == nil || !s.spotify.Configured() {
// Try webplayer search if available (auth-free)
if s.webplayer != nil && s.webplayer.Configured() {
return s.searchViaWebPlayer(ctx, req)
}
return SearchResponse{}, spotify.ErrNotConfigured
}
itemType := strings.ToLower(strings.TrimSpace(req.Type))
if itemType == "" {
itemType = "track"
}
if !validSearchType(itemType) {
return SearchResponse{}, errors.New("search type must be track, album, artist, or playlist")
}
limit := capSearchLimit(req.Limit)
market := s.market(req.Market)
result, _, warnings, err := s.spotifySearch(ctx, req.Query, itemType, market, limit)
if err != nil {
return SearchResponse{}, err
}
ids, idWarnings := s.trackIDsFromSearch(ctx, result, itemType, market, limit)
warnings = append(warnings, idWarnings...)
tracks := make([]recommendation.Track, 0, len(ids))
skipped := 0
for _, id := range ids {
track, trackWarnings, ok := s.buildTrack(ctx, id, market, boolDefault(req.EnrichMusicBrainz, true), req.AllowMissingFields)
warnings = append(warnings, trackWarnings...)
if !ok {
skipped++
continue
}
tracks = append(tracks, track)
}
persisted := 0
if req.Persist && len(tracks) > 0 {
if err := s.store.UpsertTracks(ctx, tracks); err != nil {
return SearchResponse{}, err
}
if err := s.upsertTrackEnrichments(ctx, tracks); err != nil {
return SearchResponse{}, err
}
persisted = len(tracks)
}
return SearchResponse{Tracks: tracks, Persisted: persisted, Skipped: skipped, Warnings: warnings}, nil
}
func (s *Service) trackIDsFromSearch(ctx context.Context, result spotify.SearchResult, itemType, market string, limit int) ([]string, []string) {
var warnings []string
ids := make([]string, 0, limit)
addID := func(id string) {
if id == "" || len(ids) >= limit {
return
}
ids = append(ids, id)
}
switch itemType {
case "track":
for _, item := range result.Tracks.Items {
addID(item.ID)
}
case "album":
for _, album := range result.Albums.Items {
refs, _, cacheWarnings, err := s.spotifyAlbumTracks(ctx, album.ID, market, limit-len(ids))
warnings = append(warnings, cacheWarnings...)
if err != nil {
warnings = append(warnings, fmt.Sprintf("spotify album %s skipped: %v", album.ID, err))
continue
}
for _, ref := range refs {
addID(ref.ID)
}
}
case "artist":
for _, artist := range result.Artists.Items {
items, _, cacheWarnings, err := s.spotifyArtistTopTracks(ctx, artist.ID, market)
warnings = append(warnings, cacheWarnings...)
if err != nil {
warnings = append(warnings, fmt.Sprintf("spotify artist %s skipped: %v", artist.ID, err))
continue
}
for _, item := range items {
addID(item.ID)
}
}
case "playlist":
for _, playlist := range result.Playlists.Items {
refs, _, cacheWarnings, err := s.spotifyPlaylistTracks(ctx, playlist.ID, market, limit-len(ids))
warnings = append(warnings, cacheWarnings...)
if err != nil {
warnings = append(warnings, fmt.Sprintf("spotify playlist %s skipped: %v", playlist.ID, err))
continue
}
for _, ref := range refs {
addID(ref.ID)
}
}
}
return ids, warnings
}
func (s *Service) EnrichMusicBrainz(ctx context.Context, req EnrichRequest) (EnrichResponse, error) {
if s.musicbrainz == nil || !s.musicbrainz.Configured() {
return EnrichResponse{}, errors.New("musicbrainz app name and contact are required")
}
tracks, err := s.store.GetTracksByIDs(ctx, req.TrackIDs)
if err != nil {
return EnrichResponse{}, err
}
byID := make(map[string]recommendation.Track, len(tracks))
for _, track := range tracks {
byID[track.ID] = track
}
var warnings []string
updated, skipped := 0, 0
for _, id := range req.TrackIDs {
track, ok := byID[id]
if !ok {
skipped++
warnings = append(warnings, "track not found: "+id)
continue
}
if !req.Force && track.External["musicbrainz_recording_id"] != "" {
skipped++
continue
}
mb, raw, warn, ok := s.enrichTrack(ctx, track)
if warn != "" {
warnings = append(warnings, warn)
}
if !ok {
skipped++
continue
}
if track.External == nil {
track.External = map[string]string{}
}
track.External["musicbrainz_recording_id"] = mb.ID
if mb.ArtistID != "" {
track.External["musicbrainz_artist_id"] = mb.ArtistID
}
if mb.ISRC != "" && track.External["isrc"] == "" {
track.External["isrc"] = mb.ISRC
}
track.Genres = mergeStrings(track.Genres, mb.Genres...)
track.Genres = mergeStrings(track.Genres, mb.Tags...)
if err := s.store.UpsertTrack(ctx, track); err != nil {
return EnrichResponse{}, err
}
if err := s.store.UpsertTrackEnrichment(ctx, TrackEnrichment{
TrackID: track.ID,
Provider: ProviderMusicBrainz,
MusicBrainzRecordingID: mb.ID,
MusicBrainzArtistID: mb.ArtistID,
ISRC: mb.ISRC,
Payload: raw,
UpdatedAt: s.now(),
}); err != nil {
return EnrichResponse{}, err
}
updated++
}
return EnrichResponse{Updated: updated, Skipped: skipped, Warnings: warnings}, nil
}
func (s *Service) Status(ctx context.Context) StatusResponse {
stats, _ := s.store.ProviderCacheStats(ctx)
now := s.now()
spotifyStatus := ProviderStatus{CheckedAt: now}
if s.spotify != nil {
spotifyStatus.Configured = s.spotify.Configured()
spotifyStatus.TokenMode = s.spotify.TokenMode()
spotifyStatus.Available = s.spotify.Configured() && s.spotify.LastError() == ""
spotifyStatus.LastError = s.spotify.LastError()
}
mbStatus := ProviderStatus{CheckedAt: now}
if s.musicbrainz != nil {
mbStatus.Configured = s.musicbrainz.Configured()
mbStatus.TokenMode = "user_agent"
mbStatus.Available = s.musicbrainz.Configured() && s.musicbrainz.LastError() == ""
mbStatus.LastError = s.musicbrainz.LastError()
}
return StatusResponse{Spotify: spotifyStatus, MusicBrainz: mbStatus, Cache: stats}
}
func (s *Service) importSpotifyTracks(ctx context.Context, parsed spotify.ParsedSource, market string, limit int, enrichMB, allowMissing bool) ([]recommendation.Track, int, []string, error) {
ids := []string{parsed.ID}
var warnings []string
switch parsed.Type {
case "track":
case "album":
refs, _, cacheWarnings, err := s.spotifyAlbumTracks(ctx, parsed.ID, market, limit)
if err != nil {
return nil, 0, warnings, err
}
warnings = append(warnings, cacheWarnings...)
ids = ids[:0]
for _, ref := range refs {
if ref.ID != "" {
ids = append(ids, ref.ID)
}
}
case "playlist":
refs, _, cacheWarnings, err := s.spotifyPlaylistTracks(ctx, parsed.ID, market, limit)
if err != nil {
return nil, 0, warnings, err
}
warnings = append(warnings, cacheWarnings...)
ids = ids[:0]
for _, ref := range refs {
if ref.ID != "" {
ids = append(ids, ref.ID)
}
}
case "artist":
items, _, cacheWarnings, err := s.spotifyArtistTopTracks(ctx, parsed.ID, market)
if err != nil {
return nil, 0, warnings, err
}
warnings = append(warnings, cacheWarnings...)
ids = ids[:0]
for _, item := range items {
if item.ID != "" {
ids = append(ids, item.ID)
if limit > 0 && len(ids) >= limit {
break
}
}
}
default:
return nil, 0, warnings, errors.New("unsupported Spotify source type")
}
tracks := make([]recommendation.Track, 0, len(ids))
skipped := 0
for _, id := range ids {
track, trackWarnings, ok := s.buildTrack(ctx, id, market, enrichMB, allowMissing)
warnings = append(warnings, trackWarnings...)
if !ok {
skipped++
continue
}
tracks = append(tracks, track)
}
return tracks, skipped, warnings, nil
}
func (s *Service) buildTrack(ctx context.Context, id, market string, enrichMB, allowMissing bool) (recommendation.Track, []string, bool) {
var warnings []string
item, _, cacheWarnings, err := s.spotifyTrack(ctx, id, market)
warnings = append(warnings, cacheWarnings...)
if err != nil {
warnings = append(warnings, fmt.Sprintf("spotify track %s skipped: %v", id, err))
return recommendation.Track{}, warnings, false
}
features, _, cacheWarnings, err := s.spotifyAudioFeatures(ctx, id)
warnings = append(warnings, cacheWarnings...)
missingFeatures := false
if err != nil {
if !allowMissing {
warnings = append(warnings, fmt.Sprintf("spotify track %s skipped: audio features unavailable", id))
return recommendation.Track{}, warnings, false
}
missingFeatures = true
warnings = append(warnings, fmt.Sprintf("spotify track %s imported without audio features", id))
}
var mb musicbrainz.Recording
if enrichMB {
recording, _, warn, ok := s.enrichSpotifyTrack(ctx, item)
if warn != "" {
warnings = append(warnings, warn)
}
if ok {
mb = recording
}
}
return mapSpotifyTrack(item, features, mb, missingFeatures), warnings, true
}
func (s *Service) upsertTrackEnrichments(ctx context.Context, tracks []recommendation.Track) error {
for _, track := range tracks {
if track.External["musicbrainz_recording_id"] == "" {
continue
}
if err := s.store.UpsertTrackEnrichment(ctx, TrackEnrichment{
TrackID: track.ID,
Provider: ProviderMusicBrainz,
MusicBrainzRecordingID: track.External["musicbrainz_recording_id"],
MusicBrainzArtistID: track.External["musicbrainz_artist_id"],
ISRC: track.External["isrc"],
UpdatedAt: s.now(),
}); err != nil {
return err
}
}
return nil
}
func (s *Service) enrichSpotifyTrack(ctx context.Context, track spotify.Track) (musicbrainz.Recording, []byte, string, bool) {
if s.musicbrainz == nil || !s.musicbrainz.Configured() {
return musicbrainz.Recording{}, nil, "", false
}
if isrc := strings.ToUpper(strings.TrimSpace(track.ExternalIDs["isrc"])); isrc != "" {
mb, raw, warnings, err := s.musicBrainzISRC(ctx, isrc)
if err == nil {
return mb, raw, "", true
}
return musicbrainz.Recording{}, raw, appendWarning(warnings, "musicbrainz isrc lookup failed for "+isrc), false
}
artist := ""
if len(track.Artists) > 0 {
artist = track.Artists[0].Name
}
mb, raw, warnings, err := s.musicBrainzSearch(ctx, track.Name, artist)
if err != nil {
return musicbrainz.Recording{}, raw, appendWarning(warnings, "musicbrainz search failed for "+track.Name), false
}
return mb, raw, "", true
}
func (s *Service) enrichTrack(ctx context.Context, track recommendation.Track) (musicbrainz.Recording, []byte, string, bool) {
if isrc := strings.TrimSpace(track.External["isrc"]); isrc != "" {
mb, raw, warnings, err := s.musicBrainzISRC(ctx, isrc)
if err == nil {
return mb, raw, "", true
}
return musicbrainz.Recording{}, raw, appendWarning(warnings, "musicbrainz isrc lookup failed for "+isrc), false
}
mb, raw, warnings, err := s.musicBrainzSearch(ctx, track.Title, track.Artist)
if err != nil {
return musicbrainz.Recording{}, raw, appendWarning(warnings, "musicbrainz search failed for "+track.ID), false
}
return mb, raw, "", true
}
func (s *Service) spotifyTrack(ctx context.Context, id, market string) (spotify.Track, []byte, []string, error) {
var out spotify.Track
payload, warnings, err := s.cachedJSON(ctx, ProviderSpotify, "track", id, market, func(context.Context) ([]byte, error) {
_, raw, err := s.spotify.GetTrack(ctx, id, market)
return raw, err
}, &out)
return out, payload, warnings, err
}
func (s *Service) spotifyAudioFeatures(ctx context.Context, id string) (spotify.AudioFeatures, []byte, []string, error) {
var out spotify.AudioFeatures
payload, warnings, err := s.cachedJSON(ctx, ProviderSpotify, "audio_features", id, "", func(context.Context) ([]byte, error) {
_, raw, err := s.spotify.GetAudioFeatures(ctx, id)
return raw, err
}, &out)
return out, payload, warnings, err
}
func (s *Service) spotifySearch(ctx context.Context, query, itemType, market string, limit int) (spotify.SearchResult, []byte, []string, error) {
var out spotify.SearchResult
itemID := itemType + ":" + query + ":" + fmt.Sprint(limit)
payload, warnings, err := s.cachedJSON(ctx, ProviderSpotify, "search", itemID, market, func(context.Context) ([]byte, error) {
_, raw, err := s.spotify.Search(ctx, query, itemType, market, limit)
return raw, err
}, &out)
return out, payload, warnings, err
}
func (s *Service) spotifyAlbumTracks(ctx context.Context, id, market string, limit int) ([]spotify.TrackRef, []byte, []string, error) {
var out []spotify.TrackRef
payload, warnings, err := s.cachedJSON(ctx, ProviderSpotify, "album_tracks", id+":"+fmt.Sprint(limit), market, func(context.Context) ([]byte, error) {
refs, _, err := s.spotify.GetAlbumTracks(ctx, id, market, limit)
if err != nil {
return nil, err
}
return json.Marshal(refs)
}, &out)
return out, payload, warnings, err
}
func (s *Service) spotifyPlaylistTracks(ctx context.Context, id, market string, limit int) ([]spotify.TrackRef, []byte, []string, error) {
var out []spotify.TrackRef
payload, warnings, err := s.cachedJSON(ctx, ProviderSpotify, "playlist_tracks", id+":"+fmt.Sprint(limit), market, func(context.Context) ([]byte, error) {
refs, _, err := s.spotify.GetPlaylistTracks(ctx, id, market, limit)
if err != nil {
return nil, err
}
return json.Marshal(refs)
}, &out)
return out, payload, warnings, err
}
func (s *Service) spotifyArtistTopTracks(ctx context.Context, id, market string) ([]spotify.Track, []byte, []string, error) {
var out []spotify.Track
payload, warnings, err := s.cachedJSON(ctx, ProviderSpotify, "artist_top_tracks", id, market, func(context.Context) ([]byte, error) {
tracks, _, err := s.spotify.GetArtistTopTracks(ctx, id, market)
if err != nil {
return nil, err
}
return json.Marshal(tracks)
}, &out)
return out, payload, warnings, err
}
func (s *Service) musicBrainzISRC(ctx context.Context, isrc string) (musicbrainz.Recording, []byte, []string, error) {
var out musicbrainz.Recording
payload, warnings, err := s.cachedJSON(ctx, ProviderMusicBrainz, "isrc", isrc, "", func(context.Context) ([]byte, error) {
recording, raw, err := s.musicbrainz.LookupByISRC(ctx, isrc)
if err != nil {
return raw, err
}
return json.Marshal(recording)
}, &out)
return out, payload, warnings, err
}
func (s *Service) musicBrainzSearch(ctx context.Context, title, artist string) (musicbrainz.Recording, []byte, []string, error) {
var out musicbrainz.Recording
itemID := title + ":" + artist
payload, warnings, err := s.cachedJSON(ctx, ProviderMusicBrainz, "recording_search", itemID, "", func(context.Context) ([]byte, error) {
recording, raw, err := s.musicbrainz.SearchRecording(ctx, title, artist)
if err != nil {
return raw, err
}
return json.Marshal(recording)
}, &out)
return out, payload, warnings, err
}
func (s *Service) cachedJSON(ctx context.Context, providerName, itemType, itemID, market string, fetch func(context.Context) ([]byte, error), out any) ([]byte, []string, error) {
var warnings []string
now := s.now()
cached, ok, err := s.store.GetProviderCache(ctx, providerName, itemType, itemID, market)
if err != nil {
return nil, warnings, err
}
if ok && cached.Fresh(now) {
if err := json.Unmarshal(cached.Payload, out); err != nil {
return cached.Payload, warnings, err
}
return cached.Payload, warnings, nil
}
payload, err := fetch(ctx)
if err != nil {
if ok && len(cached.Payload) > 0 {
warnings = append(warnings, fmt.Sprintf("using stale %s %s cache after provider error", providerName, itemType))
if decodeErr := json.Unmarshal(cached.Payload, out); decodeErr != nil {
return cached.Payload, warnings, decodeErr
}
return cached.Payload, warnings, nil
}
_ = s.store.UpsertProviderCache(ctx, CacheEntry{
Provider: providerName,
ItemType: itemType,
ItemID: itemID,
Market: market,
FetchedAt: now,
ExpiresAt: now,
LastError: err.Error(),
})
return payload, warnings, err
}
if err := json.Unmarshal(payload, out); err != nil {
return payload, warnings, err
}
if err := s.store.UpsertProviderCache(ctx, CacheEntry{
Provider: providerName,
ItemType: itemType,
ItemID: itemID,
Market: market,
Payload: payload,
FetchedAt: now,
ExpiresAt: now.Add(s.cacheTTL),
}); err != nil {
return payload, warnings, err
}
return payload, warnings, nil
}
func (s *Service) market(value string) string {
if value = strings.ToUpper(strings.TrimSpace(value)); value != "" {
return value
}
return s.defaultMarket
}
func capSearchLimit(value int) int {
if value <= 0 {
return 5
}
if value > 10 {
return 10
}
return value
}
func validSearchType(value string) bool {
switch value {
case "track", "album", "artist", "playlist":
return true
default:
return false
}
}
func capLimit(value, maxValue int) int {
if value <= 0 {
return maxValue
}
if value > maxValue {
return maxValue
}
return value
}
func boolDefault(value *bool, fallback bool) bool {
if value == nil {
return fallback
}
return *value
}
func newID(prefix string) string {
var b [12]byte
if _, err := rand.Read(b[:]); err != nil {
return prefix + "_" + strings.ReplaceAll(time.Now().UTC().Format(time.RFC3339Nano), ":", "")
}
return prefix + "_" + hex.EncodeToString(b[:])
}
func appendWarning(warnings []string, fallback string) string {
if len(warnings) == 0 {
return fallback
}
return warnings[0]
}
// importFromWebPlayer imports tracks using the native auth-free webplayer client
func (s *Service) importFromWebPlayer(ctx context.Context, req ImportRequest) (ImportResponse, error) {
persist := true
if req.Persist != nil {
persist = *req.Persist
}
// Parse the URL to get the Spotify track ID
itemType, itemID, err := webplayer.ParseSpotifyURL(req.Source.Value)
if err != nil {
parsedURL := s.urlparser.ParseURL(req.Source.Value)
if parsedURL == nil || parsedURL.Service == urlparser.Spotify || s.songlink == nil || !s.songlink.Configured() {
return ImportResponse{}, fmt.Errorf("invalid Spotify URL: %w", err)
}
links, linkErr := s.songlink.GetLinks(parsedURL.URL)
if linkErr != nil {
return ImportResponse{}, fmt.Errorf("could not resolve %s URL to Spotify: %w", parsedURL.Service, linkErr)
}
if strings.TrimSpace(links.SpotifyID) == "" {
return ImportResponse{}, fmt.Errorf("could not resolve %s URL to a Spotify track", parsedURL.Service)
}
itemType = "track"
itemID = strings.TrimSpace(links.SpotifyID)
}
if itemType != "track" {
return ImportResponse{}, fmt.Errorf("unsupported item type: %s (only tracks supported for web player import)", itemType)
}
job := ImportJob{
ID: newID("import"),
Provider: ProviderSpotify,
SourceType: itemType,
SourceValue: itemID,
Status: "running",
StartedAt: s.now(),
}
if persist {
if err := s.store.CreateImportJob(ctx, job); err != nil {
return ImportResponse{}, err
}
}
// Fetch track from web player (auth-free using TOTP)
wpTrack, err := s.webplayer.GetTrack(itemID)
if err != nil {
job.Status = "failed"
job.Warnings = []string{err.Error()}
job.FinishedAt = s.now()
if persist {
_ = s.store.FinishImportJob(ctx, job)
}
return ImportResponse{}, fmt.Errorf("web player fetch failed: %w", err)
}
// Convert artist list to string
artistName := ""
if len(wpTrack.Artists) > 0 {
artistNames := make([]string, len(wpTrack.Artists))
for i, a := range wpTrack.Artists {
artistNames[i] = a.Name
}
artistName = strings.Join(artistNames, ", ")
}
// Build external URLs
externalURLs := map[string]string{
"spotify": fmt.Sprintf("https://open.spotify.com/track/%s", wpTrack.ID),
}
// Get cross-platform links from Song.link
if s.songlink != nil && s.songlink.Configured() {
if links, err := s.songlink.GetLinksFromSpotifyID(wpTrack.ID); err == nil && links != nil {
for platform, link := range links.Links {
externalURLs[platform] = link.URL
}
}
}
// Convert to recommendation.Track
track := recommendation.Track{
ID: wpTrack.ID,
Title: wpTrack.Name,
Artist: artistName,
Album: wpTrack.Album.Name,
DurationMS: wpTrack.DurationMs,
Explicit: wpTrack.Explicit,
Popularity: 0.5, // Web player doesn't provide popularity
External: externalURLs,
CreatedAt: s.now(),
UpdatedAt: s.now(),
}
// Add image URL if available
if len(wpTrack.Album.Images) > 0 {
track.External["image_url"] = wpTrack.Album.Images[0].URL
}
// Optionally enrich with MusicBrainz
if boolDefault(req.EnrichMusicBrainz, true) && s.musicbrainz != nil {
mb, _, _, ok := s.enrichTrack(ctx, track)
if ok && mb.ID != "" {
track.External["musicbrainz_recording_id"] = mb.ID
if mb.ISRC != "" {
track.External["isrc"] = mb.ISRC
}
}
}
// Store the track
imported, updated := 0, 0
if persist {
existing, _ := s.store.GetTracksByIDs(ctx, []string{track.ID})
if len(existing) > 0 {
updated = 1
} else {
imported = 1
}
if err := s.store.UpsertTrack(ctx, track); err != nil {
return ImportResponse{}, err
}
if err := s.upsertTrackEnrichments(ctx, []recommendation.Track{track}); err != nil {
return ImportResponse{}, err
}
}
job.Status = "succeeded"
job.ImportedTracks = imported
job.UpdatedTracks = updated
job.FinishedAt = s.now()
if persist {
if err := s.store.FinishImportJob(ctx, job); err != nil {
return ImportResponse{}, err
}
}
return ImportResponse{
ImportID: job.ID,
ImportedTracks: imported,
UpdatedTracks: updated,
Skipped: 0,
Warnings: []string{"imported via webplayer (auth-free, native Go)"},
}, nil
}
// searchViaWebPlayer searches using the native webplayer client
func (s *Service) searchViaWebPlayer(ctx context.Context, req SearchRequest) (SearchResponse, error) {
// Use the webplayer's search capability
wpTracks, err := s.webplayer.Search(req.Query, req.Limit)
if err != nil {
return SearchResponse{}, err
}
var tracks []recommendation.Track
for _, wpTrack := range wpTracks {
artistName := ""
if len(wpTrack.Artists) > 0 {
artistNames := make([]string, len(wpTrack.Artists))
for i, a := range wpTrack.Artists {
artistNames[i] = a.Name
}
artistName = strings.Join(artistNames, ", ")
}
track := recommendation.Track{
ID: wpTrack.ID,
Title: wpTrack.Name,
Artist: artistName,
Album: wpTrack.Album.Name,
DurationMS: wpTrack.DurationMs,
Explicit: wpTrack.Explicit,
Popularity: 0.5,
External: map[string]string{
"spotify": fmt.Sprintf("https://open.spotify.com/track/%s", wpTrack.ID),
},
}
tracks = append(tracks, track)
}
return SearchResponse{
Tracks: tracks,
Persisted: 0,
Skipped: 0,
Warnings: []string{"search results from webplayer (auth-free)"},
}, nil
}
apps/backend/internal/provider/service_test.go
+216
View File
@@ -0,0 +1,216 @@
package provider_test
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider/musicbrainz"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider/songlink"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider/spotify"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider/webplayer"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/recommendation"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/storage/memory"
)
func TestImportSpotifyTrackPersistsRecommendableTrack(t *testing.T) {
store := memory.New()
spotifyServer := fakeSpotifyServer(t)
defer spotifyServer.Close()
mbServer := fakeMusicBrainzServer(t)
defer mbServer.Close()
service := provider.NewService(store,
spotify.New(spotify.Config{BearerToken: "token", APIBaseURL: spotifyServer.URL + "/v1"}),
webplayer.NewClient(),
songlink.NewClient(),
musicbrainz.New(musicbrainz.Config{AppName: "SpotifyRecAlg", Contact: "test@example.com", BaseURL: mbServer.URL + "/ws/2", MinDelay: time.Nanosecond}),
provider.ServiceConfig{DefaultMarket: "US", CacheTTL: time.Hour},
)
resp, err := service.ImportSpotify(context.Background(), provider.ImportRequest{
Source: provider.Source{Type: "url", Value: "https://open.spotify.com/track/good"},
Market: "US",
EnrichMusicBrainz: boolPtr(true),
})
if err != nil {
t.Fatalf("import spotify: %v", err)
}
if resp.ImportedTracks != 1 || resp.Skipped != 0 {
t.Fatalf("unexpected import response: %+v", resp)
}
engine := recommendation.NewEngine(recommendation.EngineConfig{
ContentWeight: 0.5,
PopularityWeight: 0.2,
ExplorationWeight: 0.3,
DiversityLambda: 0.7,
})
recs, _, err := engine.Recommend(context.Background(), store, recommendation.RecommendRequest{UserID: "user", Limit: 1})
if err != nil {
t.Fatalf("recommend after import: %v", err)
}
if len(recs) != 1 || recs[0].Track.ID != "spotify:track:good" {
t.Fatalf("unexpected recommendations: %+v", recs)
}
if got := recs[0].Track.External["musicbrainz_recording_id"]; got != "mb-recording" {
t.Fatalf("musicbrainz recording id = %q", got)
}
}
func boolPtr(value bool) *bool {
return &value
}
func TestSearchSpotifyCapsLimitAndPersistFalse(t *testing.T) {
store := memory.New()
spotifyServer := fakeSpotifyServer(t)
defer spotifyServer.Close()
service := provider.NewService(store,
spotify.New(spotify.Config{BearerToken: "token", APIBaseURL: spotifyServer.URL + "/v1"}),
webplayer.NewClient(),
songlink.NewClient(),
nil,
provider.ServiceConfig{DefaultMarket: "US", CacheTTL: time.Hour},
)
resp, err := service.SearchSpotify(context.Background(), provider.SearchRequest{Query: "hello", Type: "track", Limit: 50, Persist: false})
if err != nil {
t.Fatalf("search spotify: %v", err)
}
if len(resp.Tracks) != 1 || resp.Persisted != 0 {
t.Fatalf("unexpected search response: %+v", resp)
}
if _, _, err := recommendation.NewEngine(recommendation.EngineConfig{}).Recommend(context.Background(), store, recommendation.RecommendRequest{UserID: "user", Limit: 1}); err == nil {
t.Fatal("expected empty catalog because persist=false")
}
}
func TestProviderCacheUsesStaleOnError(t *testing.T) {
store := memory.New()
spotifyServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.Error(w, "upstream down", http.StatusInternalServerError)
}))
defer spotifyServer.Close()
service := provider.NewService(store,
spotify.New(spotify.Config{BearerToken: "token", APIBaseURL: spotifyServer.URL + "/v1", MaxRetries: 1}),
webplayer.NewClient(),
songlink.NewClient(),
nil,
provider.ServiceConfig{DefaultMarket: "US", CacheTTL: time.Hour},
)
now := time.Now().UTC()
trackPayload := []byte(`{"id":"cached","name":"Cached","artists":[{"name":"Artist"}],"album":{"name":"Album"},"popularity":50}`)
if err := store.UpsertProviderCache(context.Background(), provider.CacheEntry{
Provider: provider.ProviderSpotify,
ItemType: "track",
ItemID: "cached",
Market: "US",
Payload: trackPayload,
FetchedAt: now.Add(-2 * time.Hour),
ExpiresAt: now.Add(-time.Hour),
}); err != nil {
t.Fatalf("upsert track cache: %v", err)
}
featuresPayload := []byte(`{"danceability":0.5,"energy":0.6,"loudness":-7,"speechiness":0.03,"acousticness":0.2,"instrumentalness":0,"liveness":0.1,"valence":0.4,"tempo":100,"time_signature":4,"key":1,"mode":1}`)
if err := store.UpsertProviderCache(context.Background(), provider.CacheEntry{
Provider: provider.ProviderSpotify,
ItemType: "audio_features",
ItemID: "cached",
Payload: featuresPayload,
FetchedAt: now.Add(-2 * time.Hour),
ExpiresAt: now.Add(-time.Hour),
}); err != nil {
t.Fatalf("upsert features cache: %v", err)
}
resp, err := service.ImportSpotify(context.Background(), provider.ImportRequest{
Source: provider.Source{Type: "url", Value: "https://open.spotify.com/track/cached"},
Market: "US",
})
if err != nil {
t.Fatalf("import with stale cache: %v", err)
}
if resp.ImportedTracks != 1 || len(resp.Warnings) == 0 {
t.Fatalf("expected stale fallback import with warning, got %+v", resp)
}
}
func fakeSpotifyServer(t *testing.T) *httptest.Server {
t.Helper()
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
switch r.URL.Path {
case "/v1/search":
if got := r.URL.Query().Get("limit"); got != "10" {
t.Fatalf("search limit = %q, want 10", got)
}
_ = json.NewEncoder(w).Encode(map[string]any{
"tracks": map[string]any{"items": []map[string]any{{"id": "good"}}},
})
case "/v1/tracks/good":
_ = json.NewEncoder(w).Encode(map[string]any{
"id": "good",
"name": "Good Song",
"artists": []map[string]any{{"id": "spotify-artist", "name": "Good Artist"}},
"album": map[string]any{"id": "album", "name": "Good Album", "release_date": "2024-01-01", "images": []map[string]any{{"url": "https://img.example/good.jpg"}}},
"duration_ms": 210000,
"popularity": 80,
"explicit": false,
"external_ids": map[string]string{"isrc": "USRC17607839"},
"external_urls": map[string]string{
"spotify": "https://open.spotify.com/track/good",
},
})
case "/v1/audio-features/good":
_ = json.NewEncoder(w).Encode(map[string]any{
"danceability": 0.7,
"energy": 0.8,
"loudness": -5.0,
"speechiness": 0.04,
"acousticness": 0.1,
"instrumentalness": 0.0,
"liveness": 0.12,
"valence": 0.6,
"tempo": 120,
"time_signature": 4,
"key": 2,
"mode": 1,
})
default:
http.NotFound(w, r)
}
}))
}
func fakeMusicBrainzServer(t *testing.T) *httptest.Server {
t.Helper()
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if got := r.Header.Get("User-Agent"); got == "" {
t.Fatal("missing User-Agent")
}
w.Header().Set("Content-Type", "application/json")
switch r.URL.Path {
case "/ws/2/isrc/USRC17607839":
_ = json.NewEncoder(w).Encode(map[string]any{
"recordings": []map[string]any{{
"id": "mb-recording",
"title": "Good Song",
"artist-credit": []map[string]any{{
"artist": map[string]string{"id": "mb-artist", "name": "Good Artist"},
}},
"isrcs": []string{"USRC17607839"},
"tags": []map[string]string{{"name": "indie"}},
}},
})
default:
http.NotFound(w, r)
}
}))
}
apps/backend/internal/provider/songlink/client.go
+202
View File
@@ -0,0 +1,202 @@
// Package songlink provides a client for the Song.link/Odesli API.
// Song.link offers free cross-platform music URL mapping.
package songlink
import (
"encoding/json"
"fmt"
"net/http"
"net/url"
"sync"
"time"
)
const (
apiBase = "https://api.song.link/v1-alpha.1"
minRequestInterval = 7 * time.Second
maxRequestsPerMinute = 9
)
// PlatformLink represents a link to a track on a specific platform
type PlatformLink struct {
Platform string `json:"platform"`
URL string `json:"url"`
EntityType string `json:"entity_type"`
ID string `json:"id,omitempty"`
NativeURI string `json:"native_uri,omitempty"`
}
// CrossPlatformLinks holds links for a track across multiple platforms
type CrossPlatformLinks struct {
SpotifyID string `json:"spotify_id"`
ISRC string `json:"isrc,omitempty"`
Links map[string]PlatformLink `json:"links"`
}
// Client for Song.link API
type Client struct {
httpClient *http.Client
lastRequestTime time.Time
requestCount int
countResetTime time.Time
mu sync.Mutex
}
// NewClient creates a new Song.link client
func NewClient() *Client {
return &Client{
httpClient: &http.Client{
Timeout: 30 * time.Second,
},
countResetTime: time.Now(),
}
}
// Configured always returns true for Song.link (no API key needed)
func (c *Client) Configured() bool {
return true
}
func (c *Client) rateLimit() {
c.mu.Lock()
defer c.mu.Unlock()
now := time.Now()
// Reset counter every minute
if now.Sub(c.countResetTime) >= time.Minute {
c.requestCount = 0
c.countResetTime = now
}
// Check if we've hit the per-minute limit
if c.requestCount >= maxRequestsPerMinute {
waitTime := time.Minute - now.Sub(c.countResetTime)
if waitTime > 0 {
time.Sleep(waitTime)
c.requestCount = 0
c.countResetTime = time.Now()
}
}
// Ensure minimum interval between requests
elapsed := now.Sub(c.lastRequestTime)
if elapsed < minRequestInterval {
time.Sleep(minRequestInterval - elapsed)
}
c.lastRequestTime = time.Now()
c.requestCount++
}
// GetLinksFromSpotifyID gets cross-platform links from a Spotify track ID
func (c *Client) GetLinksFromSpotifyID(spotifyID string) (*CrossPlatformLinks, error) {
spotifyURL := fmt.Sprintf("https://open.spotify.com/track/%s", spotifyID)
return c.GetLinks(spotifyURL)
}
// GetLinks gets cross-platform links from any music URL
func (c *Client) GetLinks(musicURL string) (*CrossPlatformLinks, error) {
c.rateLimit()
params := url.Values{
"url": {musicURL},
"userCountry": {"US"},
}
apiURL := fmt.Sprintf("%s/links?%s", apiBase, params.Encode())
req, err := http.NewRequest("GET", apiURL, nil)
if err != nil {
return nil, err
}
req.Header.Set("User-Agent", "SpotifyRecAlg/1.0")
req.Header.Set("Accept", "application/json")
resp, err := c.httpClient.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode == http.StatusTooManyRequests {
// Rate limited - wait and retry once
retryAfter := 15
if ra := resp.Header.Get("Retry-After"); ra != "" {
fmt.Sscanf(ra, "%d", &retryAfter)
}
time.Sleep(time.Duration(retryAfter) * time.Second)
return c.GetLinks(musicURL)
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("song.link API error: HTTP %d", resp.StatusCode)
}
var data struct {
EntityUniqueID string `json:"entityUniqueId"`
UserCountry string `json:"userCountry"`
PageURL string `json:"pageUrl"`
LinksByPlatform map[string]struct {
URL string `json:"url"`
EntityUniqueID string `json:"entityUniqueId"`
} `json:"linksByPlatform"`
EntitiesByUniqueID map[string]struct {
ID string `json:"id"`
Type string `json:"type"`
Title string `json:"title"`
Artist string `json:"artistName"`
ThumbnailURL string `json:"thumbnailUrl"`
APIProvider string `json:"apiProvider"`
Platforms []string `json:"platforms"`
} `json:"entitiesByUniqueId"`
}
if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
return nil, err
}
links := &CrossPlatformLinks{
Links: make(map[string]PlatformLink),
}
// Extract Spotify ID
for uniqueID, entity := range data.EntitiesByUniqueID {
if entity.APIProvider == "spotify" {
links.SpotifyID = entity.ID
}
if entity.Type == "song" {
// ISRC can sometimes be derived from the unique ID format
_ = uniqueID
}
}
// Platform name mapping
platformNames := map[string]string{
"spotify": "spotify",
"tidal": "tidal",
"qobuz": "qobuz",
"amazonMusic": "amazonMusic",
"amazonStore": "amazon",
"deezer": "deezer",
"appleMusic": "appleMusic",
"youtube": "youtube",
"youtubeMusic": "youtubeMusic",
"soundcloud": "soundcloud",
"napster": "napster",
"pandora": "pandora",
}
for platform, linkData := range data.LinksByPlatform {
if name, ok := platformNames[platform]; ok {
links.Links[name] = PlatformLink{
Platform: platform,
URL: linkData.URL,
EntityType: "track",
}
}
}
return links, nil
}
apps/backend/internal/provider/spotify/client.go
+463
View File
@@ -0,0 +1,463 @@
package spotify
import (
"bytes"
"context"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"strconv"
"strings"
"sync"
"time"
)
const (
defaultAccountsBaseURL = "https://accounts.spotify.com"
defaultAPIBaseURL = "https://api.spotify.com/v1"
defaultTimeout = 10 * time.Second
)
var ErrNotConfigured = errors.New("spotify credentials are not configured")
type Config struct {
ClientID string
ClientSecret string
BearerToken string
Market string
AccountsBaseURL string
APIBaseURL string
HTTPClient *http.Client
Timeout time.Duration
MaxRetries int
}
type Client struct {
clientID string
clientSecret string
staticToken string
defaultMarket string
accountsBaseURL string
apiBaseURL string
httpClient *http.Client
timeout time.Duration
maxRetries int
mu sync.Mutex
token string
expiresAt time.Time
lastError string
}
func New(cfg Config) *Client {
timeout := cfg.Timeout
if timeout <= 0 {
timeout = defaultTimeout
}
httpClient := cfg.HTTPClient
if httpClient == nil {
httpClient = &http.Client{Timeout: timeout}
}
accountsBaseURL := strings.TrimRight(cfg.AccountsBaseURL, "/")
if accountsBaseURL == "" {
accountsBaseURL = defaultAccountsBaseURL
}
apiBaseURL := strings.TrimRight(cfg.APIBaseURL, "/")
if apiBaseURL == "" {
apiBaseURL = defaultAPIBaseURL
}
maxRetries := cfg.MaxRetries
if maxRetries <= 0 {
maxRetries = 2
}
return &Client{
clientID: strings.TrimSpace(cfg.ClientID),
clientSecret: strings.TrimSpace(cfg.ClientSecret),
staticToken: strings.TrimSpace(cfg.BearerToken),
defaultMarket: strings.ToUpper(strings.TrimSpace(cfg.Market)),
accountsBaseURL: accountsBaseURL,
apiBaseURL: apiBaseURL,
httpClient: httpClient,
timeout: timeout,
maxRetries: maxRetries,
}
}
func (c *Client) Configured() bool {
return c.staticToken != "" || (c.clientID != "" && c.clientSecret != "")
}
func (c *Client) TokenMode() string {
if c.staticToken != "" {
return "static_bearer"
}
if c.clientID != "" && c.clientSecret != "" {
return "client_credentials"
}
return "unconfigured"
}
func (c *Client) LastError() string {
c.mu.Lock()
defer c.mu.Unlock()
return c.lastError
}
func (c *Client) GetTrack(ctx context.Context, id, market string) (Track, []byte, error) {
var out Track
payload, err := c.get(ctx, "/tracks/"+url.PathEscape(id), marketParams(marketOrDefault(market, c.defaultMarket)), &out)
return out, payload, err
}
func (c *Client) GetAudioFeatures(ctx context.Context, id string) (AudioFeatures, []byte, error) {
var out AudioFeatures
payload, err := c.get(ctx, "/audio-features/"+url.PathEscape(id), nil, &out)
return out, payload, err
}
func (c *Client) Search(ctx context.Context, query, itemType, market string, limit int) (SearchResult, []byte, error) {
itemType = strings.ToLower(strings.TrimSpace(itemType))
if itemType == "" {
itemType = "track"
}
if limit <= 0 {
limit = 5
}
if limit > 10 {
limit = 10
}
params := url.Values{}
params.Set("q", query)
params.Set("type", itemType)
params.Set("limit", strconv.Itoa(limit))
if market = marketOrDefault(market, c.defaultMarket); market != "" {
params.Set("market", market)
}
var out SearchResult
payload, err := c.get(ctx, "/search", params, &out)
return out, payload, err
}
func (c *Client) GetAlbumTracks(ctx context.Context, id, market string, limit int) ([]TrackRef, []byte, error) {
return c.getPagedTrackRefs(ctx, "/albums/"+url.PathEscape(id)+"/tracks", "items", market, limit)
}
func (c *Client) GetPlaylistTracks(ctx context.Context, id, market string, limit int) ([]TrackRef, []byte, error) {
limit = normalizeCollectionLimit(limit)
refs := make([]TrackRef, 0, limit)
var lastPayload []byte
for offset := 0; len(refs) < limit; offset += 50 {
params := marketParams(marketOrDefault(market, c.defaultMarket))
params.Set("limit", strconv.Itoa(minInt(50, limit-len(refs))))
params.Set("offset", strconv.Itoa(offset))
params.Set("fields", "items(track(id,is_local,type)),next")
payload, err := c.getRaw(ctx, "/playlists/"+url.PathEscape(id)+"/tracks", params)
if err != nil {
return nil, payload, err
}
lastPayload = payload
var page struct {
Items []struct {
Track TrackRef `json:"track"`
} `json:"items"`
Next string `json:"next"`
}
if err := json.Unmarshal(payload, &page); err != nil {
return nil, payload, fmt.Errorf("decode playlist tracks: %w", err)
}
for _, item := range page.Items {
if item.Track.ID != "" && !item.Track.IsLocal {
refs = append(refs, item.Track)
if len(refs) >= limit {
break
}
}
}
if page.Next == "" || len(page.Items) == 0 {
break
}
}
return refs, lastPayload, nil
}
func (c *Client) GetArtistTopTracks(ctx context.Context, id, market string) ([]Track, []byte, error) {
params := marketParams(marketOrDefault(market, c.defaultMarket))
if params.Get("market") == "" {
params.Set("market", "US")
}
var out struct {
Tracks []Track `json:"tracks"`
}
payload, err := c.get(ctx, "/artists/"+url.PathEscape(id)+"/top-tracks", params, &out)
return out.Tracks, payload, err
}
func (c *Client) getPagedTrackRefs(ctx context.Context, path, listField, market string, limit int) ([]TrackRef, []byte, error) {
limit = normalizeCollectionLimit(limit)
refs := make([]TrackRef, 0, limit)
var lastPayload []byte
for offset := 0; len(refs) < limit; offset += 50 {
params := marketParams(marketOrDefault(market, c.defaultMarket))
params.Set("limit", strconv.Itoa(minInt(50, limit-len(refs))))
params.Set("offset", strconv.Itoa(offset))
payload, err := c.getRaw(ctx, path, params)
if err != nil {
return nil, payload, err
}
lastPayload = payload
var page struct {
Items []TrackRef `json:"items"`
Next string `json:"next"`
}
if err := json.Unmarshal(payload, &page); err != nil {
return nil, payload, fmt.Errorf("decode %s: %w", listField, err)
}
for _, item := range page.Items {
if item.ID != "" {
refs = append(refs, item)
if len(refs) >= limit {
break
}
}
}
if page.Next == "" || len(page.Items) == 0 {
break
}
}
return refs, lastPayload, nil
}
func (c *Client) get(ctx context.Context, path string, params url.Values, out any) ([]byte, error) {
payload, err := c.getRaw(ctx, path, params)
if err != nil {
return payload, err
}
if err := json.Unmarshal(payload, out); err != nil {
return payload, fmt.Errorf("decode spotify response: %w", err)
}
return payload, nil
}
func (c *Client) getRaw(ctx context.Context, path string, params url.Values) ([]byte, error) {
if params == nil {
params = url.Values{}
}
endpoint := c.apiBaseURL + path
if encoded := params.Encode(); encoded != "" {
endpoint += "?" + encoded
}
return c.doJSON(ctx, http.MethodGet, endpoint, nil, true)
}
func (c *Client) accessToken(ctx context.Context) (string, error) {
if c.staticToken != "" {
return c.staticToken, nil
}
if c.clientID == "" || c.clientSecret == "" {
c.setLastError(ErrNotConfigured.Error())
return "", ErrNotConfigured
}
c.mu.Lock()
if c.token != "" && time.Now().Add(60*time.Second).Before(c.expiresAt) {
token := c.token
c.mu.Unlock()
return token, nil
}
c.mu.Unlock()
body := strings.NewReader("grant_type=client_credentials")
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.accountsBaseURL+"/api/token", body)
if err != nil {
return "", err
}
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
credential := base64.StdEncoding.EncodeToString([]byte(c.clientID + ":" + c.clientSecret))
req.Header.Set("Authorization", "Basic "+credential)
resp, err := c.httpClient.Do(req)
if err != nil {
c.setLastError(err.Error())
return "", err
}
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode > 299 {
_, _ = io.Copy(io.Discard, resp.Body)
err := fmt.Errorf("spotify token request failed with status %d", resp.StatusCode)
c.setLastError(err.Error())
return "", err
}
var decoded struct {
AccessToken string `json:"access_token"`
ExpiresIn int `json:"expires_in"`
TokenType string `json:"token_type"`
}
if err := json.NewDecoder(resp.Body).Decode(&decoded); err != nil {
c.setLastError(err.Error())
return "", fmt.Errorf("decode spotify token: %w", err)
}
if decoded.AccessToken == "" {
err := errors.New("spotify token response did not include an access token")
c.setLastError(err.Error())
return "", err
}
expiresIn := time.Duration(decoded.ExpiresIn) * time.Second
if expiresIn <= 0 {
expiresIn = time.Hour
}
c.mu.Lock()
c.token = decoded.AccessToken
c.expiresAt = time.Now().Add(expiresIn)
c.lastError = ""
c.mu.Unlock()
return decoded.AccessToken, nil
}
func (c *Client) doJSON(ctx context.Context, method, endpoint string, body []byte, authenticate bool) ([]byte, error) {
var lastErr error
for attempt := 0; attempt <= c.maxRetries; attempt++ {
if attempt > 0 {
select {
case <-ctx.Done():
return nil, ctx.Err()
case <-time.After(time.Duration(attempt) * 250 * time.Millisecond):
}
}
var reader io.Reader
if len(body) > 0 {
reader = bytes.NewReader(body)
}
req, err := http.NewRequestWithContext(ctx, method, endpoint, reader)
if err != nil {
return nil, err
}
req.Header.Set("Accept", "application/json")
if authenticate {
token, err := c.accessToken(ctx)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+token)
}
resp, err := c.httpClient.Do(req)
if err != nil {
if ctx.Err() != nil {
return nil, ctx.Err()
}
lastErr = err
continue
}
payload, readErr := io.ReadAll(io.LimitReader(resp.Body, 8<<20))
closeErr := resp.Body.Close()
if readErr != nil {
return payload, readErr
}
if closeErr != nil {
return payload, closeErr
}
if resp.StatusCode >= 200 && resp.StatusCode <= 299 {
c.setLastError("")
return payload, nil
}
if resp.StatusCode == http.StatusUnauthorized {
c.mu.Lock()
c.token = ""
c.expiresAt = time.Time{}
c.mu.Unlock()
}
lastErr = spotifyHTTPError{StatusCode: resp.StatusCode, Body: string(payload)}
if resp.StatusCode == http.StatusTooManyRequests {
wait := retryAfter(resp.Header.Get("Retry-After"))
if wait > 0 && attempt < c.maxRetries {
select {
case <-ctx.Done():
return payload, ctx.Err()
case <-time.After(wait):
continue
}
}
}
if resp.StatusCode < 500 && resp.StatusCode != http.StatusUnauthorized && resp.StatusCode != http.StatusTooManyRequests {
break
}
}
if lastErr == nil {
lastErr = errors.New("spotify request failed")
}
c.setLastError(lastErr.Error())
return nil, lastErr
}
func (c *Client) setLastError(message string) {
c.mu.Lock()
defer c.mu.Unlock()
c.lastError = message
}
type spotifyHTTPError struct {
StatusCode int
Body string
}
func (e spotifyHTTPError) Error() string {
if e.Body == "" {
return fmt.Sprintf("spotify request failed with status %d", e.StatusCode)
}
return fmt.Sprintf("spotify request failed with status %d", e.StatusCode)
}
func IsNotFound(err error) bool {
var httpErr spotifyHTTPError
return errors.As(err, &httpErr) && httpErr.StatusCode == http.StatusNotFound
}
func retryAfter(value string) time.Duration {
if value == "" {
return 0
}
if seconds, err := strconv.Atoi(strings.TrimSpace(value)); err == nil {
return time.Duration(seconds) * time.Second
}
if when, err := http.ParseTime(value); err == nil {
return time.Until(when)
}
return 0
}
func marketParams(market string) url.Values {
params := url.Values{}
if market = strings.ToUpper(strings.TrimSpace(market)); market != "" {
params.Set("market", market)
}
return params
}
func marketOrDefault(market, fallback string) string {
if market = strings.ToUpper(strings.TrimSpace(market)); market != "" {
return market
}
return strings.ToUpper(strings.TrimSpace(fallback))
}
func normalizeCollectionLimit(limit int) int {
if limit <= 0 {
return 100
}
if limit > 100 {
return 100
}
return limit
}
func minInt(a, b int) int {
if a < b {
return a
}
return b
}
apps/backend/internal/provider/spotify/client_test.go
+111
View File
@@ -0,0 +1,111 @@
package spotify
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"sync/atomic"
"testing"
"time"
)
func TestClientCredentialsTokenIsCached(t *testing.T) {
var tokenRequests atomic.Int64
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/token":
tokenRequests.Add(1)
if got := r.Header.Get("Authorization"); !strings.HasPrefix(got, "Basic ") {
t.Fatalf("missing basic authorization header")
}
_ = json.NewEncoder(w).Encode(map[string]any{"access_token": "token-a", "expires_in": 3600, "token_type": "Bearer"})
case "/v1/tracks/abc":
if got := r.Header.Get("Authorization"); got != "Bearer token-a" {
t.Fatalf("got authorization %q", got)
}
writeTrack(w, "abc")
default:
http.NotFound(w, r)
}
}))
defer server.Close()
client := New(Config{
ClientID: "client-id",
ClientSecret: "client-secret",
AccountsBaseURL: server.URL,
APIBaseURL: server.URL + "/v1",
})
for i := 0; i < 2; i++ {
if _, _, err := client.GetTrack(context.Background(), "abc", "US"); err != nil {
t.Fatalf("get track %d: %v", i, err)
}
}
if got := tokenRequests.Load(); got != 1 {
t.Fatalf("token requests = %d, want 1", got)
}
}
func TestClientRetriesRateLimitedRequest(t *testing.T) {
var calls atomic.Int64
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v1/tracks/abc" {
if calls.Add(1) == 1 {
w.Header().Set("Retry-After", "0")
w.WriteHeader(http.StatusTooManyRequests)
return
}
writeTrack(w, "abc")
return
}
http.NotFound(w, r)
}))
defer server.Close()
client := New(Config{BearerToken: "token", APIBaseURL: server.URL + "/v1", MaxRetries: 1})
if _, _, err := client.GetTrack(context.Background(), "abc", "US"); err != nil {
t.Fatalf("get track after retry: %v", err)
}
if got := calls.Load(); got != 2 {
t.Fatalf("calls = %d, want 2", got)
}
}
func TestClientReportsMalformedJSONAndContextCancellation(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write([]byte(`{`))
}))
defer server.Close()
client := New(Config{BearerToken: "token", APIBaseURL: server.URL, MaxRetries: 0})
if _, _, err := client.GetTrack(context.Background(), "abc", "US"); err == nil {
t.Fatal("expected malformed JSON error")
}
ctx, cancel := context.WithCancel(context.Background())
cancel()
if _, _, err := client.GetTrack(ctx, "abc", "US"); err == nil {
t.Fatal("expected context cancellation error")
}
}
func writeTrack(w http.ResponseWriter, id string) {
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(Track{
ID: id,
Name: "Track",
Artists: []Artist{{Name: "Artist"}},
Album: Album{Name: "Album", ReleaseDate: "2024-01-01"},
DurationMS: int((3 * time.Minute).Milliseconds()),
Popularity: 77,
ExternalIDs: map[string]string{
"isrc": "USRC17607839",
},
ExternalURLs: map[string]string{
"spotify": "https://open.spotify.com/track/" + id,
},
})
}
apps/backend/internal/provider/spotify/models.go
+75
View File
@@ -0,0 +1,75 @@
package spotify
type Track struct {
ID string `json:"id"`
Name string `json:"name"`
Artists []Artist `json:"artists"`
Album Album `json:"album"`
DurationMS int `json:"duration_ms"`
Popularity int `json:"popularity"`
Explicit bool `json:"explicit"`
ExternalIDs map[string]string `json:"external_ids"`
ExternalURLs map[string]string `json:"external_urls"`
Type string `json:"type"`
IsLocal bool `json:"is_local"`
}
type TrackRef struct {
ID string `json:"id"`
Type string `json:"type"`
IsLocal bool `json:"is_local"`
}
type Artist struct {
ID string `json:"id"`
Name string `json:"name"`
Genres []string `json:"genres"`
ExternalURLs map[string]string `json:"external_urls"`
}
type Album struct {
ID string `json:"id"`
Name string `json:"name"`
ReleaseDate string `json:"release_date"`
Images []Image `json:"images"`
Artists []Artist `json:"artists"`
}
type Image struct {
URL string `json:"url"`
Height int `json:"height"`
Width int `json:"width"`
}
type AudioFeatures struct {
Danceability float64 `json:"danceability"`
Energy float64 `json:"energy"`
Loudness float64 `json:"loudness"`
Speechiness float64 `json:"speechiness"`
Acousticness float64 `json:"acousticness"`
Instrumentalness float64 `json:"instrumentalness"`
Liveness float64 `json:"liveness"`
Valence float64 `json:"valence"`
Tempo float64 `json:"tempo"`
TimeSignature float64 `json:"time_signature"`
Key float64 `json:"key"`
Mode float64 `json:"mode"`
}
type SearchResult struct {
Tracks struct {
Items []Track `json:"items"`
} `json:"tracks"`
Albums struct {
Items []Album `json:"items"`
} `json:"albums"`
Artists struct {
Items []Artist `json:"items"`
} `json:"artists"`
Playlists struct {
Items []struct {
ID string `json:"id"`
Name string `json:"name"`
} `json:"items"`
} `json:"playlists"`
}
apps/backend/internal/provider/spotify/parser.go
+153
View File
@@ -0,0 +1,153 @@
package spotify
import (
"errors"
"net/url"
"regexp"
"strings"
)
var (
uriPattern = regexp.MustCompile(`(?i)^spotify:(track|album|playlist|artist):([A-Za-z0-9]+)$`)
idPattern = regexp.MustCompile(`^[A-Za-z0-9]{10,}$`)
pathIDPattern = regexp.MustCompile(`^[A-Za-z0-9]+$`)
)
type ParsedSource struct {
Type string
ID string
URL string
}
func ParseSource(sourceType, value string) (ParsedSource, error) {
sourceType = strings.ToLower(strings.TrimSpace(sourceType))
value = strings.TrimSpace(value)
if value == "" {
return ParsedSource{}, errors.New("source value is required")
}
if sourceType == "" || sourceType == "url" {
parsed, err := ParseURL(value)
if err == nil {
return parsed, nil
}
if sourceType == "url" {
return ParsedSource{}, err
}
}
if sourceType == "" {
sourceType = "track"
}
if !validSpotifyType(sourceType) {
return ParsedSource{}, errors.New("source type must be track, album, playlist, artist, or url")
}
if parsed, err := ParseURL(value); err == nil {
if parsed.Type != sourceType {
return ParsedSource{}, errors.New("source URL type does not match requested type")
}
return parsed, nil
}
if !idPattern.MatchString(value) {
return ParsedSource{}, errors.New("source value must be a Spotify ID, URI, or open.spotify.com URL")
}
return ParsedSource{Type: sourceType, ID: value, URL: "https://open.spotify.com/" + sourceType + "/" + value}, nil
}
func ParseURL(raw string) (ParsedSource, error) {
raw = strings.TrimSpace(raw)
if raw == "" {
return ParsedSource{}, errors.New("url is required")
}
if match := uriPattern.FindStringSubmatch(raw); len(match) == 3 {
return ParsedSource{Type: strings.ToLower(match[1]), ID: match[2], URL: "https://open.spotify.com/" + strings.ToLower(match[1]) + "/" + match[2]}, nil
}
parsedURL, err := parseURLWithDefaultScheme(raw)
if err == nil {
if value := parsedURL.Query().Get("uri"); value != "" {
return ParseURL(value)
}
host := spotifyHost(parsedURL.Host)
switch host {
case "open.spotify.com", "play.spotify.com":
if parsed, ok := parseSpotifyPath(parsedURL.Path); ok {
parsed.URL = canonicalURL(parsed.Type, parsed.ID)
return parsed, nil
}
case "embed.spotify.com":
if parsed, ok := parseSpotifyPath(parsedURL.Path); ok {
parsed.URL = canonicalURL(parsed.Type, parsed.ID)
return parsed, nil
}
}
}
return ParsedSource{}, errors.New("unsupported Spotify URL")
}
func parseURLWithDefaultScheme(raw string) (*url.URL, error) {
if strings.Contains(raw, "://") {
return url.Parse(raw)
}
lower := strings.ToLower(raw)
if strings.HasPrefix(lower, "open.spotify.com/") ||
strings.HasPrefix(lower, "play.spotify.com/") ||
strings.HasPrefix(lower, "embed.spotify.com/") {
return url.Parse("https://" + raw)
}
return url.Parse(raw)
}
func spotifyHost(host string) string {
host = strings.ToLower(strings.TrimSpace(host))
host = strings.TrimPrefix(host, "www.")
return host
}
func parseSpotifyPath(path string) (ParsedSource, bool) {
parts := pathSegments(path)
if len(parts) == 0 {
return ParsedSource{}, false
}
if strings.HasPrefix(strings.ToLower(parts[0]), "intl-") {
parts = parts[1:]
}
if len(parts) > 0 && strings.EqualFold(parts[0], "embed") {
parts = parts[1:]
}
if len(parts) >= 4 && strings.EqualFold(parts[0], "user") && strings.EqualFold(parts[2], "playlist") && pathIDPattern.MatchString(parts[3]) {
return ParsedSource{Type: "playlist", ID: parts[3]}, true
}
itemType := strings.ToLower(parts[0])
if len(parts) >= 2 && validSpotifyType(itemType) && pathIDPattern.MatchString(parts[1]) {
return ParsedSource{Type: itemType, ID: parts[1]}, true
}
return ParsedSource{}, false
}
func pathSegments(path string) []string {
rawParts := strings.Split(path, "/")
parts := make([]string, 0, len(rawParts))
for _, part := range rawParts {
part = strings.TrimSpace(part)
if part != "" {
parts = append(parts, part)
}
}
return parts
}
func canonicalURL(itemType, id string) string {
return "https://open.spotify.com/" + itemType + "/" + id
}
func validSpotifyType(value string) bool {
switch value {
case "track", "album", "playlist", "artist":
return true
default:
return false
}
}
apps/backend/internal/provider/spotify/parser_test.go
+43
View File
@@ -0,0 +1,43 @@
package spotify
import "testing"
func TestParseSource(t *testing.T) {
tests := []struct {
name string
sourceType string
value string
wantType string
wantID string
wantErr bool
}{
{name: "track URL", sourceType: "url", value: "https://open.spotify.com/track/abc123XYZ?si=ignored", wantType: "track", wantID: "abc123XYZ"},
{name: "intl track URL", sourceType: "url", value: "https://open.spotify.com/intl-cs/track/7tFiyTwD0nx5a1eklYtX2J?si=ignored", wantType: "track", wantID: "7tFiyTwD0nx5a1eklYtX2J"},
{name: "embed URI URL", sourceType: "url", value: "https://embed.spotify.com/?uri=spotify:track:7tFiyTwD0nx5a1eklYtX2J", wantType: "track", wantID: "7tFiyTwD0nx5a1eklYtX2J"},
{name: "album URI", sourceType: "url", value: "spotify:album:album123456", wantType: "album", wantID: "album123456"},
{name: "album URL with inferred type", sourceType: "", value: "https://open.spotify.com/album/1GbtB4zTqAsyfZEsm1RZfx", wantType: "album", wantID: "1GbtB4zTqAsyfZEsm1RZfx"},
{name: "playlist URL", sourceType: "playlist", value: "https://open.spotify.com/playlist/pl123456", wantType: "playlist", wantID: "pl123456"},
{name: "legacy user playlist URL", sourceType: "url", value: "https://open.spotify.com/user/someone/playlist/pl123456", wantType: "playlist", wantID: "pl123456"},
{name: "artist ID", sourceType: "artist", value: "artist123456", wantType: "artist", wantID: "artist123456"},
{name: "invalid URL", sourceType: "url", value: "https://example.com/track/abc", wantErr: true},
{name: "type mismatch", sourceType: "track", value: "https://open.spotify.com/album/abc123456", wantErr: true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := ParseSource(tt.sourceType, tt.value)
if tt.wantErr {
if err == nil {
t.Fatal("expected error")
}
return
}
if err != nil {
t.Fatalf("parse source: %v", err)
}
if got.Type != tt.wantType || got.ID != tt.wantID {
t.Fatalf("got type=%q id=%q, want type=%q id=%q", got.Type, got.ID, tt.wantType, tt.wantID)
}
})
}
}
apps/backend/internal/provider/types.go
+135
View File
@@ -0,0 +1,135 @@
package provider
import (
"context"
"time"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/recommendation"
)
const (
ProviderSpotify = "spotify"
ProviderMusicBrainz = "musicbrainz"
)
type Source struct {
Type string `json:"type" binding:"required"`
Value string `json:"value" binding:"required"`
}
type ImportRequest struct {
Source Source `json:"source" binding:"required"`
Market string `json:"market,omitempty"`
Limit int `json:"limit,omitempty"`
EnrichMusicBrainz *bool `json:"enrich_musicbrainz,omitempty"`
Persist *bool `json:"persist,omitempty"`
AllowMissingFields bool `json:"allow_missing_features,omitempty"`
}
type SearchRequest struct {
Query string `json:"query" binding:"required"`
Type string `json:"type,omitempty"`
Market string `json:"market,omitempty"`
Limit int `json:"limit,omitempty"`
Persist bool `json:"persist"`
EnrichMusicBrainz *bool `json:"enrich_musicbrainz,omitempty"`
AllowMissingFields bool `json:"allow_missing_features,omitempty"`
}
type EnrichRequest struct {
TrackIDs []string `json:"track_ids" binding:"required"`
Force bool `json:"force"`
}
type ImportResponse struct {
ImportID string `json:"import_id"`
ImportedTracks int `json:"imported_tracks"`
UpdatedTracks int `json:"updated_tracks"`
Skipped int `json:"skipped"`
Warnings []string `json:"warnings"`
}
type SearchResponse struct {
Tracks []recommendation.Track `json:"tracks"`
Persisted int `json:"persisted"`
Skipped int `json:"skipped"`
Warnings []string `json:"warnings"`
}
type EnrichResponse struct {
Updated int `json:"updated"`
Skipped int `json:"skipped"`
Warnings []string `json:"warnings"`
}
type StatusResponse struct {
Spotify ProviderStatus `json:"spotify"`
MusicBrainz ProviderStatus `json:"musicbrainz"`
Cache CacheStats `json:"cache"`
}
type ProviderStatus struct {
Configured bool `json:"configured"`
TokenMode string `json:"token_mode,omitempty"`
Available bool `json:"available"`
LastError string `json:"last_error,omitempty"`
CheckedAt time.Time `json:"checked_at"`
}
type CacheEntry struct {
Provider string
ItemType string
ItemID string
Market string
Payload []byte
FetchedAt time.Time
ExpiresAt time.Time
LastError string
}
func (e CacheEntry) Fresh(now time.Time) bool {
return len(e.Payload) > 0 && now.Before(e.ExpiresAt)
}
type CacheStats struct {
Entries int64 `json:"entries"`
FreshEntries int64 `json:"fresh_entries"`
StaleEntries int64 `json:"stale_entries"`
}
type ImportJob struct {
ID string
Provider string
SourceType string
SourceValue string
Market string
Status string
ImportedTracks int
UpdatedTracks int
Skipped int
Warnings []string
StartedAt time.Time
FinishedAt time.Time
}
type TrackEnrichment struct {
TrackID string
Provider string
MusicBrainzRecordingID string
MusicBrainzArtistID string
ISRC string
Payload []byte
UpdatedAt time.Time
}
type Store interface {
UpsertTrack(ctx context.Context, track recommendation.Track) error
UpsertTracks(ctx context.Context, tracks []recommendation.Track) error
GetTracksByIDs(ctx context.Context, ids []string) ([]recommendation.Track, error)
GetProviderCache(ctx context.Context, providerName, itemType, itemID, market string) (CacheEntry, bool, error)
UpsertProviderCache(ctx context.Context, entry CacheEntry) error
ProviderCacheStats(ctx context.Context) (CacheStats, error)
CreateImportJob(ctx context.Context, job ImportJob) error
FinishImportJob(ctx context.Context, job ImportJob) error
UpsertTrackEnrichment(ctx context.Context, enrichment TrackEnrichment) error
}
apps/backend/internal/provider/unlocker/client.go
+174
View File
@@ -0,0 +1,174 @@
package unlocker
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"time"
)
var (
ErrNotConfigured = errors.New("unlocker service not configured")
ErrTrackNotFound = errors.New("track not found")
)
// Client for the Python unlocker service (auth-free Spotify access)
type Client struct {
baseURL string
client *http.Client
}
func NewClient(baseURL string) *Client {
if baseURL == "" {
return nil
}
return &Client{
baseURL: baseURL,
client: &http.Client{Timeout: 30 * time.Second},
}
}
func (c *Client) Configured() bool {
return c != nil && c.baseURL != ""
}
type TrackResponse struct {
ID string `json:"id"`
Title string `json:"title"`
Artist string `json:"artist"`
Artists []string `json:"artists"`
Album string `json:"album"`
DurationMS int `json:"duration_ms"`
Explicit bool `json:"explicit"`
ExternalURLs map[string]string `json:"external_urls"`
}
type ImportRequest struct {
URL string `json:"url"`
}
type ImportResponse struct {
Track *TrackResponse `json:"track"`
Links map[string]string `json:"links"`
Parsed *ParsedInfo `json:"parsed,omitempty"`
Note string `json:"note,omitempty"`
}
type ParsedInfo struct {
Service string `json:"service"`
Type string `json:"type"`
ID string `json:"id"`
}
type LinksResponse struct {
SpotifyID string `json:"spotify_id"`
ISRC string `json:"isrc"`
Links map[string]LinkDetails `json:"links"`
}
type LinkDetails struct {
URL string `json:"url"`
ID string `json:"id"`
}
// ImportFromURL imports a track from any streaming service URL (auth-free)
func (c *Client) ImportFromURL(ctx context.Context, url string) (*ImportResponse, error) {
if !c.Configured() {
return nil, ErrNotConfigured
}
reqBody, _ := json.Marshal(ImportRequest{URL: url})
req, err := http.NewRequestWithContext(ctx, "POST", c.baseURL+"/import", bytes.NewReader(reqBody))
if err != nil {
return nil, err
}
req.Header.Set("Content-Type", "application/json")
resp, err := c.client.Do(req)
if err != nil {
return nil, fmt.Errorf("unlocker request failed: %w", err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("unlocker returned %d: %s", resp.StatusCode, string(body))
}
var result ImportResponse
if err := json.Unmarshal(body, &result); err != nil {
return nil, fmt.Errorf("failed to parse unlocker response: %w", err)
}
return &result, nil
}
// GetTrack gets a track by Spotify ID (auth-free)
func (c *Client) GetTrack(ctx context.Context, trackID string) (*TrackResponse, error) {
if !c.Configured() {
return nil, ErrNotConfigured
}
req, err := http.NewRequestWithContext(ctx, "GET", c.baseURL+"/spotify/track/"+trackID, nil)
if err != nil {
return nil, err
}
resp, err := c.client.Do(req)
if err != nil {
return nil, fmt.Errorf("unlocker request failed: %w", err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
if resp.StatusCode == http.StatusNotFound {
return nil, ErrTrackNotFound
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("unlocker returned %d: %s", resp.StatusCode, string(body))
}
var result TrackResponse
if err := json.Unmarshal(body, &result); err != nil {
return nil, fmt.Errorf("failed to parse unlocker response: %w", err)
}
return &result, nil
}
// GetLinks gets cross-platform links for a Spotify track
func (c *Client) GetLinks(ctx context.Context, spotifyID string) (*LinksResponse, error) {
if !c.Configured() {
return nil, ErrNotConfigured
}
req, err := http.NewRequestWithContext(ctx, "GET", c.baseURL+"/links/"+spotifyID, nil)
if err != nil {
return nil, err
}
resp, err := c.client.Do(req)
if err != nil {
return nil, fmt.Errorf("unlocker request failed: %w", err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("unlocker returned %d: %s", resp.StatusCode, string(body))
}
var result LinksResponse
if err := json.Unmarshal(body, &result); err != nil {
return nil, fmt.Errorf("failed to parse unlocker response: %w", err)
}
return &result, nil
}
apps/backend/internal/provider/urlparser/parser.go
+265
View File
@@ -0,0 +1,265 @@
// Package urlparser provides universal music URL parsing for multiple streaming services.
package urlparser
import (
neturl "net/url"
"regexp"
"strings"
)
// Service represents a music streaming service
type Service string
const (
Spotify Service = "spotify"
Tidal Service = "tidal"
AppleMusic Service = "apple_music"
YouTube Service = "youtube"
YouTubeMusic Service = "youtube_music"
SoundCloud Service = "soundcloud"
Deezer Service = "deezer"
Bandcamp Service = "bandcamp"
MusicBrainz Service = "musicbrainz"
)
// ParsedURL represents a parsed music service URL
type ParsedURL struct {
Service Service
URL string
ItemType string
ID string
Metadata map[string]string
}
// Parser for music service URLs
type Parser struct {
patterns map[Service][]*regexp.Regexp
services []Service
}
// NewParser creates a new URL parser
func NewParser() *Parser {
return &Parser{
services: []Service{
Spotify,
Tidal,
AppleMusic,
YouTubeMusic,
YouTube,
SoundCloud,
Deezer,
Bandcamp,
MusicBrainz,
},
patterns: map[Service][]*regexp.Regexp{
Spotify: {
regexp.MustCompile(`(?i)^spotify:(track|album|playlist|artist):([a-zA-Z0-9]+)$`),
regexp.MustCompile(`(?i)https?://open\.spotify\.com/(?:intl-[a-z]{2}/)?(?:embed/)?(track|album|playlist|artist)/([a-zA-Z0-9]+)`),
regexp.MustCompile(`(?i)https://spotify\.link/([a-zA-Z0-9]+)`),
},
Tidal: {
regexp.MustCompile(`(?i)https://tidal\.com/(?:browse/)?(track|album|playlist|artist)/(\d+)`),
regexp.MustCompile(`(?i)https://listen\.tidal\.com/(?:browse/)?(track|album|playlist|artist)/(\d+)`),
},
AppleMusic: {
regexp.MustCompile(`(?i)https://music\.apple\.com/([a-z]{2})/(song|album|playlist|artist)/(?:[^/]+/)?(\d+)`),
},
YouTubeMusic: {
regexp.MustCompile(`(?i)https://music\.youtube\.com/(watch|playlist|channel)\?([^#]+)`),
},
YouTube: {
regexp.MustCompile(`(?i)https://(?:www\.)?youtube\.com/watch\?v=([a-zA-Z0-9_-]+)`),
regexp.MustCompile(`(?i)https://youtu\.be/([a-zA-Z0-9_-]+)`),
regexp.MustCompile(`(?i)https://(?:www\.)?youtube\.com/playlist\?list=([a-zA-Z0-9_-]+)`),
},
SoundCloud: {
regexp.MustCompile(`(?i)https://soundcloud\.com/([^/]+)/sets/([^/?#]+)`),
regexp.MustCompile(`(?i)https://soundcloud\.com/([^/]+)/([^/]+)`),
},
Deezer: {
regexp.MustCompile(`(?i)https://www\.deezer\.com/(?:[a-z]{2}/)?(track|album|playlist|artist)/(\d+)`),
},
Bandcamp: {
regexp.MustCompile(`(?i)https://([a-zA-Z0-9-]+)\.bandcamp\.com/(track|album)/(.+)`),
},
MusicBrainz: {
regexp.MustCompile(`(?i)https://musicbrainz\.org/(recording|release|release-group|artist)/([a-f0-9-]+)`),
},
},
}
}
// ParseURL parses a music service URL and extracts service, type, and ID
func (p *Parser) ParseURL(url string) *ParsedURL {
url = strings.TrimSpace(url)
if url == "" {
return nil
}
for _, service := range p.services {
patterns := p.patterns[service]
for _, pattern := range patterns {
matches := pattern.FindStringSubmatch(url)
if matches != nil {
return p.extractServiceInfo(service, matches, url)
}
}
}
return nil
}
func (p *Parser) extractServiceInfo(service Service, matches []string, url string) *ParsedURL {
switch service {
case Spotify:
if len(matches) >= 3 {
return &ParsedURL{
Service: service,
URL: url,
ItemType: matches[1],
ID: matches[2],
}
}
if len(matches) == 2 {
return &ParsedURL{
Service: service,
URL: url,
ItemType: "short",
ID: matches[1],
}
}
case Tidal:
if len(matches) >= 3 {
return &ParsedURL{
Service: service,
URL: url,
ItemType: matches[1],
ID: matches[2],
}
}
case AppleMusic:
if len(matches) >= 4 {
itemType := matches[2]
id := matches[3]
if parsed, err := neturl.Parse(url); err == nil && itemType == "album" {
if trackID := parsed.Query().Get("i"); trackID != "" {
itemType = "song"
id = trackID
}
}
return &ParsedURL{
Service: service,
URL: url,
ItemType: itemType,
ID: id,
Metadata: map[string]string{
"region": matches[1],
},
}
}
case YouTube, YouTubeMusic:
if parsed, err := neturl.Parse(url); err == nil {
if v := parsed.Query().Get("v"); v != "" {
return &ParsedURL{Service: service, URL: url, ItemType: "video", ID: v}
}
if list := parsed.Query().Get("list"); list != "" {
return &ParsedURL{Service: service, URL: url, ItemType: "playlist", ID: list}
}
}
return &ParsedURL{
Service: service,
URL: url,
ItemType: "video",
ID: matches[1],
}
case SoundCloud:
if len(matches) >= 3 {
itemType := "track"
if strings.EqualFold(matches[1], "sets") || strings.Contains(strings.ToLower(url), "/sets/") {
itemType = "playlist"
}
return &ParsedURL{
Service: service,
URL: url,
ItemType: itemType,
ID: matches[1] + "/" + matches[2],
}
}
case Deezer:
if len(matches) >= 3 {
return &ParsedURL{
Service: service,
URL: url,
ItemType: matches[1],
ID: matches[2],
}
}
case Bandcamp:
if len(matches) >= 4 {
return &ParsedURL{
Service: service,
URL: url,
ItemType: matches[2],
ID: matches[1] + "/" + matches[3],
}
}
case MusicBrainz:
if len(matches) >= 3 {
return &ParsedURL{
Service: service,
URL: url,
ItemType: matches[1],
ID: matches[2],
}
}
}
return nil
}
// GetServiceFromURL quickly identifies the service from a URL without full parsing
func (p *Parser) GetServiceFromURL(url string) Service {
urlLower := strings.ToLower(url)
if strings.Contains(urlLower, "spotify.com") || strings.Contains(urlLower, "spotify.link") {
return Spotify
}
if strings.Contains(urlLower, "tidal.com") || strings.Contains(urlLower, "listen.tidal.com") {
return Tidal
}
if strings.Contains(urlLower, "music.apple.com") {
return AppleMusic
}
if strings.Contains(urlLower, "music.youtube.com") {
return YouTubeMusic
}
if strings.Contains(urlLower, "youtube.com") || strings.Contains(urlLower, "youtu.be") {
return YouTube
}
if strings.Contains(urlLower, "soundcloud.com") {
return SoundCloud
}
if strings.Contains(urlLower, "deezer.com") {
return Deezer
}
if strings.Contains(urlLower, "bandcamp.com") {
return Bandcamp
}
if strings.Contains(urlLower, "musicbrainz.org") {
return MusicBrainz
}
return ""
}
// ValidateURL checks if a URL is from a supported service
func (p *Parser) ValidateURL(url string) bool {
return p.ParseURL(url) != nil
}
apps/backend/internal/provider/urlparser/parser_test.go
+34
View File
@@ -0,0 +1,34 @@
package urlparser
import "testing"
func TestParseURLDetectsSupportedMusicLinks(t *testing.T) {
parser := NewParser()
tests := []struct {
name string
url string
service Service
itemType string
id string
}{
{name: "spotify intl", url: "https://open.spotify.com/intl-us/track/7tFiyTwD0nx5a1eklYtX2J?si=x", service: Spotify, itemType: "track", id: "7tFiyTwD0nx5a1eklYtX2J"},
{name: "spotify uri", url: "spotify:album:1GbtB4zTqAsyfZEsm1RZfx", service: Spotify, itemType: "album", id: "1GbtB4zTqAsyfZEsm1RZfx"},
{name: "apple album track", url: "https://music.apple.com/us/album/example/1440857781?i=1440857782", service: AppleMusic, itemType: "song", id: "1440857782"},
{name: "youtube music video", url: "https://music.youtube.com/watch?v=abc_DEF-123&si=x", service: YouTubeMusic, itemType: "video", id: "abc_DEF-123"},
{name: "youtube playlist", url: "https://www.youtube.com/playlist?list=PL123", service: YouTube, itemType: "playlist", id: "PL123"},
{name: "soundcloud set", url: "https://soundcloud.com/artist/sets/mixtape", service: SoundCloud, itemType: "playlist", id: "artist/mixtape"},
{name: "tidal", url: "https://listen.tidal.com/browse/track/12345", service: Tidal, itemType: "track", id: "12345"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := parser.ParseURL(tt.url)
if got == nil {
t.Fatal("expected parsed URL")
}
if got.Service != tt.service || got.ItemType != tt.itemType || got.ID != tt.id {
t.Fatalf("got service=%q type=%q id=%q, want service=%q type=%q id=%q", got.Service, got.ItemType, got.ID, tt.service, tt.itemType, tt.id)
}
})
}
}
apps/backend/internal/provider/webplayer/client.go
+815
View File
@@ -0,0 +1,815 @@
// Package webplayer provides a Go native Spotify Web Player client using TOTP authentication.
// This is a port of the Python implementation, allowing auth-free access to Spotify metadata.
package webplayer
import (
"bytes"
"crypto/hmac"
"crypto/sha1"
"encoding/base32"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"math/rand"
"net/http"
"net/http/cookiejar"
"net/url"
"regexp"
"strconv"
"strings"
"sync"
"time"
)
const (
// Hardcoded TOTP secret from Spotify Web Player (publicly known)
totpSecret = "GM3TMMJTGYZTQNZVGM4DINJZHA4TGOBYGMZTCMRTGEYDSMJRHE4TEOBUG4YTCMRUGQ4DQOJUGQYTAMRRGA2TCMJSHE3TCMBY"
totpVersion = 61
clientVersion = "1.2.40"
minRequestInterval = 100 * time.Millisecond
)
// GraphQL persisted query hashes
var graphqlHashes = map[string]string{
"getTrack": "612585ae06ba435ad26369870deaae23b5c8800a256cd8a57e08eddc25a37294",
"getAlbum": "b9bfabef66ed756e5e13f68a942deb60bd4125ec1f1be8cc42769dc0259b4b10",
"fetchPlaylist": "bb67e0af06e8d6f52b531f97468ee4acd44cd0f82b988e15c2ea47b1148efc77",
"getArtist": "2e7f695dd9c0a6591c2d4f3b9e6e0a7c8d5b4a3f2e1d0c9b8a7f6e5d4c3b2a1",
}
// Track represents Spotify track metadata
type Track struct {
ID string `json:"id"`
Name string `json:"name"`
Artists []Artist `json:"artists"`
Album Album `json:"album"`
DurationMs int `json:"duration_ms"`
Explicit bool `json:"explicit"`
ExternalURLs map[string]string `json:"external_urls"`
}
// Artist represents a Spotify artist
type Artist struct {
ID string `json:"id"`
Name string `json:"name"`
URI string `json:"uri"`
}
// Album represents a Spotify album
type Album struct {
ID string `json:"id"`
Name string `json:"name"`
URI string `json:"uri"`
Images []Image `json:"images"`
}
// Image represents an image asset
type Image struct {
URL string `json:"url"`
Width int `json:"width"`
Height int `json:"height"`
}
// token holds the Spotify access token
type token struct {
AccessToken string
ClientID string
DeviceID string
ClientVersion string
ExpiresAt time.Time
ClientToken string
}
// Client is the Spotify Web Player API client
type Client struct {
httpClient *http.Client
baseURL string
token *token
mu sync.RWMutex
lastRequest time.Time
cookies map[string]string
}
// NewClient creates a new Web Player client
func NewClient() *Client {
jar, _ := cookiejar.New(nil)
return &Client{
httpClient: &http.Client{
Timeout: 30 * time.Second,
Jar: jar,
},
baseURL: "https://open.spotify.com",
cookies: make(map[string]string),
}
}
// Configured returns true if the client is functional (always true for this client)
func (c *Client) Configured() bool {
return true
}
// generateTOTP generates a TOTP code using the hardcoded secret
func generateTOTP() string {
// Base32 decode the secret
secretBytes, _ := base32.StdEncoding.DecodeString(totpSecret)
// Get current time in 30-second intervals
currentTime := uint64(time.Now().Unix() / 30)
// Convert to bytes (big-endian, 8 bytes)
timeBytes := make([]byte, 8)
for i := 7; i >= 0; i-- {
timeBytes[i] = byte(currentTime & 0xFF)
currentTime >>= 8
}
// HMAC-SHA1
h := hmac.New(sha1.New, secretBytes)
h.Write(timeBytes)
hmacResult := h.Sum(nil)
// Dynamic truncation
offset := hmacResult[len(hmacResult)-1] & 0x0F
code := int(hmacResult[offset]&0x7F)<<24 |
int(hmacResult[offset+1]&0xFF)<<16 |
int(hmacResult[offset+2]&0xFF)<<8 |
int(hmacResult[offset+3]&0xFF)
// Get 6-digit code
totpCode := fmt.Sprintf("%06d", code%1000000)
return totpCode
}
func (c *Client) rateLimit() {
c.mu.Lock()
defer c.mu.Unlock()
now := time.Now()
elapsed := now.Sub(c.lastRequest)
if elapsed < minRequestInterval {
time.Sleep(minRequestInterval - elapsed)
}
c.lastRequest = time.Now()
}
func (c *Client) ensureToken() error {
c.mu.RLock()
tok := c.token
c.mu.RUnlock()
if tok == nil || time.Now().After(tok.ExpiresAt.Add(-60*time.Second)) {
return c.getAccessToken()
}
if tok.ClientToken == "" {
return c.getClientToken()
}
return nil
}
func (c *Client) getAccessToken() error {
// Try TOTP generation first (same as official Web Player)
if err := c.getAccessTokenTOTP(); err == nil {
// Client token is optional
_ = c.getClientToken()
return nil
}
// Fall back to tokener API
if err := c.getAccessTokenTokener(); err == nil {
// Client token is optional - try to get it but don't fail if unavailable
_ = c.getClientToken()
return nil
}
return errors.New("failed to obtain access token")
}
func (c *Client) getAccessTokenTOTP() error {
c.rateLimit()
totpCode := generateTOTP()
params := url.Values{
"reason": {"init"},
"productType": {"web-player"},
"totp": {totpCode},
"totpVer": {strconv.Itoa(totpVersion)},
"totpServer": {totpCode},
}
tokenURL := fmt.Sprintf("%s/api/token?%s", c.baseURL, params.Encode())
req, err := http.NewRequest("GET", tokenURL, nil)
if err != nil {
return err
}
req.Header.Set("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36")
req.Header.Set("Accept", "application/json, text/plain, */*")
req.Header.Set("Accept-Language", "en-US,en;q=0.9")
req.Header.Set("Referer", "https://open.spotify.com/")
resp, err := c.httpClient.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
// Read body for debugging - check content length first
var bodyBytes []byte
if resp.ContentLength > 0 {
bodyBytes = make([]byte, resp.ContentLength)
_, err = io.ReadFull(resp.Body, bodyBytes)
if err != nil {
return fmt.Errorf("failed to read response body: %w", err)
}
} else {
bodyBytes, err = io.ReadAll(resp.Body)
if err != nil {
return fmt.Errorf("failed to read response body: %w", err)
}
}
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("TOTP token request failed: HTTP %d, body: %s, content-length: %d", resp.StatusCode, string(bodyBytes), resp.ContentLength)
}
// Extract cookies
for _, cookie := range resp.Cookies() {
c.cookies[cookie.Name] = cookie.Value
}
var data struct {
AccessToken string `json:"accessToken"`
ClientID string `json:"clientId"`
}
if err := json.Unmarshal(bodyBytes, &data); err != nil {
return fmt.Errorf("failed to decode JSON: %w, body: %s", err, string(bodyBytes))
}
deviceID := c.cookies["sp_t"]
if deviceID == "" {
deviceID = generateDeviceID()
}
c.mu.Lock()
c.token = &token{
AccessToken: data.AccessToken,
ClientID: data.ClientID,
DeviceID: deviceID,
ClientVersion: clientVersion,
ExpiresAt: time.Now().Add(time.Hour),
}
c.mu.Unlock()
return nil
}
func (c *Client) getAccessTokenTokener() error {
c.rateLimit()
resp, err := c.httpClient.Get("https://spotify-tokener-api.vercel.app/api/getToken")
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("tokener API failed: HTTP %d", resp.StatusCode)
}
var data struct {
AccessToken string `json:"accessToken"`
ClientID string `json:"clientId"`
}
if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
return err
}
if data.AccessToken == "" || data.ClientID == "" {
return errors.New("tokener API returned invalid data")
}
c.mu.Lock()
c.token = &token{
AccessToken: data.AccessToken,
ClientID: data.ClientID,
DeviceID: generateDeviceID(),
ClientVersion: clientVersion,
ExpiresAt: time.Now().Add(time.Hour),
}
c.mu.Unlock()
return nil
}
func (c *Client) getClientToken() error {
c.mu.RLock()
tok := c.token
c.mu.RUnlock()
if tok == nil {
return errors.New("no access token available")
}
c.rateLimit()
payload := map[string]interface{}{
"client_data": map[string]interface{}{
"client_version": tok.ClientVersion,
"client_id": tok.ClientID,
"js_sdk_data": map[string]interface{}{
"device_brand": "unknown",
"device_model": "unknown",
"os": "windows",
"os_version": "NT 10.0",
"device_id": tok.DeviceID,
"device_type": "computer",
},
},
}
jsonPayload, _ := json.Marshal(payload)
req, err := http.NewRequest("POST", "https://clienttoken.spotify.com/v1/clienttoken", bytes.NewReader(jsonPayload))
if err != nil {
return err
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36")
req.Header.Set("Accept", "application/json")
req.Header.Set("Accept-Language", "en-US,en;q=0.9")
resp, err := c.httpClient.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("client token request failed: HTTP %d", resp.StatusCode)
}
body, _ := io.ReadAll(resp.Body)
var data struct {
ResponseType string `json:"response_type"`
GrantedToken struct {
Token string `json:"token"`
} `json:"granted_token"`
}
if err := json.Unmarshal(body, &data); err != nil {
return err
}
if data.ResponseType != "RESPONSE_GRANTED_TOKEN_RESPONSE" {
return errors.New("invalid client token response type: " + data.ResponseType)
}
c.mu.Lock()
c.token.ClientToken = data.GrantedToken.Token
c.mu.Unlock()
return nil
}
func (c *Client) graphqlQuery(operationName string, variables map[string]interface{}) (map[string]interface{}, error) {
if err := c.ensureToken(); err != nil {
return nil, err
}
hash, ok := graphqlHashes[operationName]
if !ok {
return nil, fmt.Errorf("unknown GraphQL operation: %s", operationName)
}
c.mu.RLock()
tok := c.token
c.mu.RUnlock()
// Use struct with explicit field order to match Python's JSON key ordering
// The SHA256 hash is computed on the exact JSON string
payload := struct {
Variables map[string]interface{} `json:"variables"`
OperationName string `json:"operationName"`
Extensions struct {
PersistedQuery struct {
Version int `json:"version"`
Sha256Hash string `json:"sha256Hash"`
} `json:"persistedQuery"`
} `json:"extensions"`
}{
Variables: variables,
OperationName: operationName,
}
payload.Extensions.PersistedQuery.Version = 1
payload.Extensions.PersistedQuery.Sha256Hash = hash
jsonPayload, _ := json.Marshal(payload)
c.rateLimit()
req, err := http.NewRequest("POST", "https://api-partner.spotify.com/pathfinder/v1/query", bytes.NewReader(jsonPayload))
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+tok.AccessToken)
if tok.ClientToken != "" {
req.Header.Set("Client-Token", tok.ClientToken)
}
req.Header.Set("Spotify-App-Version", tok.ClientVersion)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json")
req.Header.Set("Accept-Language", "en-US,en;q=0.9")
req.Header.Set("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36")
resp, err := c.httpClient.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
if resp.StatusCode == http.StatusUnauthorized {
// Token expired, refresh and retry
c.mu.Lock()
c.token = nil
c.mu.Unlock()
if err := c.ensureToken(); err != nil {
return nil, err
}
// Retry request
c.mu.RLock()
tok = c.token
c.mu.RUnlock()
req.Header.Set("Authorization", "Bearer "+tok.AccessToken)
if tok.ClientToken != "" {
req.Header.Set("Client-Token", tok.ClientToken)
}
resp, err = c.httpClient.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, _ = io.ReadAll(resp.Body)
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("GraphQL query failed: HTTP %d: %s", resp.StatusCode, string(body))
}
var result map[string]interface{}
if err := json.Unmarshal(body, &result); err != nil {
return nil, err
}
return result, nil
}
// GetTrack fetches track metadata by ID
func (c *Client) GetTrack(trackID string) (*Track, error) {
variables := map[string]interface{}{
"uri": fmt.Sprintf("spotify:track:%s", trackID),
}
data, err := c.graphqlQuery("getTrack", variables)
if err != nil {
return nil, err
}
trackData, ok := getNestedMap(data, "data", "trackUnion")
if !ok {
return nil, errors.New("track not found in response")
}
if getString(trackData, "__typename") != "Track" {
return nil, errors.New("item is not a track")
}
// Extract artists
var artists []Artist
if firstArtist, ok := getNestedMap(trackData, "firstArtist"); ok {
if profile, ok := getNestedMap(firstArtist, "profile"); ok {
artists = append(artists, Artist{
ID: getString(firstArtist, "id"),
Name: getString(profile, "name"),
URI: getString(firstArtist, "uri"),
})
}
}
if otherArtists, ok := getNestedMap(trackData, "otherArtists"); ok {
if items, ok := otherArtists["items"].([]interface{}); ok {
for _, item := range items {
if artist, ok := item.(map[string]interface{}); ok {
if profile, ok := getNestedMap(artist, "profile"); ok {
artists = append(artists, Artist{
ID: getString(artist, "id"),
Name: getString(profile, "name"),
URI: getString(artist, "uri"),
})
}
}
}
}
}
// Extract album
var album Album
if albumData, ok := getNestedMap(trackData, "albumOfTrack"); ok {
album = Album{
ID: getString(albumData, "id"),
Name: getString(albumData, "name"),
URI: getString(albumData, "uri"),
}
if visualIdentity, ok := getNestedMap(albumData, "visualIdentity"); ok {
if avatarImage, ok := getNestedMap(visualIdentity, "avatarImage"); ok {
if sources, ok := avatarImage["sources"].([]interface{}); ok && len(sources) > 0 {
if img, ok := sources[0].(map[string]interface{}); ok {
album.Images = append(album.Images, Image{
URL: getString(img, "url"),
Width: int(getFloat(img, "width")),
Height: int(getFloat(img, "height")),
})
}
}
}
}
}
// Get duration
durationMs := 0
if duration, ok := getNestedMap(trackData, "duration"); ok {
durationMs = int(getFloat(duration, "totalMilliseconds"))
}
// Check explicit
explicit := false
if contentRating, ok := getNestedMap(trackData, "contentRating"); ok {
explicit = getString(contentRating, "label") == "EXPLICIT"
}
track := &Track{
ID: getString(trackData, "id"),
Name: getString(trackData, "name"),
Artists: artists,
Album: album,
DurationMs: durationMs,
Explicit: explicit,
ExternalURLs: map[string]string{
"spotify": fmt.Sprintf("https://open.spotify.com/track/%s", trackID),
},
}
return track, nil
}
// Search searches for tracks (uses public search endpoint)
func (c *Client) Search(query string, limit int) ([]Track, error) {
if err := c.ensureToken(); err != nil {
return nil, err
}
c.mu.RLock()
tok := c.token
c.mu.RUnlock()
if limit <= 0 {
limit = 20
}
if limit > 50 {
limit = 50
}
params := url.Values{
"q": {query},
"type": {"track"},
"limit": {strconv.Itoa(limit)},
"market": {"US"},
}
searchURL := fmt.Sprintf("https://api.spotify.com/v1/search?%s", params.Encode())
c.rateLimit()
req, err := http.NewRequest("GET", searchURL, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+tok.AccessToken)
resp, err := c.httpClient.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("search failed: HTTP %d", resp.StatusCode)
}
var data struct {
Tracks struct {
Items []struct {
ID string `json:"id"`
Name string `json:"name"`
Artists []struct {
ID string `json:"id"`
Name string `json:"name"`
} `json:"artists"`
Album struct {
ID string `json:"id"`
Name string `json:"name"`
Images []struct {
URL string `json:"url"`
Width int `json:"width"`
Height int `json:"height"`
} `json:"images"`
} `json:"album"`
DurationMs int `json:"duration_ms"`
Explicit bool `json:"explicit"`
} `json:"items"`
} `json:"tracks"`
}
if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
return nil, err
}
var tracks []Track
for _, item := range data.Tracks.Items {
var artists []Artist
for _, a := range item.Artists {
artists = append(artists, Artist{
ID: a.ID,
Name: a.Name,
})
}
var images []Image
for _, img := range item.Album.Images {
images = append(images, Image{
URL: img.URL,
Width: img.Width,
Height: img.Height,
})
}
tracks = append(tracks, Track{
ID: item.ID,
Name: item.Name,
Artists: artists,
DurationMs: item.DurationMs,
Explicit: item.Explicit,
Album: Album{
ID: item.Album.ID,
Name: item.Album.Name,
Images: images,
},
ExternalURLs: map[string]string{
"spotify": fmt.Sprintf("https://open.spotify.com/track/%s", item.ID),
},
})
}
return tracks, nil
}
// Helper functions
func generateDeviceID() string {
b := make([]byte, 16)
rand.Read(b)
return hex.EncodeToString(b)
}
func getNestedMap(m map[string]interface{}, keys ...string) (map[string]interface{}, bool) {
current := m
for _, key := range keys {
next, ok := current[key].(map[string]interface{})
if !ok {
return nil, false
}
current = next
}
return current, true
}
func getString(m map[string]interface{}, key string) string {
if v, ok := m[key].(string); ok {
return v
}
return ""
}
func getFloat(m map[string]interface{}, key string) float64 {
switch v := m[key].(type) {
case float64:
return v
case float32:
return float64(v)
case int:
return float64(v)
case string:
f, _ := strconv.ParseFloat(v, 64)
return f
}
return 0
}
// URL parsing helpers
var spotifyIDRegex = regexp.MustCompile(`^[A-Za-z0-9]{10,}$`)
// ParseSpotifyURL extracts the type and ID from a Spotify URL
func ParseSpotifyURL(urlStr string) (itemType, itemID string, err error) {
urlStr = strings.TrimSpace(urlStr)
if urlStr == "" {
return "", "", errors.New("invalid Spotify URL")
}
if matches := regexp.MustCompile(`(?i)^spotify:(track|album|playlist|artist):([A-Za-z0-9]+)$`).FindStringSubmatch(urlStr); len(matches) == 3 {
return strings.ToLower(matches[1]), matches[2], nil
}
parsed, parseErr := parseSpotifyWebURL(urlStr)
if parseErr != nil {
return "", "", parseErr
}
return parsed.itemType, parsed.itemID, nil
}
type parsedSpotifyWebURL struct {
itemType string
itemID string
}
func parseSpotifyWebURL(raw string) (parsedSpotifyWebURL, error) {
if !strings.Contains(raw, "://") {
lower := strings.ToLower(raw)
if strings.HasPrefix(lower, "open.spotify.com/") || strings.HasPrefix(lower, "play.spotify.com/") {
raw = "https://" + raw
}
}
u, err := url.Parse(raw)
if err != nil {
return parsedSpotifyWebURL{}, err
}
if value := u.Query().Get("uri"); value != "" {
itemType, itemID, err := ParseSpotifyURL(value)
if err != nil {
return parsedSpotifyWebURL{}, err
}
return parsedSpotifyWebURL{itemType: itemType, itemID: itemID}, nil
}
host := strings.TrimPrefix(strings.ToLower(u.Host), "www.")
if host != "open.spotify.com" && host != "play.spotify.com" && host != "embed.spotify.com" {
return parsedSpotifyWebURL{}, errors.New("invalid Spotify URL")
}
parts := make([]string, 0, 4)
for _, part := range strings.Split(u.Path, "/") {
part = strings.TrimSpace(part)
if part != "" {
parts = append(parts, part)
}
}
if len(parts) > 0 && strings.HasPrefix(strings.ToLower(parts[0]), "intl-") {
parts = parts[1:]
}
if len(parts) > 0 && strings.EqualFold(parts[0], "embed") {
parts = parts[1:]
}
if len(parts) >= 4 && strings.EqualFold(parts[0], "user") && strings.EqualFold(parts[2], "playlist") && spotifyIDRegex.MatchString(parts[3]) {
return parsedSpotifyWebURL{itemType: "playlist", itemID: parts[3]}, nil
}
if len(parts) >= 2 {
itemType := strings.ToLower(parts[0])
switch itemType {
case "track", "album", "playlist", "artist":
if spotifyIDRegex.MatchString(parts[1]) {
return parsedSpotifyWebURL{itemType: itemType, itemID: parts[1]}, nil
}
}
}
return parsedSpotifyWebURL{}, errors.New("invalid Spotify URL")
}
apps/backend/internal/provider/webplayer/client_test.go
+139
View File
@@ -0,0 +1,139 @@
package webplayer
import (
"os"
"testing"
)
func TestParseSpotifyURLVariants(t *testing.T) {
tests := []struct {
name string
url string
wantType string
wantID string
}{
{name: "open URL", url: "https://open.spotify.com/track/7tFiyTwD0nx5a1eklYtX2J?si=ignored", wantType: "track", wantID: "7tFiyTwD0nx5a1eklYtX2J"},
{name: "intl URL", url: "https://open.spotify.com/intl-cs/album/1GbtB4zTqAsyfZEsm1RZfx", wantType: "album", wantID: "1GbtB4zTqAsyfZEsm1RZfx"},
{name: "URI", url: "spotify:playlist:37i9dQZF1DXcBWIGoYBM5M", wantType: "playlist", wantID: "37i9dQZF1DXcBWIGoYBM5M"},
{name: "embed URI", url: "https://embed.spotify.com/?uri=spotify:track:7tFiyTwD0nx5a1eklYtX2J", wantType: "track", wantID: "7tFiyTwD0nx5a1eklYtX2J"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
itemType, itemID, err := ParseSpotifyURL(tt.url)
if err != nil {
t.Fatalf("parse: %v", err)
}
if itemType != tt.wantType || itemID != tt.wantID {
t.Fatalf("got type=%q id=%q, want type=%q id=%q", itemType, itemID, tt.wantType, tt.wantID)
}
})
}
}
// TestWebPlayerIntegration tests against real Spotify endpoints
// Run with: go test -v -run TestWebPlayerIntegration ./... -tags=integration
// Or set WEBPLAYER_TEST=1 environment variable
func TestWebPlayerIntegration(t *testing.T) {
if os.Getenv("WEBPLAYER_TEST") == "" {
t.Skip("Skipping integration test. Set WEBPLAYER_TEST=1 to run")
}
client := NewClient()
t.Run("GetTrack", func(t *testing.T) {
// Test with "Bohemian Rhapsody" - a well-known track
track, err := client.GetTrack("7tFiyTwD0nx5a1eklYtX2J")
if err != nil {
t.Fatalf("GetTrack failed: %v", err)
}
if track.ID == "" {
t.Error("track ID is empty")
}
if track.Name == "" {
t.Error("track name is empty")
}
if len(track.Artists) == 0 {
t.Error("no artists found")
}
if track.Album.Name == "" {
t.Error("album name is empty")
}
t.Logf("Got track: %s by %s (%d artists) from album %s, duration=%dms",
track.Name,
track.Artists[0].Name,
len(track.Artists),
track.Album.Name,
track.DurationMs,
)
})
t.Run("Search", func(t *testing.T) {
tracks, err := client.Search("Bohemian Rhapsody Queen", 5)
if err != nil {
t.Fatalf("Search failed: %v", err)
}
if len(tracks) == 0 {
t.Error("no tracks found in search results")
}
for i, track := range tracks {
t.Logf("Result %d: %s by %s", i+1, track.Name, track.Artists[0].Name)
}
})
t.Run("ParseSpotifyURL", func(t *testing.T) {
tests := []struct {
url string
wantType string
wantID string
}{
{
url: "https://open.spotify.com/track/7tFiyTwD0nx5a1eklYtX2J",
wantType: "track",
wantID: "7tFiyTwD0nx5a1eklYtX2J",
},
{
url: "https://open.spotify.com/album/1GbtB4zTqAsyfZEsm1RZfx",
wantType: "album",
wantID: "1GbtB4zTqAsyfZEsm1RZfx",
},
}
for _, tt := range tests {
itemType, itemID, err := ParseSpotifyURL(tt.url)
if err != nil {
t.Errorf("ParseSpotifyURL(%q) error: %v", tt.url, err)
continue
}
if itemType != tt.wantType {
t.Errorf("ParseSpotifyURL(%q) type = %q, want %q", tt.url, itemType, tt.wantType)
}
if itemID != tt.wantID {
t.Errorf("ParseSpotifyURL(%q) ID = %q, want %q", tt.url, itemID, tt.wantID)
}
}
})
}
// TestTOTPGeneration verifies TOTP generation produces valid codes
func TestTOTPGeneration(t *testing.T) {
totp := generateTOTP()
// TOTP should be 6 digits
if len(totp) != 6 {
t.Errorf("TOTP length = %d, want 6", len(totp))
}
// Should only contain digits
for _, c := range totp {
if c < '0' || c > '9' {
t.Errorf("TOTP contains non-digit character: %c", c)
}
}
t.Logf("Generated TOTP: %s", totp)
}
apps/backend/internal/recommendation/engine.go
+732
View File
@@ -0,0 +1,732 @@
package recommendation
import (
"cmp"
"context"
"errors"
"fmt"
"math"
"slices"
"strings"
"time"
)
type SnapshotProvider interface {
Snapshot(ctx context.Context, userID string) (CatalogSnapshot, error)
}
type Engine struct {
now func() time.Time
contentWeight float64
collabWeight float64
popularityWeight float64
explorationWeight float64
diversityLambda float64
}
type EngineConfig struct {
Now func() time.Time
ContentWeight float64
CollabWeight float64
PopularityWeight float64
ExplorationWeight float64
DiversityLambda float64
}
func NewEngine(cfg EngineConfig) *Engine {
if cfg.Now == nil {
cfg.Now = time.Now
}
return &Engine{
now: cfg.Now,
contentWeight: cfg.ContentWeight,
collabWeight: cfg.CollabWeight,
popularityWeight: cfg.PopularityWeight,
explorationWeight: cfg.ExplorationWeight,
diversityLambda: cfg.DiversityLambda,
}
}
func (e *Engine) Recommend(ctx context.Context, provider SnapshotProvider, req RecommendRequest) ([]Recommendation, TasteProfile, error) {
if strings.TrimSpace(req.UserID) == "" {
return nil, TasteProfile{}, errors.New("user_id is required")
}
if req.Limit <= 0 {
req.Limit = 20
}
if req.Limit > 100 {
req.Limit = 100
}
snapshot, err := provider.Snapshot(ctx, req.UserID)
if err != nil {
return nil, TasteProfile{}, err
}
if len(snapshot.Tracks) == 0 {
return nil, TasteProfile{}, errors.New("catalog is empty")
}
b := bounds(snapshot.Tracks)
byTrackID := indexTracks(snapshot.Tracks)
userInteractions := interactionsForUser(snapshot.Interactions, req.UserID)
tasteVector, positiveIDs, confidence, hasTasteAudio := e.tasteVector(snapshot.Tracks, byTrackID, userInteractions, req, b)
preferences := e.preferenceProfile(byTrackID, userInteractions, req)
neighborScores := e.collaborativeScores(snapshot.Interactions, req.UserID)
controls := mergeControls(snapshot.Controls, req)
candidates := make([]Recommendation, 0, len(snapshot.Tracks))
for _, track := range snapshot.Tracks {
if shouldFilter(track, positiveIDs, controls, req) {
continue
}
trackVector := normalize(track.Features, b)
trackHasAudio := hasAudioFeatures(track.Features)
metadataScore := metadataAffinity(track, preferences)
contentScore := metadataScore
if hasTasteAudio && trackHasAudio {
contentScore = clamp01(cosine(tasteVector, trackVector)*0.78 + metadataScore*0.22)
}
collabScore := clamp01(neighborScores[track.ID])
popularityScore := popularityFit(track.Popularity, req.Mode)
explorationScore := 0.5
if hasTasteAudio && trackHasAudio {
explorationScore = e.explorationScore(track, trackVector, tasteVector, req)
}
safetyScore := safetyScore(track, controls) * (1 - 0.52*negativeAffinity(track, preferences))
commercialScore := commercialScore(track, contentScore)
final := 0.0
final += e.contentWeight * contentScore
final += e.collabWeight * collabScore
final += e.popularityWeight * popularityScore
final += e.explorationWeight * explorationScore
final *= safetyScore
final += commercialScore
candidates = append(candidates, Recommendation{
Track: track,
Score: final,
Reason: reason(contentScore, collabScore, explorationScore, metadataScore, hasTasteAudio && trackHasAudio),
ScoreBreakdown: ScoreBreakdown{
Content: round(contentScore),
Collaborative: round(collabScore),
Popularity: round(popularityScore),
Exploration: round(explorationScore),
Safety: round(safetyScore),
Commercial: round(commercialScore),
Final: round(final),
},
Explanation: featureExplanation(tasteVector, trackVector),
})
}
slices.SortFunc(candidates, func(a, b Recommendation) int {
return cmp.Compare(b.Score, a.Score)
})
selected := e.diversify(candidates, req.Limit, b)
for i := range selected {
selected[i].Rank = i + 1
selected[i].Score = round(selected[i].Score)
selected[i].ScoreBreakdown.Final = selected[i].Score
}
profile := TasteProfile{
UserID: req.UserID,
Vector: arrayToSlice(tasteVector),
TopGenres: topGenres(snapshot.Tracks, byTrackID, userInteractions),
InteractionCount: len(userInteractions),
Confidence: round(confidence),
ExplorationReadiness: round(explorationReadiness(confidence, userInteractions)),
UpdatedAt: e.now().UTC(),
}
return selected, profile, nil
}
func (e *Engine) TasteProfile(ctx context.Context, provider SnapshotProvider, userID string) (TasteProfile, error) {
recs, profile, err := e.Recommend(ctx, provider, RecommendRequest{UserID: userID, Limit: 1})
if err != nil {
return TasteProfile{}, err
}
_ = recs
return profile, nil
}
func (e *Engine) tasteVector(tracks []Track, byTrackID map[string]Track, interactions []Interaction, req RecommendRequest, b featureBounds) ([featureCount]float64, map[string]struct{}, float64, bool) {
var sum [featureCount]float64
var total float64
var audioTotal float64
positive := make(map[string]struct{})
for _, seedID := range req.SeedTrackIDs {
track, ok := byTrackID[seedID]
if !ok {
continue
}
positive[seedID] = struct{}{}
if !hasAudioFeatures(track.Features) {
continue
}
addWeighted(&sum, normalize(track.Features, b), 1.25)
total += 1.25
audioTotal += 1.25
}
for _, interaction := range interactions {
track, ok := byTrackID[interaction.TrackID]
if !ok {
continue
}
weight := interactionWeight(interaction)
if weight > 0 {
positive[interaction.TrackID] = struct{}{}
}
if !hasAudioFeatures(track.Features) {
continue
}
decay := timeDecay(e.now(), interaction.OccurredAt)
addWeighted(&sum, normalize(track.Features, b), weight*decay)
total += math.Abs(weight * decay)
audioTotal += math.Abs(weight * decay)
}
if req.FeatureTargets != nil {
addWeighted(&sum, normalize(*req.FeatureTargets, b), 1.15)
total += 1.15
audioTotal += 1.15
}
if total == 0 {
return catalogCentroid(tracks, b), positive, 0, false
}
for i := range featureCount {
sum[i] = clamp01(sum[i] / total)
}
confidence := clamp01(math.Log1p(total) / math.Log(32))
return sum, positive, confidence, audioTotal > 0
}
func (e *Engine) collaborativeScores(interactions []Interaction, activeUserID string) map[string]float64 {
userRatings := make(map[string]map[string]float64)
for _, interaction := range interactions {
if userRatings[interaction.UserID] == nil {
userRatings[interaction.UserID] = make(map[string]float64)
}
userRatings[interaction.UserID][interaction.TrackID] += interactionWeight(interaction)
}
active := userRatings[activeUserID]
scores := make(map[string]float64)
if len(active) == 0 {
return scores
}
for userID, ratings := range userRatings {
if userID == activeUserID {
continue
}
similarity, overlap := pearson(active, ratings)
if similarity <= 0 {
continue
}
similarity *= float64(overlap) / float64(overlap+3)
for trackID, rating := range ratings {
if _, alreadyKnown := active[trackID]; alreadyKnown || rating <= 0 {
continue
}
scores[trackID] += similarity * rating
}
}
maxScore := 0.0
for _, score := range scores {
if score > maxScore {
maxScore = score
}
}
if maxScore == 0 {
return scores
}
for trackID, score := range scores {
scores[trackID] = clamp01(score / maxScore)
}
return scores
}
type preferenceProfile struct {
artists map[string]float64
genres map[string]float64
negativeArtists map[string]float64
negativeGenres map[string]float64
}
func (e *Engine) preferenceProfile(byTrackID map[string]Track, interactions []Interaction, req RecommendRequest) preferenceProfile {
profile := preferenceProfile{
artists: make(map[string]float64),
genres: make(map[string]float64),
negativeArtists: make(map[string]float64),
negativeGenres: make(map[string]float64),
}
for _, seedID := range req.SeedTrackIDs {
if track, ok := byTrackID[seedID]; ok {
addTrackPreference(profile.artists, profile.genres, track, 1.25)
}
}
for _, interaction := range interactions {
track, ok := byTrackID[interaction.TrackID]
if !ok {
continue
}
weight := interactionWeight(interaction) * timeDecay(e.now(), interaction.OccurredAt)
switch {
case weight > 0:
addTrackPreference(profile.artists, profile.genres, track, weight)
case weight < 0:
addTrackPreference(profile.negativeArtists, profile.negativeGenres, track, math.Abs(weight))
}
}
normalizeMap(profile.artists)
normalizeMap(profile.genres)
normalizeMap(profile.negativeArtists)
normalizeMap(profile.negativeGenres)
return profile
}
func addTrackPreference(artists, genres map[string]float64, track Track, weight float64) {
if artist := normalizedToken(track.Artist); artist != "" {
artists[artist] += weight
}
for _, genre := range track.Genres {
if genre = normalizedToken(genre); genre != "" {
genres[genre] += weight
}
}
}
func normalizeMap(values map[string]float64) {
maxValue := 0.0
for _, value := range values {
maxValue = math.Max(maxValue, value)
}
if maxValue == 0 {
return
}
for key, value := range values {
values[key] = clamp01(value / maxValue)
}
}
func metadataAffinity(track Track, profile preferenceProfile) float64 {
artistScore := profile.artists[normalizedToken(track.Artist)]
genreScore := genreAffinity(track.Genres, profile.genres)
switch {
case artistScore == 0 && genreScore == 0:
return 0.42
case artistScore == 0:
return clamp01(0.32 + 0.68*genreScore)
case genreScore == 0:
return clamp01(0.38 + 0.62*artistScore)
default:
return clamp01(0.48*artistScore + 0.52*genreScore)
}
}
func negativeAffinity(track Track, profile preferenceProfile) float64 {
artistScore := profile.negativeArtists[normalizedToken(track.Artist)]
genreScore := genreAffinity(track.Genres, profile.negativeGenres)
return clamp01(math.Max(artistScore*0.9, genreScore*0.7))
}
func genreAffinity(genres []string, profile map[string]float64) float64 {
best := 0.0
for _, genre := range genres {
best = math.Max(best, profile[normalizedToken(genre)])
}
return best
}
func normalizedToken(value string) string {
return strings.ToLower(strings.TrimSpace(value))
}
func popularityFit(popularity float64, mode string) float64 {
popularity = clamp01(popularity)
switch strings.ToLower(strings.TrimSpace(mode)) {
case "comfort":
return clamp01(0.35 + 0.65*popularity)
case "discovery":
return clamp01(1 - math.Abs(popularity-0.52)*1.25)
default:
familiarity := popularity
midTail := clamp01(1 - math.Abs(popularity-0.62)*1.15)
return clamp01(0.55*familiarity + 0.45*midTail)
}
}
func (e *Engine) explorationScore(track Track, trackVector, tasteVector [featureCount]float64, req RecommendRequest) float64 {
target := req.ExplorationTarget
if target == 0 {
target = 0.22
}
if strings.EqualFold(req.Mode, "discovery") {
target = math.Max(target, 0.34)
}
if strings.EqualFold(req.Mode, "comfort") {
target = math.Min(target, 0.10)
}
d := distance(trackVector, tasteVector)
return clamp01(1 - math.Abs(d-target))
}
func (e *Engine) diversify(candidates []Recommendation, limit int, b featureBounds) []Recommendation {
if len(candidates) <= limit {
return candidates
}
selected := make([]Recommendation, 0, limit)
remaining := slices.Clone(candidates)
for len(selected) < limit && len(remaining) > 0 {
bestIndex := 0
bestScore := math.Inf(-1)
for i, candidate := range remaining {
diversity := minDistanceToSelected(candidate.Track, selected, b)
score := e.diversityLambda*candidate.Score + (1-e.diversityLambda)*diversity
if score > bestScore {
bestScore = score
bestIndex = i
}
}
chosen := remaining[bestIndex]
chosen.ScoreBreakdown.Diversity = round(minDistanceToSelected(chosen.Track, selected, b))
selected = append(selected, chosen)
remaining = append(remaining[:bestIndex], remaining[bestIndex+1:]...)
}
return selected
}
func indexTracks(tracks []Track) map[string]Track {
out := make(map[string]Track, len(tracks))
for _, track := range tracks {
out[track.ID] = track
}
return out
}
func interactionsForUser(interactions []Interaction, userID string) []Interaction {
out := make([]Interaction, 0)
for _, interaction := range interactions {
if interaction.UserID == userID {
out = append(out, interaction)
}
}
return out
}
func interactionWeight(interaction Interaction) float64 {
if interaction.Weight != 0 {
return interaction.Weight
}
switch interaction.Type {
case InteractionLike:
return 1
case InteractionSave:
return 0.9
case InteractionPlay:
if interaction.CompletedMS > 30_000 {
return 0.45
}
return 0.20
case InteractionSkip:
return -0.55
case InteractionDislike:
return -1
case InteractionHide:
return -1.25
default:
return 0
}
}
func timeDecay(now, occurredAt time.Time) float64 {
if occurredAt.IsZero() {
return 0.7
}
days := now.Sub(occurredAt).Hours() / 24
if days <= 0 {
return 1
}
return math.Exp(-days / 120)
}
func addWeighted(sum *[featureCount]float64, value [featureCount]float64, weight float64) {
for i := range featureCount {
sum[i] += value[i] * weight
}
}
func catalogCentroid(tracks []Track, b featureBounds) [featureCount]float64 {
var sum [featureCount]float64
count := 0
for _, track := range tracks {
if !hasAudioFeatures(track.Features) {
continue
}
addWeighted(&sum, normalize(track.Features, b), 1)
count++
}
if count == 0 {
return sum
}
for i := range featureCount {
sum[i] /= float64(count)
}
return sum
}
func pearson(a, b map[string]float64) (float64, int) {
common := make([]string, 0)
for trackID := range a {
if _, ok := b[trackID]; ok {
common = append(common, trackID)
}
}
if len(common) < 2 {
return 0, len(common)
}
var meanA, meanB float64
for _, trackID := range common {
meanA += a[trackID]
meanB += b[trackID]
}
meanA /= float64(len(common))
meanB /= float64(len(common))
var numerator, denomA, denomB float64
for _, trackID := range common {
da := a[trackID] - meanA
db := b[trackID] - meanB
numerator += da * db
denomA += da * da
denomB += db * db
}
if denomA == 0 || denomB == 0 {
return 0, len(common)
}
return numerator / (math.Sqrt(denomA) * math.Sqrt(denomB)), len(common)
}
func shouldFilter(track Track, positive map[string]struct{}, controls UserControls, req RecommendRequest) bool {
if _, known := positive[track.ID]; known {
return true
}
if req.MinPopularity != nil && track.Popularity < *req.MinPopularity {
return true
}
if req.MaxPopularity != nil && track.Popularity > *req.MaxPopularity {
return true
}
includeExplicit := controls.AllowExplicit
if req.IncludeExplicit != nil {
includeExplicit = *req.IncludeExplicit
}
if track.Explicit && !includeExplicit {
return true
}
if contains(controls.ExcludedTracks, track.ID) || contains(controls.PostponedTracks, track.ID) {
return true
}
if contains(controls.ExcludedArtists, track.Artist) {
return true
}
for _, genre := range track.Genres {
if containsFold(controls.ExcludedGenres, genre) {
return true
}
}
return false
}
func mergeControls(controls UserControls, req RecommendRequest) UserControls {
if controls.UserID == "" {
controls.UserID = req.UserID
controls.AllowExplicit = true
}
controls.ExcludedTracks = append(controls.ExcludedTracks, req.ExcludedTrackIDs...)
controls.ExcludedArtists = append(controls.ExcludedArtists, req.ExcludedArtistIDs...)
controls.ExcludedGenres = append(controls.ExcludedGenres, req.ExcludedGenres...)
return controls
}
func safetyScore(track Track, controls UserControls) float64 {
if track.QualityPenalty > 0 {
return clamp01(1 - track.QualityPenalty)
}
if !controls.AllowExplicit && track.Explicit {
return 0
}
return 1
}
func commercialScore(track Track, contentScore float64) float64 {
if !track.DiscoveryAllowed || track.CommercialBoost <= 0 || contentScore < 0.72 {
return 0
}
return math.Min(track.CommercialBoost, 0.035)
}
func reason(contentScore, collabScore, explorationScore, metadataScore float64, hasAudioFeatures bool) string {
if !hasAudioFeatures && metadataScore >= 0.65 {
return "matched by genre, artist, and catalog signals while audio features were limited"
}
if !hasAudioFeatures {
return "balanced catalog match while audio features were limited"
}
switch {
case collabScore >= 0.65:
return "listeners with overlapping taste responded strongly to this track"
case explorationScore >= 0.82 && contentScore >= 0.58:
return "close enough to your taste profile while adding useful variety"
case contentScore >= 0.78:
return "audio features closely match your current taste profile"
default:
return "balanced recommendation from catalog, taste, and diversity signals"
}
}
func featureExplanation(taste, track [featureCount]float64) map[string]float64 {
out := make(map[string]float64, featureCount)
for i, name := range featureNames {
out[name] = round(1 - math.Abs(taste[i]-track[i]))
}
return out
}
func minDistanceToSelected(track Track, selected []Recommendation, b featureBounds) float64 {
if len(selected) == 0 {
return 1
}
minDistance := math.Inf(1)
for _, other := range selected {
d := trackDistance(track, other.Track, b)
if d < minDistance {
minDistance = d
}
}
return clamp01(minDistance)
}
func trackDistance(a, b Track, bounds featureBounds) float64 {
if hasAudioFeatures(a.Features) && hasAudioFeatures(b.Features) {
return distance(normalize(a.Features, bounds), normalize(b.Features, bounds))
}
if strings.EqualFold(a.Artist, b.Artist) && a.Artist != "" {
return 0.12
}
if genreOverlap(a.Genres, b.Genres) {
return 0.38
}
return 0.78
}
func genreOverlap(a, b []string) bool {
seen := make(map[string]struct{}, len(a))
for _, genre := range a {
if genre = normalizedToken(genre); genre != "" {
seen[genre] = struct{}{}
}
}
for _, genre := range b {
if _, ok := seen[normalizedToken(genre)]; ok {
return true
}
}
return false
}
func topGenres(tracks []Track, byTrackID map[string]Track, interactions []Interaction) map[string]float64 {
scores := make(map[string]float64)
for _, interaction := range interactions {
track, ok := byTrackID[interaction.TrackID]
if !ok {
continue
}
weight := interactionWeight(interaction)
if weight <= 0 {
continue
}
for _, genre := range track.Genres {
scores[strings.ToLower(genre)] += weight
}
}
maxScore := 0.0
for _, score := range scores {
maxScore = math.Max(maxScore, score)
}
if maxScore == 0 {
return scores
}
for genre, score := range scores {
scores[genre] = round(score / maxScore)
}
return scores
}
func explorationReadiness(confidence float64, interactions []Interaction) float64 {
negative := 0.0
for _, interaction := range interactions {
if interactionWeight(interaction) < 0 {
negative++
}
}
friction := 0.0
if len(interactions) > 0 {
friction = negative / float64(len(interactions))
}
return clamp01((0.45 + confidence*0.55) * (1 - friction*0.6))
}
func arrayToSlice(value [featureCount]float64) []float64 {
out := make([]float64, featureCount)
for i := range value {
out[i] = round(value[i])
}
return out
}
func contains(values []string, value string) bool {
return slices.Contains(values, value)
}
func containsFold(values []string, value string) bool {
for _, candidate := range values {
if strings.EqualFold(candidate, value) {
return true
}
}
return false
}
func round(value float64) float64 {
return math.Round(value*10000) / 10000
}
func ValidateTrack(track Track) error {
if strings.TrimSpace(track.ID) == "" {
return fmt.Errorf("track id is required")
}
if strings.TrimSpace(track.Title) == "" {
return fmt.Errorf("track title is required")
}
if strings.TrimSpace(track.Artist) == "" {
return fmt.Errorf("track artist is required")
}
return nil
}
apps/backend/internal/recommendation/engine_test.go
+161
View File
@@ -0,0 +1,161 @@
package recommendation
import (
"context"
"testing"
"time"
)
type testProvider struct {
snapshot CatalogSnapshot
}
func (p testProvider) Snapshot(context.Context, string) (CatalogSnapshot, error) {
return p.snapshot, nil
}
func TestRecommendBlendsContentAndCollaborativeSignals(t *testing.T) {
now := time.Date(2026, 4, 13, 12, 0, 0, 0, time.UTC)
engine := NewEngine(EngineConfig{
Now: func() time.Time { return now },
ContentWeight: 0.44,
CollabWeight: 0.28,
PopularityWeight: 0.08,
ExplorationWeight: 0.20,
DiversityLambda: 0.74,
})
tracks := []Track{
track("liked", "Known Good", "A", []string{"synth"}, 0.7, AudioFeatures{Danceability: 0.8, Energy: 0.8, Loudness: -5, Valence: 0.7, Tempo: 120, TimeSignature: 4, Key: 1, Mode: 1}),
track("neighbor", "Neighbor Pick", "B", []string{"synth"}, 0.6, AudioFeatures{Danceability: 0.76, Energy: 0.77, Loudness: -6, Valence: 0.66, Tempo: 121, TimeSignature: 4, Key: 2, Mode: 1}),
track("far", "Far Away", "C", []string{"folk"}, 0.5, AudioFeatures{Danceability: 0.2, Energy: 0.2, Loudness: -18, Acousticness: 0.9, Valence: 0.3, Tempo: 80, TimeSignature: 3, Key: 9, Mode: 0}),
}
recs, profile, err := engine.Recommend(context.Background(), testProvider{snapshot: CatalogSnapshot{
Tracks: tracks,
Interactions: []Interaction{
{UserID: "u1", TrackID: "liked", Type: InteractionLike, OccurredAt: now.Add(-time.Hour)},
{UserID: "u1", TrackID: "far", Type: InteractionSkip, OccurredAt: now.Add(-2 * time.Hour)},
{UserID: "n1", TrackID: "liked", Type: InteractionLike, OccurredAt: now.Add(-3 * time.Hour)},
{UserID: "n1", TrackID: "far", Type: InteractionSkip, OccurredAt: now.Add(-4 * time.Hour)},
{UserID: "n1", TrackID: "neighbor", Type: InteractionLike, OccurredAt: now.Add(-5 * time.Hour)},
},
Controls: UserControls{UserID: "u1", AllowExplicit: true},
}}, RecommendRequest{UserID: "u1", Limit: 2})
if err != nil {
t.Fatalf("recommend: %v", err)
}
if len(recs) == 0 {
t.Fatal("expected recommendations")
}
if recs[0].Track.ID != "neighbor" {
t.Fatalf("expected neighbor track first, got %q", recs[0].Track.ID)
}
if profile.Confidence <= 0 {
t.Fatalf("expected non-zero confidence, got %v", profile.Confidence)
}
}
func TestRecommendRespectsControls(t *testing.T) {
now := time.Date(2026, 4, 13, 12, 0, 0, 0, time.UTC)
engine := NewEngine(EngineConfig{Now: func() time.Time { return now }, ContentWeight: 1, DiversityLambda: 0.8})
explicit := track("explicit", "Explicit", "A", []string{"pop"}, 0.5, AudioFeatures{Danceability: 0.7, Energy: 0.7, Loudness: -6, Valence: 0.7, Tempo: 120, TimeSignature: 4})
explicit.Explicit = true
clean := track("clean", "Clean", "A", []string{"pop"}, 0.5, AudioFeatures{Danceability: 0.69, Energy: 0.71, Loudness: -6, Valence: 0.68, Tempo: 121, TimeSignature: 4})
recs, _, err := engine.Recommend(context.Background(), testProvider{snapshot: CatalogSnapshot{
Tracks: []Track{
track("seed", "Seed", "A", []string{"pop"}, 0.5, AudioFeatures{Danceability: 0.7, Energy: 0.7, Loudness: -6, Valence: 0.7, Tempo: 120, TimeSignature: 4}),
explicit,
clean,
},
Interactions: []Interaction{{UserID: "u1", TrackID: "seed", Type: InteractionLike, OccurredAt: now}},
Controls: UserControls{UserID: "u1", AllowExplicit: false},
}}, RecommendRequest{UserID: "u1", Limit: 10})
if err != nil {
t.Fatalf("recommend: %v", err)
}
for _, rec := range recs {
if rec.Track.ID == "explicit" {
t.Fatal("explicit track should be filtered")
}
}
if len(recs) != 1 || recs[0].Track.ID != "clean" {
t.Fatalf("expected only clean track, got %#v", recs)
}
}
func TestRecommendUsesMetadataWhenAudioFeaturesAreMissing(t *testing.T) {
now := time.Date(2026, 4, 13, 12, 0, 0, 0, time.UTC)
engine := NewEngine(EngineConfig{Now: func() time.Time { return now }, ContentWeight: 1, DiversityLambda: 0.85})
recs, _, err := engine.Recommend(context.Background(), testProvider{snapshot: CatalogSnapshot{
Tracks: []Track{
track("seed", "Seed", "Seed Artist", []string{"synthpop"}, 0.5, AudioFeatures{}),
track("genre-match", "Genre Match", "Other Artist", []string{"synthpop"}, 0.5, AudioFeatures{}),
track("unrelated", "Unrelated", "Far Artist", []string{"folk"}, 0.5, AudioFeatures{}),
},
Controls: UserControls{UserID: "u1", AllowExplicit: true},
}}, RecommendRequest{UserID: "u1", SeedTrackIDs: []string{"seed"}, Limit: 2})
if err != nil {
t.Fatalf("recommend: %v", err)
}
if len(recs) == 0 {
t.Fatal("expected recommendations")
}
if recs[0].Track.ID != "genre-match" {
t.Fatalf("expected metadata genre match first, got %q", recs[0].Track.ID)
}
for _, rec := range recs {
if rec.Track.ID == "seed" {
t.Fatal("seed track should not be recommended back")
}
}
}
func TestRecommendPenalizesSkippedNeighborhoods(t *testing.T) {
now := time.Date(2026, 4, 13, 12, 0, 0, 0, time.UTC)
engine := NewEngine(EngineConfig{
Now: func() time.Time { return now },
ContentWeight: 0.74,
PopularityWeight: 0.08,
ExplorationWeight: 0.18,
DiversityLambda: 0.9,
})
audio := AudioFeatures{Danceability: 0.74, Energy: 0.76, Loudness: -5, Speechiness: 0.05, Acousticness: 0.12, Instrumentalness: 0.04, Liveness: 0.12, Valence: 0.66, Tempo: 124, TimeSignature: 4, Key: 2, Mode: 1}
recs, _, err := engine.Recommend(context.Background(), testProvider{snapshot: CatalogSnapshot{
Tracks: []Track{
track("liked", "Liked", "A", []string{"dance"}, 0.7, audio),
track("skipped", "Skipped", "B", []string{"metal"}, 0.7, audio),
track("penalized", "Penalized", "C", []string{"metal"}, 0.7, audio),
track("safe", "Safe", "D", []string{"dance"}, 0.62, AudioFeatures{Danceability: 0.72, Energy: 0.74, Loudness: -6, Speechiness: 0.05, Acousticness: 0.14, Instrumentalness: 0.05, Liveness: 0.1, Valence: 0.64, Tempo: 125, TimeSignature: 4, Key: 3, Mode: 1}),
},
Interactions: []Interaction{
{UserID: "u1", TrackID: "liked", Type: InteractionLike, OccurredAt: now.Add(-time.Hour)},
{UserID: "u1", TrackID: "skipped", Type: InteractionSkip, OccurredAt: now.Add(-30 * time.Minute)},
},
Controls: UserControls{UserID: "u1", AllowExplicit: true},
}}, RecommendRequest{UserID: "u1", Limit: 2})
if err != nil {
t.Fatalf("recommend: %v", err)
}
if len(recs) == 0 {
t.Fatal("expected recommendations")
}
if recs[0].Track.ID != "safe" {
t.Fatalf("expected non-skipped neighborhood first, got %q", recs[0].Track.ID)
}
}
func track(id, title, artist string, genres []string, popularity float64, features AudioFeatures) Track {
return Track{
ID: id,
Title: title,
Artist: artist,
Genres: genres,
Popularity: popularity,
Features: features,
DiscoveryAllowed: true,
}
}
apps/backend/internal/recommendation/model.go
+127
View File
@@ -0,0 +1,127 @@
package recommendation
import "time"
type Track struct {
ID string `json:"id"`
Title string `json:"title"`
Artist string `json:"artist"`
Album string `json:"album,omitempty"`
Genres []string `json:"genres,omitempty"`
ReleaseDate string `json:"release_date,omitempty"`
DurationMS int `json:"duration_ms,omitempty"`
Popularity float64 `json:"popularity"`
Explicit bool `json:"explicit"`
Features AudioFeatures `json:"features"`
External map[string]string `json:"external,omitempty"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
CommercialBoost float64 `json:"commercial_boost,omitempty"`
QualityPenalty float64 `json:"quality_penalty,omitempty"`
DiscoveryAllowed bool `json:"discovery_allowed"`
}
type AudioFeatures struct {
Danceability float64 `json:"danceability"`
Energy float64 `json:"energy"`
Loudness float64 `json:"loudness"`
Speechiness float64 `json:"speechiness"`
Acousticness float64 `json:"acousticness"`
Instrumentalness float64 `json:"instrumentalness"`
Liveness float64 `json:"liveness"`
Valence float64 `json:"valence"`
Tempo float64 `json:"tempo"`
TimeSignature float64 `json:"time_signature"`
Key float64 `json:"key"`
Mode float64 `json:"mode"`
}
type InteractionType string
const (
InteractionPlay InteractionType = "play"
InteractionSkip InteractionType = "skip"
InteractionLike InteractionType = "like"
InteractionDislike InteractionType = "dislike"
InteractionSave InteractionType = "save"
InteractionHide InteractionType = "hide"
)
type Interaction struct {
UserID string `json:"user_id"`
TrackID string `json:"track_id"`
Type InteractionType `json:"type"`
Weight float64 `json:"weight,omitempty"`
OccurredAt time.Time `json:"occurred_at"`
Context Context `json:"context,omitempty"`
CompletedMS int `json:"completed_ms,omitempty"`
}
type Context struct {
Locale string `json:"locale,omitempty"`
Device string `json:"device,omitempty"`
TimeOfDay string `json:"time_of_day,omitempty"`
Activity string `json:"activity,omitempty"`
Mood string `json:"mood,omitempty"`
}
type UserControls struct {
UserID string `json:"user_id"`
AllowExplicit bool `json:"allow_explicit"`
ExcludedTracks []string `json:"excluded_tracks,omitempty"`
ExcludedArtists []string `json:"excluded_artists,omitempty"`
ExcludedGenres []string `json:"excluded_genres,omitempty"`
PostponedTracks []string `json:"postponed_tracks,omitempty"`
}
type RecommendRequest struct {
UserID string `json:"user_id"`
Limit int `json:"limit"`
SeedTrackIDs []string `json:"seed_track_ids,omitempty"`
FeatureTargets *AudioFeatures `json:"feature_targets,omitempty"`
Context Context `json:"context,omitempty"`
Mode string `json:"mode,omitempty"`
ExplorationTarget float64 `json:"exploration_target,omitempty"`
MinPopularity *float64 `json:"min_popularity,omitempty"`
MaxPopularity *float64 `json:"max_popularity,omitempty"`
IncludeExplicit *bool `json:"include_explicit,omitempty"`
ExcludedTrackIDs []string `json:"excluded_track_ids,omitempty"`
ExcludedArtistIDs []string `json:"excluded_artist_ids,omitempty"`
ExcludedGenres []string `json:"excluded_genres,omitempty"`
}
type Recommendation struct {
Track Track `json:"track"`
Score float64 `json:"score"`
Rank int `json:"rank"`
Reason string `json:"reason"`
ScoreBreakdown ScoreBreakdown `json:"score_breakdown"`
Explanation map[string]float64 `json:"explanation"`
}
type ScoreBreakdown struct {
Content float64 `json:"content"`
Collaborative float64 `json:"collaborative"`
Popularity float64 `json:"popularity"`
Exploration float64 `json:"exploration"`
Diversity float64 `json:"diversity"`
Safety float64 `json:"safety"`
Commercial float64 `json:"commercial"`
Final float64 `json:"final"`
}
type TasteProfile struct {
UserID string `json:"user_id"`
Vector []float64 `json:"vector"`
TopGenres map[string]float64 `json:"top_genres"`
InteractionCount int `json:"interaction_count"`
Confidence float64 `json:"confidence"`
ExplorationReadiness float64 `json:"exploration_readiness"`
UpdatedAt time.Time `json:"updated_at"`
}
type CatalogSnapshot struct {
Tracks []Track
Interactions []Interaction
Controls UserControls
}
apps/backend/internal/recommendation/vector.go
+147
View File
@@ -0,0 +1,147 @@
package recommendation
import "math"
const featureCount = 12
var featureNames = []string{
"danceability",
"energy",
"loudness",
"speechiness",
"acousticness",
"instrumentalness",
"liveness",
"valence",
"tempo",
"time_signature",
"key",
"mode",
}
type featureSpec struct {
min float64
max float64
weight float64
}
var featureSpecs = [featureCount]featureSpec{
{min: 0, max: 1, weight: 1.12}, // danceability
{min: 0, max: 1, weight: 1.18}, // energy
{min: -60, max: 0, weight: 0.78}, // loudness
{min: 0, max: 1, weight: 0.72}, // speechiness
{min: 0, max: 1, weight: 1.02}, // acousticness
{min: 0, max: 1, weight: 0.82}, // instrumentalness
{min: 0, max: 1, weight: 0.44}, // liveness
{min: 0, max: 1, weight: 1.08}, // valence
{min: 40, max: 220, weight: 0.92}, // tempo
{min: 1, max: 7, weight: 0.22}, // time signature
{min: 0, max: 11, weight: 0.20}, // key
{min: 0, max: 1, weight: 0.16}, // mode
}
type featureBounds struct {
min [featureCount]float64
max [featureCount]float64
}
func vector(features AudioFeatures) [featureCount]float64 {
return [featureCount]float64{
features.Danceability,
features.Energy,
features.Loudness,
features.Speechiness,
features.Acousticness,
features.Instrumentalness,
features.Liveness,
features.Valence,
features.Tempo,
features.TimeSignature,
features.Key,
features.Mode,
}
}
func bounds(tracks []Track) featureBounds {
var b featureBounds
for i := range featureCount {
b.min[i] = featureSpecs[i].min
b.max[i] = featureSpecs[i].max
}
for _, track := range tracks {
if !hasAudioFeatures(track.Features) {
continue
}
v := vector(track.Features)
for i, value := range v {
if value < b.min[i] {
b.min[i] = value
}
if value > b.max[i] {
b.max[i] = value
}
}
}
for i := range featureCount {
if math.IsInf(b.min[i], 0) || math.IsInf(b.max[i], 0) || b.min[i] == b.max[i] {
b.min[i] = 0
b.max[i] = 1
}
}
return b
}
func normalize(features AudioFeatures, b featureBounds) [featureCount]float64 {
raw := vector(features)
var out [featureCount]float64
for i, value := range raw {
denominator := b.max[i] - b.min[i]
if denominator == 0 {
out[i] = 0
continue
}
out[i] = clamp01((value - b.min[i]) / denominator)
}
return out
}
func cosine(a, b [featureCount]float64) float64 {
var dot, normA, normB float64
for i := range featureCount {
weight := featureSpecs[i].weight
dot += weight * a[i] * b[i]
normA += weight * a[i] * a[i]
normB += weight * b[i] * b[i]
}
if normA == 0 || normB == 0 {
return 0
}
return dot / (math.Sqrt(normA) * math.Sqrt(normB))
}
func distance(a, b [featureCount]float64) float64 {
return 1 - cosine(a, b)
}
func clamp01(value float64) float64 {
if value < 0 {
return 0
}
if value > 1 {
return 1
}
return value
}
func hasAudioFeatures(features AudioFeatures) bool {
raw := vector(features)
nonZero := 0
for _, value := range raw {
if value != 0 {
nonZero++
}
}
return nonZero >= 4 && features.Tempo > 0 && features.TimeSignature > 0
}
apps/backend/internal/storage/memory/memory.go
+200
View File
@@ -0,0 +1,200 @@
package memory
import (
"context"
"slices"
"sync"
"time"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/recommendation"
)
type Store struct {
mu sync.RWMutex
tracks map[string]recommendation.Track
interactions []recommendation.Interaction
controls map[string]recommendation.UserControls
providerCache map[string]provider.CacheEntry
importJobs map[string]provider.ImportJob
enrichments map[string]provider.TrackEnrichment
}
func New() *Store {
return &Store{
tracks: make(map[string]recommendation.Track),
controls: make(map[string]recommendation.UserControls),
providerCache: make(map[string]provider.CacheEntry),
importJobs: make(map[string]provider.ImportJob),
enrichments: make(map[string]provider.TrackEnrichment),
}
}
func (s *Store) Ping(context.Context) error {
return nil
}
func (s *Store) UpsertTrack(_ context.Context, track recommendation.Track) error {
s.mu.Lock()
defer s.mu.Unlock()
now := time.Now().UTC()
existing := s.tracks[track.ID]
if track.CreatedAt.IsZero() {
track.CreatedAt = existing.CreatedAt
}
if track.CreatedAt.IsZero() {
track.CreatedAt = now
}
track.UpdatedAt = now
s.tracks[track.ID] = track
return nil
}
func (s *Store) UpsertTracks(ctx context.Context, tracks []recommendation.Track) error {
for _, track := range tracks {
if err := s.UpsertTrack(ctx, track); err != nil {
return err
}
}
return nil
}
func (s *Store) GetTracksByIDs(_ context.Context, ids []string) ([]recommendation.Track, error) {
s.mu.RLock()
defer s.mu.RUnlock()
out := make([]recommendation.Track, 0, len(ids))
for _, id := range ids {
if track, ok := s.tracks[id]; ok {
out = append(out, track)
}
}
return out, nil
}
func (s *Store) RecordInteraction(_ context.Context, interaction recommendation.Interaction) error {
s.mu.Lock()
defer s.mu.Unlock()
if interaction.OccurredAt.IsZero() {
interaction.OccurredAt = time.Now().UTC()
}
s.interactions = append(s.interactions, interaction)
return nil
}
func (s *Store) GetControls(_ context.Context, userID string) (recommendation.UserControls, error) {
s.mu.RLock()
defer s.mu.RUnlock()
controls, ok := s.controls[userID]
if !ok {
return recommendation.UserControls{UserID: userID, AllowExplicit: true}, nil
}
return controls, nil
}
func (s *Store) UpsertControls(_ context.Context, controls recommendation.UserControls) error {
s.mu.Lock()
defer s.mu.Unlock()
s.controls[controls.UserID] = controls
return nil
}
func (s *Store) Snapshot(_ context.Context, userID string) (recommendation.CatalogSnapshot, error) {
s.mu.RLock()
defer s.mu.RUnlock()
tracks := make([]recommendation.Track, 0, len(s.tracks))
for _, track := range s.tracks {
tracks = append(tracks, track)
}
slices.SortFunc(tracks, func(a, b recommendation.Track) int {
if a.ID < b.ID {
return -1
}
if a.ID > b.ID {
return 1
}
return 0
})
interactions := slices.Clone(s.interactions)
controls, ok := s.controls[userID]
if !ok {
controls = recommendation.UserControls{UserID: userID, AllowExplicit: true}
}
return recommendation.CatalogSnapshot{
Tracks: tracks,
Interactions: interactions,
Controls: controls,
}, nil
}
func (s *Store) GetProviderCache(_ context.Context, providerName, itemType, itemID, market string) (provider.CacheEntry, bool, error) {
s.mu.RLock()
defer s.mu.RUnlock()
entry, ok := s.providerCache[providerCacheKey(providerName, itemType, itemID, market)]
if !ok {
return provider.CacheEntry{}, false, nil
}
return cloneCacheEntry(entry), true, nil
}
func (s *Store) UpsertProviderCache(_ context.Context, entry provider.CacheEntry) error {
s.mu.Lock()
defer s.mu.Unlock()
s.providerCache[providerCacheKey(entry.Provider, entry.ItemType, entry.ItemID, entry.Market)] = cloneCacheEntry(entry)
return nil
}
func (s *Store) ProviderCacheStats(context.Context) (provider.CacheStats, error) {
s.mu.RLock()
defer s.mu.RUnlock()
now := time.Now().UTC()
stats := provider.CacheStats{Entries: int64(len(s.providerCache))}
for _, entry := range s.providerCache {
if entry.Fresh(now) {
stats.FreshEntries++
} else {
stats.StaleEntries++
}
}
return stats, nil
}
func (s *Store) CreateImportJob(_ context.Context, job provider.ImportJob) error {
s.mu.Lock()
defer s.mu.Unlock()
s.importJobs[job.ID] = job
return nil
}
func (s *Store) FinishImportJob(_ context.Context, job provider.ImportJob) error {
s.mu.Lock()
defer s.mu.Unlock()
s.importJobs[job.ID] = job
return nil
}
func (s *Store) UpsertTrackEnrichment(_ context.Context, enrichment provider.TrackEnrichment) error {
s.mu.Lock()
defer s.mu.Unlock()
s.enrichments[enrichment.TrackID+":"+enrichment.Provider] = cloneEnrichment(enrichment)
return nil
}
func providerCacheKey(providerName, itemType, itemID, market string) string {
return providerName + "\x00" + itemType + "\x00" + itemID + "\x00" + market
}
func cloneCacheEntry(entry provider.CacheEntry) provider.CacheEntry {
if len(entry.Payload) > 0 {
entry.Payload = slices.Clone(entry.Payload)
}
return entry
}
func cloneEnrichment(enrichment provider.TrackEnrichment) provider.TrackEnrichment {
if len(enrichment.Payload) > 0 {
enrichment.Payload = slices.Clone(enrichment.Payload)
}
return enrichment
}
apps/backend/internal/storage/memory/seed.go
+33
View File
@@ -0,0 +1,33 @@
package memory
import (
"context"
"time"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/recommendation"
)
func SeedDemo(store *Store) {
ctx := context.Background()
now := time.Now().UTC()
tracks := []recommendation.Track{
{ID: "trk-neon-dawn", Title: "Neon Dawn", Artist: "The Arrays", Genres: []string{"synthpop", "indie"}, Popularity: 0.72, Features: recommendation.AudioFeatures{Danceability: 0.74, Energy: 0.70, Loudness: -6, Speechiness: 0.05, Acousticness: 0.12, Instrumentalness: 0.15, Liveness: 0.12, Valence: 0.67, Tempo: 118, TimeSignature: 4, Key: 2, Mode: 1}, DiscoveryAllowed: true},
{ID: "trk-static-heart", Title: "Static Heart", Artist: "The Arrays", Genres: []string{"synthpop"}, Popularity: 0.62, Features: recommendation.AudioFeatures{Danceability: 0.68, Energy: 0.66, Loudness: -8, Speechiness: 0.04, Acousticness: 0.18, Instrumentalness: 0.20, Liveness: 0.10, Valence: 0.58, Tempo: 116, TimeSignature: 4, Key: 4, Mode: 1}, DiscoveryAllowed: true},
{ID: "trk-glass-road", Title: "Glass Road", Artist: "North Index", Genres: []string{"indie", "rock"}, Popularity: 0.51, Features: recommendation.AudioFeatures{Danceability: 0.55, Energy: 0.78, Loudness: -5, Speechiness: 0.06, Acousticness: 0.20, Instrumentalness: 0.08, Liveness: 0.18, Valence: 0.54, Tempo: 132, TimeSignature: 4, Key: 9, Mode: 1}, DiscoveryAllowed: true},
{ID: "trk-late-platform", Title: "Late Platform", Artist: "North Index", Genres: []string{"indie", "ambient"}, Popularity: 0.37, Features: recommendation.AudioFeatures{Danceability: 0.44, Energy: 0.38, Loudness: -13, Speechiness: 0.03, Acousticness: 0.66, Instrumentalness: 0.70, Liveness: 0.09, Valence: 0.35, Tempo: 92, TimeSignature: 4, Key: 11, Mode: 0}, DiscoveryAllowed: true},
{ID: "trk-velvet-motor", Title: "Velvet Motor", Artist: "Signal Choir", Genres: []string{"pop-punk", "rock"}, Popularity: 0.49, Features: recommendation.AudioFeatures{Danceability: 0.60, Energy: 0.88, Loudness: -4, Speechiness: 0.07, Acousticness: 0.09, Instrumentalness: 0.02, Liveness: 0.21, Valence: 0.62, Tempo: 148, TimeSignature: 4, Key: 5, Mode: 1}, DiscoveryAllowed: true, CommercialBoost: 0.02},
{ID: "trk-blue-hour", Title: "Blue Hour", Artist: "Mira Vale", Genres: []string{"acoustic", "folk"}, Popularity: 0.43, Features: recommendation.AudioFeatures{Danceability: 0.38, Energy: 0.31, Loudness: -14, Speechiness: 0.04, Acousticness: 0.86, Instrumentalness: 0.12, Liveness: 0.11, Valence: 0.42, Tempo: 82, TimeSignature: 4, Key: 7, Mode: 0}, DiscoveryAllowed: true},
}
_ = store.UpsertTracks(ctx, tracks)
for _, interaction := range []recommendation.Interaction{
{UserID: "demo-user", TrackID: "trk-neon-dawn", Type: recommendation.InteractionLike, OccurredAt: now.Add(-24 * time.Hour)},
{UserID: "demo-user", TrackID: "trk-static-heart", Type: recommendation.InteractionSave, OccurredAt: now.Add(-48 * time.Hour)},
{UserID: "demo-user", TrackID: "trk-blue-hour", Type: recommendation.InteractionSkip, OccurredAt: now.Add(-12 * time.Hour)},
{UserID: "neighbor-a", TrackID: "trk-neon-dawn", Type: recommendation.InteractionLike, OccurredAt: now.Add(-72 * time.Hour)},
{UserID: "neighbor-a", TrackID: "trk-glass-road", Type: recommendation.InteractionLike, OccurredAt: now.Add(-24 * time.Hour)},
{UserID: "neighbor-b", TrackID: "trk-static-heart", Type: recommendation.InteractionLike, OccurredAt: now.Add(-72 * time.Hour)},
{UserID: "neighbor-b", TrackID: "trk-velvet-motor", Type: recommendation.InteractionLike, OccurredAt: now.Add(-18 * time.Hour)},
} {
_ = store.RecordInteraction(ctx, interaction)
}
}
apps/backend/internal/storage/memory/seed_large.go
+81
View File
@@ -0,0 +1,81 @@
package memory
import (
"context"
"time"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/recommendation"
)
// SeedLargeCatalog creates a diverse catalog for realistic recommendations
func SeedLargeCatalog(store *Store) {
ctx := context.Background()
now := time.Now().UTC()
// Diverse catalog with many tracks across genres
tracks := []recommendation.Track{
// Electronic / Dance (similar to Avicii)
{ID: "25FTMokYEbEWHEdss5JLZS", Title: "The Nights", Artist: "Avicii", Genres: []string{"electronic", "dance", "edm"}, Popularity: 0.85, Features: recommendation.AudioFeatures{Danceability: 0.72, Energy: 0.78, Loudness: -5, Speechiness: 0.05, Acousticness: 0.15, Instrumentalness: 0.10, Liveness: 0.20, Valence: 0.70, Tempo: 126, TimeSignature: 4, Key: 4, Mode: 1}, DiscoveryAllowed: true},
{ID: "trk-wake-me-up", Title: "Wake Me Up", Artist: "Avicii", Genres: []string{"electronic", "dance", "country"}, Popularity: 0.90, Features: recommendation.AudioFeatures{Danceability: 0.68, Energy: 0.82, Loudness: -4, Speechiness: 0.06, Acousticness: 0.22, Instrumentalness: 0.05, Liveness: 0.15, Valence: 0.65, Tempo: 124, TimeSignature: 4, Key: 2, Mode: 1}, DiscoveryAllowed: true},
{ID: "trk-levels", Title: "Levels", Artist: "Avicii", Genres: []string{"electronic", "dance", "edm"}, Popularity: 0.88, Features: recommendation.AudioFeatures{Danceability: 0.75, Energy: 0.85, Loudness: -3, Speechiness: 0.04, Acousticness: 0.08, Instrumentalness: 0.20, Liveness: 0.25, Valence: 0.72, Tempo: 128, TimeSignature: 4, Key: 5, Mode: 1}, DiscoveryAllowed: true},
{ID: "trk-hey-brother", Title: "Hey Brother", Artist: "Avicii", Genres: []string{"electronic", "dance", "country"}, Popularity: 0.82, Features: recommendation.AudioFeatures{Danceability: 0.65, Energy: 0.80, Loudness: -5, Speechiness: 0.07, Acousticness: 0.30, Instrumentalness: 0.08, Liveness: 0.18, Valence: 0.60, Tempo: 125, TimeSignature: 4, Key: 7, Mode: 1}, DiscoveryAllowed: true},
{ID: "trk-waiting-for-love", Title: "Waiting For Love", Artist: "Avicii", Genres: []string{"electronic", "dance", "pop"}, Popularity: 0.83, Features: recommendation.AudioFeatures{Danceability: 0.70, Energy: 0.78, Loudness: -5, Speechiness: 0.05, Acousticness: 0.18, Instrumentalness: 0.12, Liveness: 0.20, Valence: 0.68, Tempo: 126, TimeSignature: 4, Key: 9, Mode: 1}, DiscoveryAllowed: true},
// Pop / Dance
{ID: "trk-counting-stars", Title: "Counting Stars", Artist: "OneRepublic", Genres: []string{"pop", "rock"}, Popularity: 0.87, Features: recommendation.AudioFeatures{Danceability: 0.68, Energy: 0.75, Loudness: -6, Speechiness: 0.08, Acousticness: 0.20, Instrumentalness: 0.02, Liveness: 0.15, Valence: 0.55, Tempo: 122, TimeSignature: 4, Key: 2, Mode: 1}, DiscoveryAllowed: true},
{ID: "trk-uptown-funk", Title: "Uptown Funk", Artist: "Bruno Mars", Genres: []string{"pop", "funk"}, Popularity: 0.89, Features: recommendation.AudioFeatures{Danceability: 0.82, Energy: 0.88, Loudness: -4, Speechiness: 0.10, Acousticness: 0.05, Instrumentalness: 0.02, Liveness: 0.30, Valence: 0.90, Tempo: 115, TimeSignature: 4, Key: 0, Mode: 1}, DiscoveryAllowed: true},
{ID: "trk-happy", Title: "Happy", Artist: "Pharrell Williams", Genres: []string{"pop", "soul"}, Popularity: 0.86, Features: recommendation.AudioFeatures{Danceability: 0.78, Energy: 0.82, Loudness: -5, Speechiness: 0.12, Acousticness: 0.10, Instrumentalness: 0.03, Liveness: 0.22, Valence: 0.95, Tempo: 160, TimeSignature: 4, Key: 5, Mode: 1}, DiscoveryAllowed: true},
{ID: "trk-cant-hold-us", Title: "Can't Hold Us", Artist: "Macklemore", Genres: []string{"hip-hop", "pop"}, Popularity: 0.84, Features: recommendation.AudioFeatures{Danceability: 0.72, Energy: 0.86, Loudness: -4, Speechiness: 0.20, Acousticness: 0.15, Instrumentalness: 0.02, Liveness: 0.28, Valence: 0.75, Tempo: 146, TimeSignature: 4, Key: 7, Mode: 1}, DiscoveryAllowed: true},
{ID: "trk-i-gotta-feeling", Title: "I Gotta Feeling", Artist: "Black Eyed Peas", Genres: []string{"pop", "dance"}, Popularity: 0.85, Features: recommendation.AudioFeatures{Danceability: 0.76, Energy: 0.80, Loudness: -5, Speechiness: 0.08, Acousticness: 0.12, Instrumentalness: 0.05, Liveness: 0.25, Valence: 0.80, Tempo: 128, TimeSignature: 4, Key: 2, Mode: 1}, DiscoveryAllowed: true},
// Alternative / Indie
{ID: "trk-do-i-wanna-know", Title: "Do I Wanna Know?", Artist: "Arctic Monkeys", Genres: []string{"alternative", "indie"}, Popularity: 0.80, Features: recommendation.AudioFeatures{Danceability: 0.58, Energy: 0.68, Loudness: -8, Speechiness: 0.05, Acousticness: 0.35, Instrumentalness: 0.25, Liveness: 0.12, Valence: 0.35, Tempo: 85, TimeSignature: 4, Key: 9, Mode: 0}, DiscoveryAllowed: true},
{ID: "trk-somebody-told-me", Title: "Somebody Told Me", Artist: "The Killers", Genres: []string{"alternative", "rock"}, Popularity: 0.82, Features: recommendation.AudioFeatures{Danceability: 0.62, Energy: 0.85, Loudness: -5, Speechiness: 0.06, Acousticness: 0.08, Instrumentalness: 0.02, Liveness: 0.20, Valence: 0.65, Tempo: 138, TimeSignature: 4, Key: 0, Mode: 1}, DiscoveryAllowed: true},
{ID: "trk-mr-brightside", Title: "Mr. Brightside", Artist: "The Killers", Genres: []string{"alternative", "rock"}, Popularity: 0.88, Features: recommendation.AudioFeatures{Danceability: 0.55, Energy: 0.90, Loudness: -4, Speechiness: 0.09, Acousticness: 0.05, Instrumentalness: 0.02, Liveness: 0.25, Valence: 0.60, Tempo: 148, TimeSignature: 4, Key: 2, Mode: 1}, DiscoveryAllowed: true},
{ID: "trk-take-me-out", Title: "Take Me Out", Artist: "Franz Ferdinand", Genres: []string{"alternative", "indie"}, Popularity: 0.78, Features: recommendation.AudioFeatures{Danceability: 0.65, Energy: 0.82, Loudness: -5, Speechiness: 0.04, Acousticness: 0.15, Instrumentalness: 0.05, Liveness: 0.18, Valence: 0.55, Tempo: 105, TimeSignature: 4, Key: 7, Mode: 1}, DiscoveryAllowed: true},
// Rock
{ID: "trk-boulevard-of-broken-dreams", Title: "Boulevard of Broken Dreams", Artist: "Green Day", Genres: []string{"rock", "punk"}, Popularity: 0.86, Features: recommendation.AudioFeatures{Danceability: 0.52, Energy: 0.78, Loudness: -5, Speechiness: 0.04, Acousticness: 0.12, Instrumentalness: 0.05, Liveness: 0.15, Valence: 0.40, Tempo: 167, TimeSignature: 4, Key: 0, Mode: 1}, DiscoveryAllowed: true},
{ID: "trk-knights-of-cydonia", Title: "Knights of Cydonia", Artist: "Muse", Genres: []string{"rock", "alternative"}, Popularity: 0.81, Features: recommendation.AudioFeatures{Danceability: 0.45, Energy: 0.92, Loudness: -3, Speechiness: 0.08, Acousticness: 0.05, Instrumentalness: 0.35, Liveness: 0.22, Valence: 0.45, Tempo: 137, TimeSignature: 4, Key: 2, Mode: 1}, DiscoveryAllowed: true},
{ID: "trk-smells-like-teen-spirit", Title: "Smells Like Teen Spirit", Artist: "Nirvana", Genres: []string{"rock", "grunge"}, Popularity: 0.87, Features: recommendation.AudioFeatures{Danceability: 0.48, Energy: 0.95, Loudness: -2, Speechiness: 0.07, Acousticness: 0.03, Instrumentalness: 0.10, Liveness: 0.30, Valence: 0.35, Tempo: 117, TimeSignature: 4, Key: 4, Mode: 0}, DiscoveryAllowed: true},
// Acoustic / Folk
{ID: "trk-ho-hey", Title: "Ho Hey", Artist: "The Lumineers", Genres: []string{"folk", "acoustic"}, Popularity: 0.79, Features: recommendation.AudioFeatures{Danceability: 0.58, Energy: 0.55, Loudness: -10, Speechiness: 0.06, Acousticness: 0.75, Instrumentalness: 0.05, Liveness: 0.18, Valence: 0.55, Tempo: 80, TimeSignature: 4, Key: 7, Mode: 1}, DiscoveryAllowed: true},
{ID: "trk-riptide", Title: "Riptide", Artist: "Vance Joy", Genres: []string{"folk", "indie"}, Popularity: 0.83, Features: recommendation.AudioFeatures{Danceability: 0.62, Energy: 0.48, Loudness: -11, Speechiness: 0.08, Acousticness: 0.72, Instrumentalness: 0.08, Liveness: 0.12, Valence: 0.60, Tempo: 102, TimeSignature: 4, Key: 9, Mode: 1}, DiscoveryAllowed: true},
{ID: "trk-let-her-go", Title: "Let Her Go", Artist: "Passenger", Genres: []string{"folk", "acoustic"}, Popularity: 0.80, Features: recommendation.AudioFeatures{Danceability: 0.52, Energy: 0.42, Loudness: -12, Speechiness: 0.05, Acousticness: 0.80, Instrumentalness: 0.05, Liveness: 0.15, Valence: 0.35, Tempo: 75, TimeSignature: 4, Key: 4, Mode: 1}, DiscoveryAllowed: true},
// Synth / New Wave
{ID: "trk-neon-dawn", Title: "Neon Dawn", Artist: "The Arrays", Genres: []string{"synthpop", "indie"}, Popularity: 0.72, Features: recommendation.AudioFeatures{Danceability: 0.74, Energy: 0.70, Loudness: -6, Speechiness: 0.05, Acousticness: 0.12, Instrumentalness: 0.15, Liveness: 0.12, Valence: 0.67, Tempo: 118, TimeSignature: 4, Key: 2, Mode: 1}, DiscoveryAllowed: true},
{ID: "trk-static-heart", Title: "Static Heart", Artist: "The Arrays", Genres: []string{"synthpop"}, Popularity: 0.62, Features: recommendation.AudioFeatures{Danceability: 0.68, Energy: 0.66, Loudness: -8, Speechiness: 0.04, Acousticness: 0.18, Instrumentalness: 0.20, Liveness: 0.10, Valence: 0.58, Tempo: 116, TimeSignature: 4, Key: 4, Mode: 1}, DiscoveryAllowed: true},
{ID: "trk-glass-road", Title: "Glass Road", Artist: "North Index", Genres: []string{"indie", "rock"}, Popularity: 0.51, Features: recommendation.AudioFeatures{Danceability: 0.55, Energy: 0.78, Loudness: -5, Speechiness: 0.06, Acousticness: 0.20, Instrumentalness: 0.08, Liveness: 0.18, Valence: 0.54, Tempo: 132, TimeSignature: 4, Key: 9, Mode: 1}, DiscoveryAllowed: true},
{ID: "trk-velvet-motor", Title: "Velvet Motor", Artist: "Signal Choir", Genres: []string{"pop-punk", "rock"}, Popularity: 0.49, Features: recommendation.AudioFeatures{Danceability: 0.60, Energy: 0.88, Loudness: -4, Speechiness: 0.07, Acousticness: 0.09, Instrumentalness: 0.02, Liveness: 0.21, Valence: 0.62, Tempo: 148, TimeSignature: 4, Key: 5, Mode: 1}, DiscoveryAllowed: true, CommercialBoost: 0.02},
}
_ = store.UpsertTracks(ctx, tracks)
// Add collaborative interactions to create taste neighborhoods
interactions := []recommendation.Interaction{
// User who likes electronic/dance
{UserID: "user-electronic", TrackID: "25FTMokYEbEWHEdss5JLZS", Type: recommendation.InteractionLike, OccurredAt: now.Add(-24 * time.Hour)},
{UserID: "user-electronic", TrackID: "trk-wake-me-up", Type: recommendation.InteractionLike, OccurredAt: now.Add(-48 * time.Hour)},
{UserID: "user-electronic", TrackID: "trk-levels", Type: recommendation.InteractionLike, OccurredAt: now.Add(-72 * time.Hour)},
// User who likes pop
{UserID: "user-pop", TrackID: "trk-counting-stars", Type: recommendation.InteractionLike, OccurredAt: now.Add(-24 * time.Hour)},
{UserID: "user-pop", TrackID: "trk-uptown-funk", Type: recommendation.InteractionLike, OccurredAt: now.Add(-48 * time.Hour)},
{UserID: "user-pop", TrackID: "25FTMokYEbEWHEdss5JLZS", Type: recommendation.InteractionLike, OccurredAt: now.Add(-36 * time.Hour)},
// User who likes alternative
{UserID: "user-alt", TrackID: "trk-do-i-wanna-know", Type: recommendation.InteractionLike, OccurredAt: now.Add(-24 * time.Hour)},
{UserID: "user-alt", TrackID: "trk-somebody-told-me", Type: recommendation.InteractionLike, OccurredAt: now.Add(-48 * time.Hour)},
// Cross-genre listener
{UserID: "user-mixed", TrackID: "25FTMokYEbEWHEdss5JLZS", Type: recommendation.InteractionLike, OccurredAt: now.Add(-24 * time.Hour)},
{UserID: "user-mixed", TrackID: "trk-boulevard-of-broken-dreams", Type: recommendation.InteractionLike, OccurredAt: now.Add(-48 * time.Hour)},
{UserID: "user-mixed", TrackID: "trk-riptide", Type: recommendation.InteractionLike, OccurredAt: now.Add(-72 * time.Hour)},
}
for _, interaction := range interactions {
_ = store.RecordInteraction(ctx, interaction)
}
}
apps/backend/internal/storage/postgres/db/catalog.sql.go
+337
View File
@@ -0,0 +1,337 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.29.0
// source: catalog.sql
package db
import (
"context"
"github.com/jackc/pgx/v5/pgtype"
)
const getControls = `-- name: GetControls :one
select user_id, allow_explicit, excluded_tracks, excluded_artists, excluded_genres, postponed_tracks
from user_controls
where user_id = $1
`
type GetControlsRow struct {
UserID string `json:"user_id"`
AllowExplicit bool `json:"allow_explicit"`
ExcludedTracks []byte `json:"excluded_tracks"`
ExcludedArtists []byte `json:"excluded_artists"`
ExcludedGenres []byte `json:"excluded_genres"`
PostponedTracks []byte `json:"postponed_tracks"`
}
func (q *Queries) GetControls(ctx context.Context, userID string) (GetControlsRow, error) {
row := q.db.QueryRow(ctx, getControls, userID)
var i GetControlsRow
err := row.Scan(
&i.UserID,
&i.AllowExplicit,
&i.ExcludedTracks,
&i.ExcludedArtists,
&i.ExcludedGenres,
&i.PostponedTracks,
)
return i, err
}
const getTracksByIDs = `-- name: GetTracksByIDs :many
select id, title, artist, album, genres, release_date, duration_ms, popularity,
explicit, features, external, created_at, updated_at, commercial_boost, quality_penalty, discovery_allowed
from tracks
where id = any($1::text[])
order by id
`
type GetTracksByIDsRow struct {
ID string `json:"id"`
Title string `json:"title"`
Artist string `json:"artist"`
Album string `json:"album"`
Genres []byte `json:"genres"`
ReleaseDate string `json:"release_date"`
DurationMs int32 `json:"duration_ms"`
Popularity float64 `json:"popularity"`
Explicit bool `json:"explicit"`
Features []byte `json:"features"`
External []byte `json:"external"`
CreatedAt pgtype.Timestamptz `json:"created_at"`
UpdatedAt pgtype.Timestamptz `json:"updated_at"`
CommercialBoost float64 `json:"commercial_boost"`
QualityPenalty float64 `json:"quality_penalty"`
DiscoveryAllowed bool `json:"discovery_allowed"`
}
func (q *Queries) GetTracksByIDs(ctx context.Context, dollar_1 []string) ([]GetTracksByIDsRow, error) {
rows, err := q.db.Query(ctx, getTracksByIDs, dollar_1)
if err != nil {
return nil, err
}
defer rows.Close()
var items []GetTracksByIDsRow
for rows.Next() {
var i GetTracksByIDsRow
if err := rows.Scan(
&i.ID,
&i.Title,
&i.Artist,
&i.Album,
&i.Genres,
&i.ReleaseDate,
&i.DurationMs,
&i.Popularity,
&i.Explicit,
&i.Features,
&i.External,
&i.CreatedAt,
&i.UpdatedAt,
&i.CommercialBoost,
&i.QualityPenalty,
&i.DiscoveryAllowed,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listRecentInteractions = `-- name: ListRecentInteractions :many
select user_id, track_id, type, weight, occurred_at, context, completed_ms
from interactions
where occurred_at >= now() - interval '365 days'
order by occurred_at desc
limit 250000
`
type ListRecentInteractionsRow struct {
UserID string `json:"user_id"`
TrackID string `json:"track_id"`
Type string `json:"type"`
Weight float64 `json:"weight"`
OccurredAt pgtype.Timestamptz `json:"occurred_at"`
Context []byte `json:"context"`
CompletedMs int32 `json:"completed_ms"`
}
func (q *Queries) ListRecentInteractions(ctx context.Context) ([]ListRecentInteractionsRow, error) {
rows, err := q.db.Query(ctx, listRecentInteractions)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ListRecentInteractionsRow
for rows.Next() {
var i ListRecentInteractionsRow
if err := rows.Scan(
&i.UserID,
&i.TrackID,
&i.Type,
&i.Weight,
&i.OccurredAt,
&i.Context,
&i.CompletedMs,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listTracks = `-- name: ListTracks :many
select id, title, artist, album, genres, release_date, duration_ms, popularity,
explicit, features, external, created_at, updated_at, commercial_boost, quality_penalty, discovery_allowed
from tracks
order by id
`
type ListTracksRow struct {
ID string `json:"id"`
Title string `json:"title"`
Artist string `json:"artist"`
Album string `json:"album"`
Genres []byte `json:"genres"`
ReleaseDate string `json:"release_date"`
DurationMs int32 `json:"duration_ms"`
Popularity float64 `json:"popularity"`
Explicit bool `json:"explicit"`
Features []byte `json:"features"`
External []byte `json:"external"`
CreatedAt pgtype.Timestamptz `json:"created_at"`
UpdatedAt pgtype.Timestamptz `json:"updated_at"`
CommercialBoost float64 `json:"commercial_boost"`
QualityPenalty float64 `json:"quality_penalty"`
DiscoveryAllowed bool `json:"discovery_allowed"`
}
func (q *Queries) ListTracks(ctx context.Context) ([]ListTracksRow, error) {
rows, err := q.db.Query(ctx, listTracks)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ListTracksRow
for rows.Next() {
var i ListTracksRow
if err := rows.Scan(
&i.ID,
&i.Title,
&i.Artist,
&i.Album,
&i.Genres,
&i.ReleaseDate,
&i.DurationMs,
&i.Popularity,
&i.Explicit,
&i.Features,
&i.External,
&i.CreatedAt,
&i.UpdatedAt,
&i.CommercialBoost,
&i.QualityPenalty,
&i.DiscoveryAllowed,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const recordInteraction = `-- name: RecordInteraction :exec
insert into interactions (user_id, track_id, type, weight, occurred_at, context, completed_ms)
values ($1, $2, $3, $4, $5, $6::jsonb, $7)
`
type RecordInteractionParams struct {
UserID string `json:"user_id"`
TrackID string `json:"track_id"`
Type string `json:"type"`
Weight float64 `json:"weight"`
OccurredAt pgtype.Timestamptz `json:"occurred_at"`
Column6 []byte `json:"column_6"`
CompletedMs int32 `json:"completed_ms"`
}
func (q *Queries) RecordInteraction(ctx context.Context, arg RecordInteractionParams) error {
_, err := q.db.Exec(ctx, recordInteraction,
arg.UserID,
arg.TrackID,
arg.Type,
arg.Weight,
arg.OccurredAt,
arg.Column6,
arg.CompletedMs,
)
return err
}
const upsertControls = `-- name: UpsertControls :exec
insert into user_controls (user_id, allow_explicit, excluded_tracks, excluded_artists, excluded_genres, postponed_tracks)
values ($1, $2, $3::jsonb, $4::jsonb, $5::jsonb, $6::jsonb)
on conflict (user_id) do update set
allow_explicit = excluded.allow_explicit,
excluded_tracks = excluded.excluded_tracks,
excluded_artists = excluded.excluded_artists,
excluded_genres = excluded.excluded_genres,
postponed_tracks = excluded.postponed_tracks,
updated_at = now()
`
type UpsertControlsParams struct {
UserID string `json:"user_id"`
AllowExplicit bool `json:"allow_explicit"`
Column3 []byte `json:"column_3"`
Column4 []byte `json:"column_4"`
Column5 []byte `json:"column_5"`
Column6 []byte `json:"column_6"`
}
func (q *Queries) UpsertControls(ctx context.Context, arg UpsertControlsParams) error {
_, err := q.db.Exec(ctx, upsertControls,
arg.UserID,
arg.AllowExplicit,
arg.Column3,
arg.Column4,
arg.Column5,
arg.Column6,
)
return err
}
const upsertTrack = `-- name: UpsertTrack :exec
insert into tracks (
id, title, artist, album, genres, release_date, duration_ms, popularity,
explicit, features, external, commercial_boost, quality_penalty, discovery_allowed
) values (
$1, $2, $3, $4, $5::jsonb, $6, $7, $8,
$9, $10::jsonb, $11::jsonb, $12, $13, $14
)
on conflict (id) do update set
title = excluded.title,
artist = excluded.artist,
album = excluded.album,
genres = excluded.genres,
release_date = excluded.release_date,
duration_ms = excluded.duration_ms,
popularity = excluded.popularity,
explicit = excluded.explicit,
features = excluded.features,
external = excluded.external,
commercial_boost = excluded.commercial_boost,
quality_penalty = excluded.quality_penalty,
discovery_allowed = excluded.discovery_allowed,
updated_at = now()
`
type UpsertTrackParams struct {
ID string `json:"id"`
Title string `json:"title"`
Artist string `json:"artist"`
Album string `json:"album"`
Column5 []byte `json:"column_5"`
ReleaseDate string `json:"release_date"`
DurationMs int32 `json:"duration_ms"`
Popularity float64 `json:"popularity"`
Explicit bool `json:"explicit"`
Column10 []byte `json:"column_10"`
Column11 []byte `json:"column_11"`
CommercialBoost float64 `json:"commercial_boost"`
QualityPenalty float64 `json:"quality_penalty"`
DiscoveryAllowed bool `json:"discovery_allowed"`
}
func (q *Queries) UpsertTrack(ctx context.Context, arg UpsertTrackParams) error {
_, err := q.db.Exec(ctx, upsertTrack,
arg.ID,
arg.Title,
arg.Artist,
arg.Album,
arg.Column5,
arg.ReleaseDate,
arg.DurationMs,
arg.Popularity,
arg.Explicit,
arg.Column10,
arg.Column11,
arg.CommercialBoost,
arg.QualityPenalty,
arg.DiscoveryAllowed,
)
return err
}
apps/backend/internal/storage/postgres/db/db.go
+32
View File
@@ -0,0 +1,32 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.29.0
package db
import (
"context"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
)
type DBTX interface {
Exec(context.Context, string, ...interface{}) (pgconn.CommandTag, error)
Query(context.Context, string, ...interface{}) (pgx.Rows, error)
QueryRow(context.Context, string, ...interface{}) pgx.Row
}
func New(db DBTX) *Queries {
return &Queries{db: db}
}
type Queries struct {
db DBTX
}
func (q *Queries) WithTx(tx pgx.Tx) *Queries {
return &Queries{
db: tx,
}
}
apps/backend/internal/storage/postgres/db/models.go
+86
View File
@@ -0,0 +1,86 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.29.0
package db
import (
"github.com/jackc/pgx/v5/pgtype"
)
type ImportJob struct {
ID string `json:"id"`
Provider string `json:"provider"`
SourceType string `json:"source_type"`
SourceValue string `json:"source_value"`
Market string `json:"market"`
Status string `json:"status"`
ImportedTracks int32 `json:"imported_tracks"`
UpdatedTracks int32 `json:"updated_tracks"`
Skipped int32 `json:"skipped"`
Warnings []byte `json:"warnings"`
StartedAt pgtype.Timestamptz `json:"started_at"`
FinishedAt pgtype.Timestamptz `json:"finished_at"`
}
type Interaction struct {
ID int64 `json:"id"`
UserID string `json:"user_id"`
TrackID string `json:"track_id"`
Type string `json:"type"`
Weight float64 `json:"weight"`
OccurredAt pgtype.Timestamptz `json:"occurred_at"`
Context []byte `json:"context"`
CompletedMs int32 `json:"completed_ms"`
}
type ProviderCache struct {
Provider string `json:"provider"`
ItemType string `json:"item_type"`
ItemID string `json:"item_id"`
Market string `json:"market"`
Payload []byte `json:"payload"`
FetchedAt pgtype.Timestamptz `json:"fetched_at"`
ExpiresAt pgtype.Timestamptz `json:"expires_at"`
LastError pgtype.Text `json:"last_error"`
}
type Track struct {
ID string `json:"id"`
Title string `json:"title"`
Artist string `json:"artist"`
Album string `json:"album"`
Genres []byte `json:"genres"`
ReleaseDate string `json:"release_date"`
DurationMs int32 `json:"duration_ms"`
Popularity float64 `json:"popularity"`
Explicit bool `json:"explicit"`
Features []byte `json:"features"`
External []byte `json:"external"`
CommercialBoost float64 `json:"commercial_boost"`
QualityPenalty float64 `json:"quality_penalty"`
DiscoveryAllowed bool `json:"discovery_allowed"`
CreatedAt pgtype.Timestamptz `json:"created_at"`
UpdatedAt pgtype.Timestamptz `json:"updated_at"`
}
type TrackEnrichment struct {
TrackID string `json:"track_id"`
Provider string `json:"provider"`
MusicbrainzRecordingID string `json:"musicbrainz_recording_id"`
MusicbrainzArtistID string `json:"musicbrainz_artist_id"`
Isrc string `json:"isrc"`
Payload []byte `json:"payload"`
UpdatedAt pgtype.Timestamptz `json:"updated_at"`
}
type UserControl struct {
UserID string `json:"user_id"`
AllowExplicit bool `json:"allow_explicit"`
ExcludedTracks []byte `json:"excluded_tracks"`
ExcludedArtists []byte `json:"excluded_artists"`
ExcludedGenres []byte `json:"excluded_genres"`
PostponedTracks []byte `json:"postponed_tracks"`
CreatedAt pgtype.Timestamptz `json:"created_at"`
UpdatedAt pgtype.Timestamptz `json:"updated_at"`
}
apps/backend/internal/storage/postgres/db/provider.sql.go
+216
View File
@@ -0,0 +1,216 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.29.0
// source: provider.sql
package db
import (
"context"
"github.com/jackc/pgx/v5/pgtype"
)
const createImportJob = `-- name: CreateImportJob :exec
insert into import_jobs (id, provider, source_type, source_value, market, status, imported_tracks, updated_tracks, skipped, warnings, started_at)
values ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10::jsonb, $11)
`
type CreateImportJobParams struct {
ID string `json:"id"`
Provider string `json:"provider"`
SourceType string `json:"source_type"`
SourceValue string `json:"source_value"`
Market string `json:"market"`
Status string `json:"status"`
ImportedTracks int32 `json:"imported_tracks"`
UpdatedTracks int32 `json:"updated_tracks"`
Skipped int32 `json:"skipped"`
Column10 []byte `json:"column_10"`
StartedAt pgtype.Timestamptz `json:"started_at"`
}
func (q *Queries) CreateImportJob(ctx context.Context, arg CreateImportJobParams) error {
_, err := q.db.Exec(ctx, createImportJob,
arg.ID,
arg.Provider,
arg.SourceType,
arg.SourceValue,
arg.Market,
arg.Status,
arg.ImportedTracks,
arg.UpdatedTracks,
arg.Skipped,
arg.Column10,
arg.StartedAt,
)
return err
}
const finishImportJob = `-- name: FinishImportJob :exec
update import_jobs
set status = $2,
imported_tracks = $3,
updated_tracks = $4,
skipped = $5,
warnings = $6::jsonb,
finished_at = $7
where id = $1
`
type FinishImportJobParams struct {
ID string `json:"id"`
Status string `json:"status"`
ImportedTracks int32 `json:"imported_tracks"`
UpdatedTracks int32 `json:"updated_tracks"`
Skipped int32 `json:"skipped"`
Column6 []byte `json:"column_6"`
FinishedAt pgtype.Timestamptz `json:"finished_at"`
}
func (q *Queries) FinishImportJob(ctx context.Context, arg FinishImportJobParams) error {
_, err := q.db.Exec(ctx, finishImportJob,
arg.ID,
arg.Status,
arg.ImportedTracks,
arg.UpdatedTracks,
arg.Skipped,
arg.Column6,
arg.FinishedAt,
)
return err
}
const getProviderCache = `-- name: GetProviderCache :one
select provider, item_type, item_id, market, payload, fetched_at, expires_at, coalesce(last_error, '') as last_error
from provider_cache
where provider = $1 and item_type = $2 and item_id = $3 and market = $4
`
type GetProviderCacheParams struct {
Provider string `json:"provider"`
ItemType string `json:"item_type"`
ItemID string `json:"item_id"`
Market string `json:"market"`
}
type GetProviderCacheRow struct {
Provider string `json:"provider"`
ItemType string `json:"item_type"`
ItemID string `json:"item_id"`
Market string `json:"market"`
Payload []byte `json:"payload"`
FetchedAt pgtype.Timestamptz `json:"fetched_at"`
ExpiresAt pgtype.Timestamptz `json:"expires_at"`
LastError string `json:"last_error"`
}
func (q *Queries) GetProviderCache(ctx context.Context, arg GetProviderCacheParams) (GetProviderCacheRow, error) {
row := q.db.QueryRow(ctx, getProviderCache,
arg.Provider,
arg.ItemType,
arg.ItemID,
arg.Market,
)
var i GetProviderCacheRow
err := row.Scan(
&i.Provider,
&i.ItemType,
&i.ItemID,
&i.Market,
&i.Payload,
&i.FetchedAt,
&i.ExpiresAt,
&i.LastError,
)
return i, err
}
const providerCacheStats = `-- name: ProviderCacheStats :one
select count(*)::bigint as entries,
count(*) filter (where expires_at > now())::bigint as fresh_entries,
count(*) filter (where expires_at <= now())::bigint as stale_entries
from provider_cache
`
type ProviderCacheStatsRow struct {
Entries int64 `json:"entries"`
FreshEntries int64 `json:"fresh_entries"`
StaleEntries int64 `json:"stale_entries"`
}
func (q *Queries) ProviderCacheStats(ctx context.Context) (ProviderCacheStatsRow, error) {
row := q.db.QueryRow(ctx, providerCacheStats)
var i ProviderCacheStatsRow
err := row.Scan(&i.Entries, &i.FreshEntries, &i.StaleEntries)
return i, err
}
const upsertProviderCache = `-- name: UpsertProviderCache :exec
insert into provider_cache (provider, item_type, item_id, market, payload, fetched_at, expires_at, last_error)
values ($1, $2, $3, $4, $5::jsonb, $6, $7, nullif($8, ''))
on conflict (provider, item_type, item_id, market) do update set
payload = excluded.payload,
fetched_at = excluded.fetched_at,
expires_at = excluded.expires_at,
last_error = excluded.last_error
`
type UpsertProviderCacheParams struct {
Provider string `json:"provider"`
ItemType string `json:"item_type"`
ItemID string `json:"item_id"`
Market string `json:"market"`
Column5 []byte `json:"column_5"`
FetchedAt pgtype.Timestamptz `json:"fetched_at"`
ExpiresAt pgtype.Timestamptz `json:"expires_at"`
Column8 interface{} `json:"column_8"`
}
func (q *Queries) UpsertProviderCache(ctx context.Context, arg UpsertProviderCacheParams) error {
_, err := q.db.Exec(ctx, upsertProviderCache,
arg.Provider,
arg.ItemType,
arg.ItemID,
arg.Market,
arg.Column5,
arg.FetchedAt,
arg.ExpiresAt,
arg.Column8,
)
return err
}
const upsertTrackEnrichment = `-- name: UpsertTrackEnrichment :exec
insert into track_enrichment (track_id, provider, musicbrainz_recording_id, musicbrainz_artist_id, isrc, payload, updated_at)
values ($1, $2, $3, $4, $5, $6::jsonb, $7)
on conflict (track_id, provider) do update set
musicbrainz_recording_id = excluded.musicbrainz_recording_id,
musicbrainz_artist_id = excluded.musicbrainz_artist_id,
isrc = excluded.isrc,
payload = excluded.payload,
updated_at = excluded.updated_at
`
type UpsertTrackEnrichmentParams struct {
TrackID string `json:"track_id"`
Provider string `json:"provider"`
MusicbrainzRecordingID string `json:"musicbrainz_recording_id"`
MusicbrainzArtistID string `json:"musicbrainz_artist_id"`
Isrc string `json:"isrc"`
Column6 []byte `json:"column_6"`
UpdatedAt pgtype.Timestamptz `json:"updated_at"`
}
func (q *Queries) UpsertTrackEnrichment(ctx context.Context, arg UpsertTrackEnrichmentParams) error {
_, err := q.db.Exec(ctx, upsertTrackEnrichment,
arg.TrackID,
arg.Provider,
arg.MusicbrainzRecordingID,
arg.MusicbrainzArtistID,
arg.Isrc,
arg.Column6,
arg.UpdatedAt,
)
return err
}
apps/backend/internal/storage/postgres/store.go
+444
View File
@@ -0,0 +1,444 @@
package postgres
import (
"context"
"encoding/json"
"errors"
"fmt"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgtype"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/provider"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/recommendation"
"github.com/tdvorak/spotifyrecalg/apps/backend/internal/storage/postgres/db"
)
type Store struct {
pool *pgxpool.Pool
queries *db.Queries
}
func New(pool *pgxpool.Pool) *Store {
return &Store{pool: pool, queries: db.New(pool)}
}
func (s *Store) Ping(ctx context.Context) error {
return s.pool.Ping(ctx)
}
func (s *Store) UpsertTrack(ctx context.Context, track recommendation.Track) error {
params, err := upsertTrackParams(track)
if err != nil {
return err
}
return s.queries.UpsertTrack(ctx, params)
}
func (s *Store) UpsertTracks(ctx context.Context, tracks []recommendation.Track) error {
tx, err := s.pool.Begin(ctx)
if err != nil {
return err
}
defer func() { _ = tx.Rollback(ctx) }()
queries := s.queries.WithTx(tx)
for _, track := range tracks {
params, err := upsertTrackParams(track)
if err != nil {
return err
}
if err := queries.UpsertTrack(ctx, params); err != nil {
return err
}
}
return tx.Commit(ctx)
}
func (s *Store) GetTracksByIDs(ctx context.Context, ids []string) ([]recommendation.Track, error) {
if len(ids) == 0 {
return nil, nil
}
rows, err := s.pool.Query(ctx, `
select id, title, artist, album, genres, release_date, duration_ms, popularity,
explicit, features, external, created_at, updated_at, commercial_boost, quality_penalty, discovery_allowed
from tracks
where id = any($1)
order by id`, ids)
if err != nil {
return nil, err
}
defer rows.Close()
tracks := make([]recommendation.Track, 0, len(ids))
for rows.Next() {
track, err := scanTrack(rows)
if err != nil {
return nil, err
}
tracks = append(tracks, track)
}
return tracks, rows.Err()
}
func upsertTrackParams(track recommendation.Track) (db.UpsertTrackParams, error) {
features, err := json.Marshal(track.Features)
if err != nil {
return db.UpsertTrackParams{}, fmt.Errorf("marshal features: %w", err)
}
genres, err := json.Marshal(track.Genres)
if err != nil {
return db.UpsertTrackParams{}, fmt.Errorf("marshal genres: %w", err)
}
external, err := json.Marshal(track.External)
if err != nil {
return db.UpsertTrackParams{}, fmt.Errorf("marshal external ids: %w", err)
}
return db.UpsertTrackParams{
ID: track.ID,
Title: track.Title,
Artist: track.Artist,
Album: track.Album,
Column5: genres,
ReleaseDate: track.ReleaseDate,
DurationMs: int32(track.DurationMS),
Popularity: track.Popularity,
Explicit: track.Explicit,
Column10: features,
Column11: external,
CommercialBoost: track.CommercialBoost,
QualityPenalty: track.QualityPenalty,
DiscoveryAllowed: track.DiscoveryAllowed,
}, nil
}
func (s *Store) RecordInteraction(ctx context.Context, interaction recommendation.Interaction) error {
if interaction.OccurredAt.IsZero() {
interaction.OccurredAt = time.Now().UTC()
}
contextJSON, err := json.Marshal(interaction.Context)
if err != nil {
return fmt.Errorf("marshal interaction context: %w", err)
}
return s.queries.RecordInteraction(ctx, db.RecordInteractionParams{
UserID: interaction.UserID,
TrackID: interaction.TrackID,
Type: string(interaction.Type),
Weight: interaction.Weight,
OccurredAt: pgtype.Timestamptz{Time: interaction.OccurredAt, Valid: true},
Column6: contextJSON,
CompletedMs: int32(interaction.CompletedMS),
})
}
func (s *Store) GetControls(ctx context.Context, userID string) (recommendation.UserControls, error) {
row, err := s.queries.GetControls(ctx, userID)
if errors.Is(err, pgx.ErrNoRows) {
return recommendation.UserControls{UserID: userID, AllowExplicit: true}, nil
}
if err != nil {
return recommendation.UserControls{}, err
}
controls := recommendation.UserControls{UserID: row.UserID, AllowExplicit: row.AllowExplicit}
if err := unmarshalStringSlice(row.ExcludedTracks, &controls.ExcludedTracks); err != nil {
return recommendation.UserControls{}, err
}
if err := unmarshalStringSlice(row.ExcludedArtists, &controls.ExcludedArtists); err != nil {
return recommendation.UserControls{}, err
}
if err := unmarshalStringSlice(row.ExcludedGenres, &controls.ExcludedGenres); err != nil {
return recommendation.UserControls{}, err
}
if err := unmarshalStringSlice(row.PostponedTracks, &controls.PostponedTracks); err != nil {
return recommendation.UserControls{}, err
}
return controls, nil
}
func (s *Store) UpsertControls(ctx context.Context, controls recommendation.UserControls) error {
excludedTracks, err := json.Marshal(controls.ExcludedTracks)
if err != nil {
return err
}
excludedArtists, err := json.Marshal(controls.ExcludedArtists)
if err != nil {
return err
}
excludedGenres, err := json.Marshal(controls.ExcludedGenres)
if err != nil {
return err
}
postponedTracks, err := json.Marshal(controls.PostponedTracks)
if err != nil {
return err
}
return s.queries.UpsertControls(ctx, db.UpsertControlsParams{
UserID: controls.UserID,
AllowExplicit: controls.AllowExplicit,
Column3: excludedTracks,
Column4: excludedArtists,
Column5: excludedGenres,
Column6: postponedTracks,
})
}
func (s *Store) Snapshot(ctx context.Context, userID string) (recommendation.CatalogSnapshot, error) {
tracks, err := s.listTracks(ctx)
if err != nil {
return recommendation.CatalogSnapshot{}, err
}
interactions, err := s.listRecentInteractions(ctx)
if err != nil {
return recommendation.CatalogSnapshot{}, err
}
controls, err := s.GetControls(ctx, userID)
if err != nil {
return recommendation.CatalogSnapshot{}, err
}
return recommendation.CatalogSnapshot{
Tracks: tracks,
Interactions: interactions,
Controls: controls,
}, nil
}
func (s *Store) listTracks(ctx context.Context) ([]recommendation.Track, error) {
rows, err := s.queries.ListTracks(ctx)
if err != nil {
return nil, err
}
tracks := make([]recommendation.Track, 0, len(rows))
for _, row := range rows {
track, err := trackFromListRow(row)
if err != nil {
return nil, err
}
tracks = append(tracks, track)
}
return tracks, nil
}
func trackFromListRow(row db.ListTracksRow) (recommendation.Track, error) {
track := recommendation.Track{
ID: row.ID,
Title: row.Title,
Artist: row.Artist,
Album: row.Album,
ReleaseDate: row.ReleaseDate,
DurationMS: int(row.DurationMs),
Popularity: row.Popularity,
Explicit: row.Explicit,
CreatedAt: row.CreatedAt.Time,
UpdatedAt: row.UpdatedAt.Time,
CommercialBoost: row.CommercialBoost,
QualityPenalty: row.QualityPenalty,
DiscoveryAllowed: row.DiscoveryAllowed,
}
if err := unmarshalStringSlice(row.Genres, &track.Genres); err != nil {
return recommendation.Track{}, err
}
if err := json.Unmarshal(row.Features, &track.Features); err != nil {
return recommendation.Track{}, err
}
if err := unmarshalStringMap(row.External, &track.External); err != nil {
return recommendation.Track{}, err
}
return track, nil
}
type rowScanner interface {
Scan(dest ...any) error
}
func scanTrack(scanner rowScanner) (recommendation.Track, error) {
var (
genres, features, external []byte
createdAt, updatedAt pgtype.Timestamptz
track recommendation.Track
)
if err := scanner.Scan(
&track.ID,
&track.Title,
&track.Artist,
&track.Album,
&genres,
&track.ReleaseDate,
&track.DurationMS,
&track.Popularity,
&track.Explicit,
&features,
&external,
&createdAt,
&updatedAt,
&track.CommercialBoost,
&track.QualityPenalty,
&track.DiscoveryAllowed,
); err != nil {
return recommendation.Track{}, err
}
track.CreatedAt = createdAt.Time
track.UpdatedAt = updatedAt.Time
if err := unmarshalStringSlice(genres, &track.Genres); err != nil {
return recommendation.Track{}, err
}
if err := json.Unmarshal(features, &track.Features); err != nil {
return recommendation.Track{}, err
}
if err := unmarshalStringMap(external, &track.External); err != nil {
return recommendation.Track{}, err
}
return track, nil
}
func (s *Store) GetProviderCache(ctx context.Context, providerName, itemType, itemID, market string) (provider.CacheEntry, bool, error) {
var entry provider.CacheEntry
err := s.pool.QueryRow(ctx, `
select provider, item_type, item_id, market, payload, fetched_at, expires_at, coalesce(last_error, '')
from provider_cache
where provider = $1 and item_type = $2 and item_id = $3 and market = $4`,
providerName, itemType, itemID, market,
).Scan(&entry.Provider, &entry.ItemType, &entry.ItemID, &entry.Market, &entry.Payload, &entry.FetchedAt, &entry.ExpiresAt, &entry.LastError)
if errors.Is(err, pgx.ErrNoRows) {
return provider.CacheEntry{}, false, nil
}
if err != nil {
return provider.CacheEntry{}, false, err
}
return entry, true, nil
}
func (s *Store) UpsertProviderCache(ctx context.Context, entry provider.CacheEntry) error {
_, err := s.pool.Exec(ctx, `
insert into provider_cache (provider, item_type, item_id, market, payload, fetched_at, expires_at, last_error)
values ($1, $2, $3, $4, $5::jsonb, $6, $7, nullif($8, ''))
on conflict (provider, item_type, item_id, market) do update set
payload = excluded.payload,
fetched_at = excluded.fetched_at,
expires_at = excluded.expires_at,
last_error = excluded.last_error`,
entry.Provider,
entry.ItemType,
entry.ItemID,
entry.Market,
emptyObjectIfNil(entry.Payload),
entry.FetchedAt,
entry.ExpiresAt,
entry.LastError,
)
return err
}
func (s *Store) ProviderCacheStats(ctx context.Context) (provider.CacheStats, error) {
var stats provider.CacheStats
err := s.pool.QueryRow(ctx, `
select count(*)::bigint,
count(*) filter (where expires_at > now())::bigint,
count(*) filter (where expires_at <= now())::bigint
from provider_cache`,
).Scan(&stats.Entries, &stats.FreshEntries, &stats.StaleEntries)
return stats, err
}
func (s *Store) CreateImportJob(ctx context.Context, job provider.ImportJob) error {
warnings, err := json.Marshal(job.Warnings)
if err != nil {
return err
}
_, err = s.pool.Exec(ctx, `
insert into import_jobs (id, provider, source_type, source_value, market, status, imported_tracks, updated_tracks, skipped, warnings, started_at)
values ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10::jsonb, $11)`,
job.ID, job.Provider, job.SourceType, job.SourceValue, job.Market, job.Status,
job.ImportedTracks, job.UpdatedTracks, job.Skipped, warnings, job.StartedAt,
)
return err
}
func (s *Store) FinishImportJob(ctx context.Context, job provider.ImportJob) error {
warnings, err := json.Marshal(job.Warnings)
if err != nil {
return err
}
_, err = s.pool.Exec(ctx, `
update import_jobs
set status = $2,
imported_tracks = $3,
updated_tracks = $4,
skipped = $5,
warnings = $6::jsonb,
finished_at = $7
where id = $1`,
job.ID, job.Status, job.ImportedTracks, job.UpdatedTracks, job.Skipped, warnings, job.FinishedAt,
)
return err
}
func (s *Store) UpsertTrackEnrichment(ctx context.Context, enrichment provider.TrackEnrichment) error {
_, err := s.pool.Exec(ctx, `
insert into track_enrichment (track_id, provider, musicbrainz_recording_id, musicbrainz_artist_id, isrc, payload, updated_at)
values ($1, $2, $3, $4, $5, $6::jsonb, $7)
on conflict (track_id, provider) do update set
musicbrainz_recording_id = excluded.musicbrainz_recording_id,
musicbrainz_artist_id = excluded.musicbrainz_artist_id,
isrc = excluded.isrc,
payload = excluded.payload,
updated_at = excluded.updated_at`,
enrichment.TrackID,
enrichment.Provider,
enrichment.MusicBrainzRecordingID,
enrichment.MusicBrainzArtistID,
enrichment.ISRC,
emptyObjectIfNil(enrichment.Payload),
enrichment.UpdatedAt,
)
return err
}
func emptyObjectIfNil(payload []byte) []byte {
if len(payload) == 0 {
return []byte(`{}`)
}
return payload
}
func (s *Store) listRecentInteractions(ctx context.Context) ([]recommendation.Interaction, error) {
rows, err := s.queries.ListRecentInteractions(ctx)
if err != nil {
return nil, err
}
interactions := make([]recommendation.Interaction, 0, len(rows))
for _, row := range rows {
interaction := recommendation.Interaction{
UserID: row.UserID,
TrackID: row.TrackID,
Type: recommendation.InteractionType(row.Type),
Weight: row.Weight,
OccurredAt: row.OccurredAt.Time,
CompletedMS: int(row.CompletedMs),
}
if len(row.Context) > 0 {
if err := json.Unmarshal(row.Context, &interaction.Context); err != nil {
return nil, err
}
}
interactions = append(interactions, interaction)
}
return interactions, nil
}
func unmarshalStringSlice(raw []byte, out *[]string) error {
if len(raw) == 0 {
*out = nil
return nil
}
return json.Unmarshal(raw, out)
}
func unmarshalStringMap(raw []byte, out *map[string]string) error {
if len(raw) == 0 || string(raw) == "null" {
*out = nil
return nil
}
return json.Unmarshal(raw, out)
}
apps/backend/migrations/00001_init.sql
+53
View File
@@ -0,0 +1,53 @@
-- +goose Up
create table if not exists tracks (
id text primary key,
title text not null,
artist text not null,
album text not null default '',
genres jsonb not null default '[]'::jsonb,
release_date text not null default '',
duration_ms integer not null default 0 check (duration_ms >= 0),
popularity double precision not null default 0 check (popularity >= 0 and popularity <= 1),
explicit boolean not null default false,
features jsonb not null,
external jsonb not null default '{}'::jsonb,
commercial_boost double precision not null default 0 check (commercial_boost >= 0 and commercial_boost <= 1),
quality_penalty double precision not null default 0 check (quality_penalty >= 0 and quality_penalty <= 1),
discovery_allowed boolean not null default true,
created_at timestamptz not null default now(),
updated_at timestamptz not null default now()
);
create table if not exists interactions (
id bigserial primary key,
user_id text not null,
track_id text not null references tracks(id) on delete cascade,
type text not null check (type in ('play', 'skip', 'like', 'dislike', 'save', 'hide')),
weight double precision not null default 0,
occurred_at timestamptz not null default now(),
context jsonb not null default '{}'::jsonb,
completed_ms integer not null default 0 check (completed_ms >= 0)
);
create table if not exists user_controls (
user_id text primary key,
allow_explicit boolean not null default true,
excluded_tracks jsonb not null default '[]'::jsonb,
excluded_artists jsonb not null default '[]'::jsonb,
excluded_genres jsonb not null default '[]'::jsonb,
postponed_tracks jsonb not null default '[]'::jsonb,
created_at timestamptz not null default now(),
updated_at timestamptz not null default now()
);
create index if not exists tracks_artist_idx on tracks (artist);
create index if not exists tracks_popularity_idx on tracks (popularity desc);
create index if not exists tracks_genres_gin_idx on tracks using gin (genres);
create index if not exists interactions_user_time_idx on interactions (user_id, occurred_at desc);
create index if not exists interactions_track_time_idx on interactions (track_id, occurred_at desc);
create index if not exists interactions_recent_idx on interactions (occurred_at desc);
-- +goose Down
drop table if exists user_controls;
drop table if exists interactions;
drop table if exists tracks;
apps/backend/migrations/00002_provider_imports.sql
+49
View File
@@ -0,0 +1,49 @@
-- +goose Up
create table if not exists provider_cache (
provider text not null,
item_type text not null,
item_id text not null,
market text not null default '',
payload jsonb not null default '{}'::jsonb,
fetched_at timestamptz not null default now(),
expires_at timestamptz not null,
last_error text,
primary key (provider, item_type, item_id, market)
);
create table if not exists import_jobs (
id text primary key,
provider text not null,
source_type text not null,
source_value text not null,
market text not null default '',
status text not null check (status in ('running', 'succeeded', 'failed')),
imported_tracks integer not null default 0 check (imported_tracks >= 0),
updated_tracks integer not null default 0 check (updated_tracks >= 0),
skipped integer not null default 0 check (skipped >= 0),
warnings jsonb not null default '[]'::jsonb,
started_at timestamptz not null default now(),
finished_at timestamptz
);
create table if not exists track_enrichment (
track_id text not null references tracks(id) on delete cascade,
provider text not null,
musicbrainz_recording_id text not null default '',
musicbrainz_artist_id text not null default '',
isrc text not null default '',
payload jsonb not null default '{}'::jsonb,
updated_at timestamptz not null default now(),
primary key (track_id, provider)
);
create index if not exists provider_cache_expiry_idx on provider_cache (expires_at);
create index if not exists import_jobs_provider_started_idx on import_jobs (provider, started_at desc);
create index if not exists track_enrichment_isrc_idx on track_enrichment (isrc) where isrc <> '';
create index if not exists tracks_external_gin_idx on tracks using gin (external);
-- +goose Down
drop index if exists tracks_external_gin_idx;
drop table if exists track_enrichment;
drop table if exists import_jobs;
drop table if exists provider_cache;
apps/backend/queries/catalog.sql
+63
View File
@@ -0,0 +1,63 @@
-- name: UpsertTrack :exec
insert into tracks (
id, title, artist, album, genres, release_date, duration_ms, popularity,
explicit, features, external, commercial_boost, quality_penalty, discovery_allowed
) values (
$1, $2, $3, $4, $5::jsonb, $6, $7, $8,
$9, $10::jsonb, $11::jsonb, $12, $13, $14
)
on conflict (id) do update set
title = excluded.title,
artist = excluded.artist,
album = excluded.album,
genres = excluded.genres,
release_date = excluded.release_date,
duration_ms = excluded.duration_ms,
popularity = excluded.popularity,
explicit = excluded.explicit,
features = excluded.features,
external = excluded.external,
commercial_boost = excluded.commercial_boost,
quality_penalty = excluded.quality_penalty,
discovery_allowed = excluded.discovery_allowed,
updated_at = now();
-- name: ListTracks :many
select id, title, artist, album, genres, release_date, duration_ms, popularity,
explicit, features, external, created_at, updated_at, commercial_boost, quality_penalty, discovery_allowed
from tracks
order by id;
-- name: GetTracksByIDs :many
select id, title, artist, album, genres, release_date, duration_ms, popularity,
explicit, features, external, created_at, updated_at, commercial_boost, quality_penalty, discovery_allowed
from tracks
where id = any($1::text[])
order by id;
-- name: RecordInteraction :exec
insert into interactions (user_id, track_id, type, weight, occurred_at, context, completed_ms)
values ($1, $2, $3, $4, $5, $6::jsonb, $7);
-- name: ListRecentInteractions :many
select user_id, track_id, type, weight, occurred_at, context, completed_ms
from interactions
where occurred_at >= now() - interval '365 days'
order by occurred_at desc
limit 250000;
-- name: GetControls :one
select user_id, allow_explicit, excluded_tracks, excluded_artists, excluded_genres, postponed_tracks
from user_controls
where user_id = $1;
-- name: UpsertControls :exec
insert into user_controls (user_id, allow_explicit, excluded_tracks, excluded_artists, excluded_genres, postponed_tracks)
values ($1, $2, $3::jsonb, $4::jsonb, $5::jsonb, $6::jsonb)
on conflict (user_id) do update set
allow_explicit = excluded.allow_explicit,
excluded_tracks = excluded.excluded_tracks,
excluded_artists = excluded.excluded_artists,
excluded_genres = excluded.excluded_genres,
postponed_tracks = excluded.postponed_tracks,
updated_at = now();
apps/backend/queries/provider.sql
+43
View File
@@ -0,0 +1,43 @@
-- name: GetProviderCache :one
select provider, item_type, item_id, market, payload, fetched_at, expires_at, coalesce(last_error, '') as last_error
from provider_cache
where provider = $1 and item_type = $2 and item_id = $3 and market = $4;
-- name: UpsertProviderCache :exec
insert into provider_cache (provider, item_type, item_id, market, payload, fetched_at, expires_at, last_error)
values ($1, $2, $3, $4, $5::jsonb, $6, $7, nullif($8, ''))
on conflict (provider, item_type, item_id, market) do update set
payload = excluded.payload,
fetched_at = excluded.fetched_at,
expires_at = excluded.expires_at,
last_error = excluded.last_error;
-- name: ProviderCacheStats :one
select count(*)::bigint as entries,
count(*) filter (where expires_at > now())::bigint as fresh_entries,
count(*) filter (where expires_at <= now())::bigint as stale_entries
from provider_cache;
-- name: CreateImportJob :exec
insert into import_jobs (id, provider, source_type, source_value, market, status, imported_tracks, updated_tracks, skipped, warnings, started_at)
values ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10::jsonb, $11);
-- name: FinishImportJob :exec
update import_jobs
set status = $2,
imported_tracks = $3,
updated_tracks = $4,
skipped = $5,
warnings = $6::jsonb,
finished_at = $7
where id = $1;
-- name: UpsertTrackEnrichment :exec
insert into track_enrichment (track_id, provider, musicbrainz_recording_id, musicbrainz_artist_id, isrc, payload, updated_at)
values ($1, $2, $3, $4, $5, $6::jsonb, $7)
on conflict (track_id, provider) do update set
musicbrainz_recording_id = excluded.musicbrainz_recording_id,
musicbrainz_artist_id = excluded.musicbrainz_artist_id,
isrc = excluded.isrc,
payload = excluded.payload,
updated_at = excluded.updated_at;
apps/backend/sqlc.yaml
+11
View File
@@ -0,0 +1,11 @@
version: "2"
sql:
- engine: "postgresql"
queries: "queries"
schema: "migrations"
gen:
go:
package: "db"
out: "internal/storage/postgres/db"
sql_package: "pgx/v5"
emit_json_tags: true