# 01 — System Architecture ## 1. Context diagram ```mermaid flowchart LR subgraph Clients WEB[Web UI
React SPA] AND[Android App
Kotlin / Media3] SUB[3rd-party Subsonic clients
DSub, Symfonium, ...] CAST[Chromecast device
Custom CAF receiver] end subgraph Server["fablepool-music server (single Go binary)"] API[API Layer
REST + Subsonic] end subgraph Storage["Remote storage (user-owned)"] S3[(S3-compatible
bucket)] DAV[(WebDAV
server)] end DB[(SQLite / PostgreSQL)] CACHEDIR[(Local cache dir
artwork / segments / transcodes)] FF[ffmpeg / ffprobe
subprocesses] WEB -->|HTTPS, native REST| API AND -->|HTTPS, native REST| API SUB -->|HTTPS, Subsonic API| API WEB -.->|Cast sender SDK| CAST AND -.->|Cast sender SDK| CAST CAST -->|HTTPS, scoped stream token| API API --> DB API --> CACHEDIR API --> FF API -->|GET w/ Range, ListObjectsV2| S3 API -->|PROPFIND, GET w/ Range| DAV ``` Key property: **the server is the only component that talks to storage backends**, with one deliberate exception — S3 **presigned URLs** may be handed to clients for raw (non-transcoded) playback when the operator enables it (see §6 and `03-storage-abstraction.md` §4.5). Chromecast devices never authenticate with user credentials; they receive short-lived scoped stream tokens (see `05-auth-and-security.md` §6). ## 2. Internal component diagram ```mermaid flowchart TB subgraph Binary["fablepool-music process"] direction TB HTTP[HTTP server
chi router] subgraph APIs REST[REST handlers /api/v1] SUBH[Subsonic handlers /rest] STREAM[Stream handler
range slicing, token check] end AUTHZ[Auth service
sessions, API keys, Subsonic tokens, stream tokens] LIB[Library service
artists/albums/tracks queries] PLS[Playlist service] HIST[Play-history & scrobble service] RECS[Recommendation engine
feature store + scorer] SCAN[Scanner
list-diff-extract pipeline] TRANS[Transcode manager
ffmpeg job pool] CACHE[Cache manager
tiered, size-bounded] STORE[Storage abstraction
StorageDriver interface] S3D[S3 driver] DAVD[WebDAV driver] JOBS[Job runner
scans, feature extraction, cache GC] EVT[Event bus
in-process pub/sub] end HTTP --> REST & SUBH & STREAM REST & SUBH --> AUTHZ REST & SUBH --> LIB & PLS & HIST & RECS STREAM --> AUTHZ STREAM --> CACHE STREAM --> TRANS CACHE --> STORE TRANS --> CACHE SCAN --> STORE SCAN --> EVT EVT --> RECS JOBS --> SCAN & RECS & CACHE STORE --> S3D & DAVD ``` ### Component responsibilities (normative) | Component | Responsibility | MUST NOT | |---|---|---| | **API layer** | Request parsing, auth dispatch, response shaping (JSON for REST; XML+JSON for Subsonic). | Contain business logic or SQL. | | **Auth service** | Credential verification, session/API-key lifecycle, Subsonic `t`+`s` token validation, stream-token mint/verify. | Log secrets; store plaintext passwords (see 05). | | **Library service** | All read queries over artists/albums/tracks/genres; search; sort/pagination. | Touch storage drivers directly. | | **Scanner** | List Source objects, diff vs DB by `(key, etag_or_mtime, size)`, fetch+parse tags for changed objects, upsert media files, derive albums/artists, emit `track.added/changed/removed` events. | Block API serving (runs on job runner with rate limits). | | **Storage abstraction** | Single `StorageDriver` interface: `List`, `Stat`, `OpenRange`, `Presign (optional)`. | Leak driver-specific types upward. | | **Cache manager** | Tiered cache (see 04): artwork, audio segments, completed transcodes, tag-read buffers. Enforces byte budgets per tier with LRU eviction. | Serve stale entries past validators (ETag/mtime). | | **Transcode manager** | ffmpeg job pool (bounded concurrency), progressive output streaming, job dedup (two listeners of the same transcode share one job). | Spawn unbounded ffmpeg processes. | | **Recommendation engine** | Maintain per-track feature vectors, per-user listening state, score candidates, answer `next-track` queries. | Block playback if scoring fails (fallback rules in 07 §8). | | **Job runner** | Cron-like + on-demand background jobs with persistence (job rows in DB), single-flight per job key. | Run two scans of the same library concurrently. | | **Event bus** | In-process pub/sub decoupling scanner → recommendations, history → recommendations, scan → websocket progress. | Persist events (DB rows are the durable record). | ## 3. Request flows ### 3.1 Stream a track (cache miss, transcoded) ```mermaid sequenceDiagram participant C as Client participant API as Stream handler participant A as Auth participant CM as Cache manager participant TM as Transcode mgr participant SD as Storage driver participant B as S3 / WebDAV C->>API: GET /api/v1/stream/{trackId}?format=opus&maxBitRate=128 (Range: bytes=0-) API->>A: validate token / session A-->>API: user, player profile API->>CM: lookup transcode(trackId, opus, 128, etag) CM-->>API: MISS API->>TM: acquire job (trackId, profile) [single-flight] TM->>CM: lookup source segments(trackId) CM-->>TM: MISS TM->>SD: OpenRange(key, 0..) SD->>B: GET (Range: bytes=0-) / ranged read B-->>SD: 206 stream SD-->>TM: io.Reader TM->>TM: ffmpeg -i pipe:0 ... -f opus pipe:1 TM-->>API: progressive reader (tee to cache) API-->>C: 200, Transfer-Encoding: chunked, audio/ogg Note over TM,CM: On EOF, transcode result is sealed into cache
with key (trackId, etag, codec, bitrate) ``` Notes (normative): - Transcoded responses are served **chunked without Content-Length** (length unknowable up front). Seeking within a transcode is handled by time-offset re-transcode (`timeOffset` param), per `04` §5.4 — never by byte-Range against an in-flight transcode. - Raw (passthrough) responses MUST honour `Range` and reply `206` with `Content-Range` and `Accept-Ranges: bytes`, sliced from cache segments or a backend ranged GET. ### 3.2 Scan flow (S3 example) ```mermaid sequenceDiagram participant J as Job runner participant SC as Scanner participant SD as S3 driver participant S3 as S3 bucket participant DB as Database participant EV as Event bus J->>SC: StartScan(libraryId, mode=incremental) loop ListObjectsV2 pages (max 1000 keys) SC->>SD: List(prefix, continuationToken) SD->>S3: ListObjectsV2 S3-->>SD: page {key, size, etag, lastModified} SD-->>SC: []ObjectInfo SC->>DB: diff page vs media_files (key, etag, size) end loop changed/new objects (bounded worker pool, default 8) SC->>SD: OpenRange(key, tag-read plan) %% header+footer ranges, see 03 §6 SC->>SC: parse tags (dhowden/tag, ffprobe fallback) SC->>DB: upsert media_file, track, album, artist SC->>EV: publish track.added / track.changed end SC->>DB: mark vanished keys missing=true (grace), delete after 2 full scans SC->>DB: finalize scan row (counts, duration, errors) ``` ### 3.3 Radio mode (auto play next) ```mermaid sequenceDiagram participant C as Client (web/Android/Cast queue) participant API as REST API participant R as Recommendation engine participant DB as Database C->>API: GET /api/v1/radio/next?seedTrackId=...&window=10 API->>DB: load last-N plays for user (N = min(request, user setting)) API->>R: NextTrack(user, seed, lastN, exclusions) R->>R: candidate generation (ANN over feature vectors + heuristics) R->>R: score + diversity/de-dup penalties R-->>API: ranked tracks (top-k) API-->>C: 200 {track, alternatives[], explain} ``` ## 4. Process & deployment model One Go binary, three supported topologies: 1. **All-in-one (default):** binary + SQLite + cache dir on one host; ffmpeg on PATH. Docker image ships ffmpeg. 2. **Postgres-backed:** same binary, `DATABASE_URL` pointing at Postgres. Required above ~200k tracks or >10 concurrent users (guideline). 3. **Split cache volume:** cache dir on separate (fast/ephemeral) volume; cache is always reconstructible, never authoritative. ```mermaid flowchart LR subgraph Host BIN[fablepool-music] VOLC[(cache volume)] VOLD[(data volume
sqlite db)] FF[ffmpeg] BIN --- VOLC & VOLD & FF end RP[Reverse proxy
Caddy / nginx / Traefik
TLS termination] --> BIN BIN --> S3[(S3)] BIN --> DAV[(WebDAV)] ``` Operational requirements (normative): - The server MUST run correctly behind a path-prefix reverse proxy (configurable base URL) — required for Subsonic clients and Cast receiver URLs. - TLS is the reverse proxy's job by default; the binary MAY terminate TLS itself (`--tls-cert/--tls-key`) for proxy-less installs. **Chromecast requires HTTPS with a publicly trusted certificate** for stream URLs — documented loudly in 06 §7. - Graceful shutdown: stop accepting requests, let active streams drain up to 30 s, SIGTERM ffmpeg children, checkpoint SQLite WAL. ## 5. Concurrency & resource governance | Resource | Bound | Default | |---|---|---| | Concurrent ffmpeg transcodes | semaphore | `min(4, NumCPU)` | | Scanner tag-extraction workers | per-scan pool | 8 (S3), 4 (WebDAV) | | Backend connections | per-source `http.Transport` limits | 16 max conns/host | | Cache size | per-tier byte budgets | see 04 §3 | | Scan ↔ stream contention | scanner backend reads deprioritised: scanner pauses listing when active stream count > threshold | threshold 8 | ## 6. Trust boundaries & data-flow security summary (Details in `05-auth-and-security.md`.) - **Clients → server:** session cookie (web), bearer API key (REST/Android), Subsonic `u/t/s` (legacy clients), or scoped stream token (query param, for Cast and `