Architecture: audiobookshelf (`/abs`) provider design
Design doc for a new `absdy` provider mounted at `/abs` that browses, searches, and plays audiobooks from a self-hosted audiobookshelf server. Shaped on the fyyd provider: an `Abs` reqwest seam faked in tests, an in-memory per-library search-term store, and a `library -> book -> tracks` tree with a per-library `search` subtree. Grounded live against the test server: bearer auth for browse, `?token=` query auth plus HTTP range (206) on the file endpoint, so a track's stream URL is fully derivable from its path with no extra call. The embedded token and the api_key are secrets, redacted from logs/Debug (hard rule). Also gitignores the abs-api-key file so the JWT never lands in a commit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Test User
parent
4e69261e02
commit
11b2a1bd38
|
|
@ -26,3 +26,9 @@ opencode.json
|
||||||
.opencode/
|
.opencode/
|
||||||
.explained/
|
.explained/
|
||||||
*.kickstart-new
|
*.kickstart-new
|
||||||
|
|
||||||
|
# mdbook build output
|
||||||
|
docs/book/
|
||||||
|
|
||||||
|
# audiobookshelf API key (secret, do not commit)
|
||||||
|
abs-api-key
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,242 @@
|
||||||
|
# audiobookshelf provider (audiobooks)
|
||||||
|
|
||||||
|
## Context and problem statement
|
||||||
|
|
||||||
|
A new library provider mounted at `/abs` that lets a user **browse, search,
|
||||||
|
and play** the audiobooks on a self-hosted
|
||||||
|
[audiobookshelf](https://www.audiobookshelf.org/) (ABS) server.
|
||||||
|
|
||||||
|
- Unlike fyyd/tidal/youtube, ABS is **the user's own private server**, reached
|
||||||
|
over an authenticated HTTP API with an **API key** (a bearer token). So —
|
||||||
|
like tidal — the provider needs credentials, and — unlike tidal — a missing
|
||||||
|
or incomplete config disables it non-fatally (it is one optional server, not
|
||||||
|
the whole app).
|
||||||
|
- ABS organizes content as **libraries → items → audio files**. An audiobook
|
||||||
|
*item* is usually split into many audio files (e.g. `Neuromancer-01.opus` …
|
||||||
|
`-30.opus`); each file is one **track**. So the tree carries the same
|
||||||
|
"one extra level" as fyyd — `library → book → tracks` — plus a per-library
|
||||||
|
**search** subtree that mirrors tidal/youtube search-term semantics.
|
||||||
|
- **Playing** a track means streaming the file's content endpoint. ABS accepts
|
||||||
|
the API key as a `?token=` query parameter on that endpoint and honors HTTP
|
||||||
|
range requests, so the URL is exactly the shape the audio player already
|
||||||
|
handles — **and the whole URL is derivable from the path**, so playback
|
||||||
|
needs no extra API call, no sidecar, no byte-proxying (contrast `/youtube`).
|
||||||
|
- **Captures** (`W`, download) come for free: any node that serves tracks and
|
||||||
|
raises `is_downloadable` gets `w`/`W` with no wire or TUI work.
|
||||||
|
|
||||||
|
## The audiobookshelf API (grounding — live-validated against the test server)
|
||||||
|
|
||||||
|
Base `https://<host>/api/`. Every call carries `Authorization: Bearer <key>`,
|
||||||
|
**except** the file endpoint which also accepts `?token=<key>`. The endpoints
|
||||||
|
we use (all verified against the provided test server):
|
||||||
|
|
||||||
|
| Purpose | Endpoint |
|
||||||
|
| --- | --- |
|
||||||
|
| List libraries | `GET /api/libraries` |
|
||||||
|
| List a library's items | `GET /api/libraries/<lib>/items?limit=<n>&sort=…` |
|
||||||
|
| Search within a library | `GET /api/libraries/<lib>/search?q=<t>&limit=<n>` |
|
||||||
|
| Item detail (tracks) | `GET /api/items/<item>?expanded=1` |
|
||||||
|
| **Stream a file** | `GET /api/items/<item>/file/<ino>?token=<key>` |
|
||||||
|
|
||||||
|
Objects (fields we read):
|
||||||
|
|
||||||
|
- **library**: `id`, `name`, `mediaType` (we handle `book`).
|
||||||
|
- **item summary** (in the items list): `id`, `mediaType`, `media.metadata`
|
||||||
|
(`title`, `authorName`), `media.numAudioFiles`, `media.duration`.
|
||||||
|
- **item detail**: `media.metadata.{title,authorName}` and `media.tracks[]`,
|
||||||
|
each track: `index`, `title` (the filename, e.g. `Neuromancer-01.opus`),
|
||||||
|
`duration` (seconds, float), **`ino`** (stable file id, an integer string),
|
||||||
|
`contentUrl` (`/api/items/<item>/file/<ino>` — same components as the path).
|
||||||
|
- **search** response: `{ "book": [ { "libraryItem": <item> }, … ], … }`.
|
||||||
|
|
||||||
|
Verified facts that shape the design: the file endpoint returns `200` with
|
||||||
|
`?token=` and `401` without; a `Range` request returns `206`; ebook-only items
|
||||||
|
report `numAudioFiles == 0`.
|
||||||
|
|
||||||
|
## Assumptions (decided here)
|
||||||
|
|
||||||
|
- The captures/creatable/editable/deletable TUI flows are provider-agnostic
|
||||||
|
(confirmed by `/youtube` and `/fyyd`): mirroring tidal's search-term
|
||||||
|
semantics costs no TUI or wire change. No proto change, no new
|
||||||
|
`ProviderCommand`.
|
||||||
|
- ABS needs credentials, so — unlike fyyd — a usable `abs.toml` must carry a
|
||||||
|
`base_url` and an `api_key`. A missing file, or one lacking either field,
|
||||||
|
**disables the provider non-fatally** (like fyyd/youtube on a failed probe):
|
||||||
|
it only costs the `/abs` subtree, never startup.
|
||||||
|
- The audio player streams the `?token=` file URL directly (its windowed-HTTP
|
||||||
|
path): ABS serves the raw file with range support, so there is no
|
||||||
|
`/youtube`-style URL-lifetime or ~1 MiB-cap problem. Audiobook files are
|
||||||
|
ordinary media (opus/mp3/m4a/flac) the existing decoder already handles.
|
||||||
|
- Items and files are addressed by their **ABS ids**: the library id and item
|
||||||
|
id are UUIDs, and the file `ino` is an integer string — all already
|
||||||
|
URL-safe. Only user-typed **search terms** are percent-encoded.
|
||||||
|
- The stream URL embeds a secret (`?token=`), so it must **never** be logged;
|
||||||
|
and `api_key` must be redacted from `Debug`/config dumps (hard rule:
|
||||||
|
redact secrets from logs and error reports).
|
||||||
|
|
||||||
|
## Decisions
|
||||||
|
|
||||||
|
### D1 — Crate `absdy`, mounted at `/abs`, non-fatal init
|
||||||
|
|
||||||
|
New workspace crate `absdy` implementing `ProviderClient`, shaped on `fyyd`
|
||||||
|
(the closest analog: remote, search-driven, one extra container level, plain
|
||||||
|
streamable URLs). Wired into `ProviderOrchestrator` with an
|
||||||
|
`abs_client: Option<Arc<absdy::Client>>` field, `abs_owns()`/`abs_provider()`
|
||||||
|
helpers, a `build()` block that reads `abs.toml` (non-fatal), a `get_lib_root`
|
||||||
|
child gated on `self.abs_client.is_some()`, and one routing arm in each
|
||||||
|
dispatch method. `crabidy-server`'s settings gain `abs` in `ALL_PROVIDERS` (now
|
||||||
|
7), in `ProviderToggles`, in `all()`, and in `provider_toggles()`.
|
||||||
|
|
||||||
|
### D2 — HTTP behind a trait, faked in tests
|
||||||
|
|
||||||
|
All network access goes through one seam — an `Abs` trait
|
||||||
|
(`libraries`, `library_items`, `search_items`, `item_detail`) behind
|
||||||
|
`Box<dyn Abs>` — with a `reqwest`-based `AbsApi` (bearer auth) for production
|
||||||
|
and a `FakeApi` in tests, exactly as `fyyd` hides `reqwest` behind `Fyyd`.
|
||||||
|
Provider logic (tree shaping, path parsing, term store, stream-URL building) is
|
||||||
|
then unit-tested with zero network. The trait's error type maps to
|
||||||
|
`ProviderError::FetchError` at the boundary; malformed paths are
|
||||||
|
`MalformedPath`; empty create/rename input is `InvalidInput`; missing
|
||||||
|
credentials at init are `Config`.
|
||||||
|
|
||||||
|
### D3 — Tree shape (libraries, books, tracks, per-library search)
|
||||||
|
|
||||||
|
Item ids are UUIDs and file inos are integers, so neither can equal the
|
||||||
|
reserved segment `search`; the parser uses that to split the two branches.
|
||||||
|
|
||||||
|
- `/abs` — children: one node per library from `/api/libraries` (a fixed
|
||||||
|
browse; the provider is useful with zero typing). Not itself queueable.
|
||||||
|
- `/abs/<lib>` — children: a reserved `search` child (`is_creatable`) **plus**
|
||||||
|
the library's items (bounded, see D5) as book children. A book child's
|
||||||
|
`is_queable` is set from `numAudioFiles > 0`, so ebook-only items show but
|
||||||
|
are not queueable/capturable.
|
||||||
|
- `/abs/<lib>/<item>` — lists that book's audio files as **tracks**;
|
||||||
|
queueable and downloadable (homogeneous tracks).
|
||||||
|
- `/abs/<lib>/<item>/<ino>` — the track leaf.
|
||||||
|
- `/abs/<lib>/search` — `is_creatable`; children are the in-memory search
|
||||||
|
terms (`RwLock<Vec<String>>`, dedup, recreated implicitly on stale paths),
|
||||||
|
each `is_editable` + `is_deletable`, like tidal/youtube/fyyd. Search terms
|
||||||
|
are stored **per library** (keyed by library id).
|
||||||
|
- `/abs/<lib>/search/<term>` — lists matching **books** as children (same book
|
||||||
|
shape as a direct library child).
|
||||||
|
- `/abs/<lib>/search/<term>/<item>` and `.../<item>/<ino>` — identical book
|
||||||
|
node and track leaf as under the direct browse; the two branches share the
|
||||||
|
book-node and track-leaf builders.
|
||||||
|
|
||||||
|
Terms are percent-encoded into one segment (`encode_segment`/
|
||||||
|
`decode_segment`); library ids, item ids, and inos are already URL-safe.
|
||||||
|
|
||||||
|
### D4 — Streams, metadata, no extra call for playback
|
||||||
|
|
||||||
|
- `get_urls_for_track`: parse `item` + `ino` from the path and build
|
||||||
|
`<base>/api/items/<item>/file/<ino>?token=<key>`. **No API call** — the URL
|
||||||
|
is fully derivable from the path; the token is appended last and never
|
||||||
|
logged. An unparseable path is `MalformedPath`.
|
||||||
|
- Track fields (from the item detail's `tracks[]`): `title` = track title (the
|
||||||
|
file name), `artist` = the book's `authorName`, `album` = the book title,
|
||||||
|
`duration` = track duration (seconds → `Option<u32>`), `provider_item_id` =
|
||||||
|
`"<item>:<ino>"` (stable per file, keys the content store for captures).
|
||||||
|
- **Metadata source.** Listing a book fetches the item detail once and builds
|
||||||
|
every track from `tracks[]` (title/duration/author all present). A
|
||||||
|
*directly* fetched track (`get_metadata_for_track` on a bare track path)
|
||||||
|
fetches the same item detail and picks the matching `ino`. A missing field
|
||||||
|
degrades to an empty string / `None`, never an error.
|
||||||
|
|
||||||
|
### D5 — Bounds and freshness
|
||||||
|
|
||||||
|
- `items_per_library` (default 200), `search_results` (default 50) cap every
|
||||||
|
listing so a huge library cannot stall the tree or queue resolution;
|
||||||
|
`call_timeout_secs` (default 30) bounds each HTTP call (hard rule: timeouts
|
||||||
|
on external calls).
|
||||||
|
- Listings are fetched fresh per call (no cross-call cache), like the other
|
||||||
|
remote providers. Only the search *terms* are stored, in memory, per library.
|
||||||
|
|
||||||
|
### D6 — Out of scope (explicitly)
|
||||||
|
|
||||||
|
- **Podcast** libraries (`mediaType: "podcast"`, `media.episodes`): the test
|
||||||
|
server has none; the book-node builder reads `media.tracks`. A podcast
|
||||||
|
library's items would show no tracks (non-queueable). Adding an episodes
|
||||||
|
branch is a later, additive change — noted as the extension point.
|
||||||
|
- ABS user accounts beyond the single API key, playback-progress sync back to
|
||||||
|
ABS, series/authors/collections/genre browse, and tag filtering.
|
||||||
|
- Transcoding / HLS sessions — we stream the raw file with range requests.
|
||||||
|
- Cover art, chapters, and ebook reading.
|
||||||
|
- Pagination past the configured caps (one page is fetched per listing).
|
||||||
|
|
||||||
|
## Structure
|
||||||
|
|
||||||
|
```d2
|
||||||
|
direction: right
|
||||||
|
|
||||||
|
server: crabidy-server {
|
||||||
|
orch: ProviderOrchestrator
|
||||||
|
}
|
||||||
|
|
||||||
|
absdy: "absdy (crate)" {
|
||||||
|
client: "Client\n(ProviderClient)"
|
||||||
|
terms: "search terms\n(in-memory, per library)"
|
||||||
|
api: "AbsApi\n(reqwest seam: Abs trait,\nbearer auth)"
|
||||||
|
client -> terms
|
||||||
|
client -> api
|
||||||
|
}
|
||||||
|
|
||||||
|
abs: "audiobookshelf\n(/api, bearer auth)" { shape: cloud }
|
||||||
|
player: audio-player { shape: hexagon }
|
||||||
|
|
||||||
|
server.orch -> absdy.client: "/abs/..."
|
||||||
|
absdy.api -> abs: "libraries / items / search / detail (JSON, timeout)"
|
||||||
|
server.orch -> player: "file URL with ?token="
|
||||||
|
player -> abs: "windowed HTTP stream (Range → 206)"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Key flow: browse a library, play a track
|
||||||
|
|
||||||
|
```d2
|
||||||
|
shape: sequence_diagram
|
||||||
|
tui: TUI
|
||||||
|
orch: Orchestrator
|
||||||
|
a: absdy
|
||||||
|
api: audiobookshelf
|
||||||
|
|
||||||
|
tui -> orch: "open /abs"
|
||||||
|
orch -> a: "get_lib_root / get_lib_node"
|
||||||
|
a -> api: "GET /api/libraries"
|
||||||
|
api -> a: "libraries (id, name)"
|
||||||
|
a -> tui: "libraries as children"
|
||||||
|
tui -> orch: "open a library"
|
||||||
|
orch -> a: "get_lib_node(/abs/<lib>)"
|
||||||
|
a -> api: "GET /api/libraries/<lib>/items"
|
||||||
|
api -> a: "items (id, title, numAudioFiles)"
|
||||||
|
a -> tui: "[search] + books as children"
|
||||||
|
tui -> orch: "open a book"
|
||||||
|
orch -> a: "get_lib_node(/abs/<lib>/<item>)"
|
||||||
|
a -> api: "GET /api/items/<item>?expanded=1"
|
||||||
|
api -> a: "tracks (ino, title, duration)"
|
||||||
|
a -> tui: "book node: files as tracks (queueable)"
|
||||||
|
tui -> orch: "queue + play a track"
|
||||||
|
orch -> a: "get_urls_for_track(.../<ino>)"
|
||||||
|
a -> a: "build /api/items/<item>/file/<ino>?token= (no call)"
|
||||||
|
orch -> orch: "player streams the file"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Risks and open questions
|
||||||
|
|
||||||
|
- **API-key lifetime.** ABS API keys are long-lived tokens, but if the
|
||||||
|
configured value is a short-lived session JWT it will eventually expire; a
|
||||||
|
`401` then surfaces as a typed `FetchError` (a skipped track / an unreadable
|
||||||
|
node), never a crash. Re-issue the key in `abs.toml` to recover.
|
||||||
|
- **Token leakage.** The stream URL embeds the key. It must never reach a log,
|
||||||
|
trace, or error report — enforced by building the URL only at the boundary,
|
||||||
|
a redacting `Debug`, and logging paths/context (never the built URL). This
|
||||||
|
is a quality gate.
|
||||||
|
- **Summary vs detail drift.** A book child's `is_queable` comes from the
|
||||||
|
summary's `numAudioFiles`; the actual track count comes from the detail.
|
||||||
|
A mismatch only means an optimistic flag — resolution of an empty book
|
||||||
|
yields no tracks (skipped), never an error.
|
||||||
|
- **Large libraries.** Capped by `items_per_library`; deep browsing past the
|
||||||
|
cap needs pagination (out of scope). The cap is `log`-ged so truncation is
|
||||||
|
visible, not silent.
|
||||||
|
- **Field / envelope drift across ABS versions.** DTOs decode defensively
|
||||||
|
(`#[serde(default)]`, missing fields degrade); a renamed field is a local
|
||||||
|
fix in `AbsApi`. **Live validation is a task-plan gate** (already exercised
|
||||||
|
during design against the test server).
|
||||||
Loading…
Reference in New Issue