crabidy/architecture/captures.md

# Captures (downloaded subtrees)

> **Superseded** by `crabidy-store.md`: captures now link into a shared
> content-addressed store under `~/.local/share/crabidy/`, de-duplicated by
> provider id and by content hash, instead of downloading audio next to each
> toml; `/captures` folds into `/crabidy`. Before that, download captures were
> already **partially superseded** by `incremental-captures.md`: they became
> incremental (re-capturing a name resumes; no tmp-and-swap), uncapturable
> tracks are recorded as *skipped* tomls instead of omitted, and the capture
> RPC streams progress. Bookmarks kept the tmp-and-swap described here.

## Context and problem statement

Bookmarks (`w`) mirror a library subtree as **link** files — replaying a
bookmark still needs the original provider. The user wants `W` on a library
node to do the same capture into a separate local provider called
**captures**, except each track's audio is **downloaded** next to its
`.cbd-track.toml`, and the toml points at that file. Playback of a capture
then needs no provider round trip at all — it is a fully local copy.

A library node decides whether it allows `W`; Tidal implements it.

## Assumptions (confirmed against the code)

- `PlayableSpec.file` already supports **relative** paths, resolved against
  the track file's directory at `get_urls_for_track` time (fs-provider D3).
  A toml next to its audio file can say `file = "0001 Song.flac"` and the
  whole capture folder stays relocatable (tmp-and-swap, rename, backup).
- `fsdy::list_dir` only surfaces directories and `*.cbd-track.toml` files;
  downloaded audio siblings are invisible to the library listing.
- The bookmark walk (`bookmark_store::write_capture`) is an iterative
  pre-order worklist whose only per-track action is "serialize and write
  one file" — exactly the seam where a download variant plugs in.
- `reqwest` (rustls, `stream`) is already a workspace dependency; tidal
  stream URLs come from `get_urls_for_track` on the orchestrator, so the
  download needs no new provider methods.
- Uppercase bindings (`K`, `J`) already exist in the TUI bindings table;
  `W` in `Scope::Library` is free.
- `LibraryNode`/`LibraryNodeChild` already model per-node capabilities
  (`is_queable`, `is_creatable`, …) — the "does this node allow `W`"
  decision extends that pattern.

## Decisions

### D1 — Fourth `fsdy` instance at `/captures`

`<config>/crabidy/captures/` is mounted read-only as `/captures` with an
editable top level (no reserved names), exactly like `/bookmarks`. Init is
non-fatal: an unopenable store disables `W` and the mount, never the
server. Loading a capture is browsing `/captures` and queueing a folder —
zero new replay mechanisms.

### D2 — One shared walk, two track sinks

Options considered:

- *(a)* Copy `bookmark_store.rs` and swap the per-track write.
- *(b)* Extract the walk into a shared `capture` module parameterized by a
  **track sink**; bookmarks and captures become thin stores over it.

**Decision: (b)** — the walk (worklist, caps, temp-and-swap, all-or-nothing
cleanup, `BadSource` mapping) is behavior we already tested once and must
not fork. `crabidy-server/src/capture.rs` owns `CaptureError`, the caps,
and `write_tree(client, source, tmp, caps, sink)`; the sink is an enum
(no async-trait indirection):

- `Sink::Link` — today's bookmark behavior, byte-identical
  (`TrackFile::from_track`, link playable).
- `Sink::Download(Downloader)` — captures (D3).

`BookmarkStore` keeps its API; `CaptureStore` (in `capture_store.rs`) is
its sibling over the captures directory.

### D3 — Download sink: audio next to the toml, toml points at it

Per track, in listing order:

1. `get_urls_for_track` through the orchestrator (any provider that yields
   URLs works; Tidal is the target). First URL wins.
2. HTTP GET via one shared `reqwest` client — connect timeout, one total
   per-track deadline covering the whole body, **no retries** (a capture
   is re-runnable and overwrite = refresh; a retry policy can come later).
   The body is streamed to `NNNN <title>.<ext>` (shared `ordered_name`
   sanitizer, same 4-digit prefix as the toml so the pair sorts together).
3. The extension comes from the response `Content-Type`
   (`audio/flac` → `flac`, `audio/mp4`/`audio/m4a` → `m4a`,
   `audio/mpeg` → `mp3`, `audio/ogg` → `ogg`, `audio/wav` → `wav`),
   falling back to the URL path's extension, then `bin` (the player probes
   by content; the extension is a hint).
4. The toml is written **after** the download succeeds, with
   `playable.file = "<audio file name>"` (relative, new
   `TrackFile::from_track_with_file`), keeping metadata identical to a
   bookmark entry.

Caps: `MAX_CAPTURE_DIRS` stays 1 000; downloads get their own
`MAX_DOWNLOAD_TRACKS = 500` and a total byte budget
`MAX_DOWNLOAD_BYTES = 4 GiB` counted while streaming — a runaway artist
capture must not fill the disk. Downloads run sequentially (gentle on the
provider, trivially bounded memory); the whole capture already runs on a
spawned task, so the orchestrator keeps serving.

All-or-nothing is kept for real failures: any failed download (bad
status, transport error, timeout) aborts the capture and removes the temp
folder. The one softening: a track whose source **cannot be captured at
all** — its stream fails to resolve, or resolves to a non-http(s) target
(a local file playable) — is *skipped* with a warning instead of
aborting. Queue and bookmark captures mix providers (D4), and one local
`/fs` entry must not kill the downloadable rest; the skipped track simply
has no pair in the capture.

### D4 — Nodes opt in via `is_downloadable`

New proto fields `LibraryNode.is_downloadable = 8` and
`LibraryNodeChild.is_downloadable = 7` (additive). Tidal sets the flag
centrally at the end of `get_lib_node`: a node is downloadable when it is
**queueable or lists tracks** (the "or lists tracks" covers search-term
result nodes, which are not queueable as a whole but whose track results
are downloadable); children mirror `is_queable`. The `/queues` and
`/bookmarks` instances opt in wholesale
(`fsdy::Client::with_downloadable_nodes`): their entries are links into
downloadable providers, so `W` on a saved queue or bookmark downloads
its resolvable tracks and skips the rest (D3). `/fs` and `/captures`
stay `false` — capturing a capture is pointless, and local trees have
nothing to download.

Tracks carry no flag: a listed track inherits its containing node's
`is_downloadable` (TUI) — a Tidal album's tracks are downloadable because
the album is. The server enforces at the capture **root**: a directory
source must report `is_downloadable`, a track source's parent node must
(`CaptureError::Unsupported` otherwise). Nested nodes inside the walk are
not re-checked — the pressed node's decision governs its subtree.

### D5 — Wire: the existing rpc gains a `download` flag

`CaptureLibraryNodeRequest` gets `bool download = 3` (additive; old
clients keep bookmarking). `ProviderCommand::CaptureLibraryNode` carries
it and the handler picks the store. Error mapping extends the bookmark
contract: `InvalidName`/`BadSource` → `invalid_argument`,
`TooLarge`/`Disabled`/`Unsupported` → `failed_precondition`, download and
disk failures → `internal`.

### D6 — TUI: `W` on the library pane

`Action::LibraryDownloadNode` bound to `W` in `Scope::Library` ("Download
selection as capture"). Gate: the bare selection must be queueable **and**
downloadable (`selected_downloadable()`; child flag for nodes, the current
node's flag for tracks; marks ignored like `w`). The existing input
overlay opens with `InputPurpose::Capture { path, download: true }`
(label `capture`), prefilled with the selection title. Submit sends
`MessageFromUi::CaptureNode { path, name, download }` → the rpc. Failures
are logged, never fatal to the poll loop.

### D7 — Out of scope (explicitly)

- Retry/resume of failed or partial downloads (re-run the capture).
- Quality/codec selection, transcoding, tagging the audio files.
- Progress display in the TUI while a capture downloads.
- Deduplicating audio across captures, or refreshing links in existing
  bookmarks into downloads.
- DRM circumvention: the download uses exactly the stream URLs the
  provider already serves for playback.

## Structure

```d2
direction: right

server: crabidy-server {
  orch: ProviderOrchestrator
  cap: "capture.rs\nshared walk + caps + swap" {
    link: "Sink::Link"
    dl: "Sink::Download\n(reqwest, timeouts, byte budget)"
  }
  bs: BookmarkStore
  cs: CaptureStore
}

tidal: "tidaldy\n(is_downloadable = is_queable)"

disk: "config/crabidy" {
  shape: cylinder
  b: "bookmarks/<name>/  (link tomls)"
  c: "captures/<name>/  (audio + file tomls)"
}

cfs: "fsdy /captures\n(editable top level)"

server.orch -> server.bs: "CaptureLibraryNode\ndownload=false"
server.orch -> server.cs: "CaptureLibraryNode\ndownload=true"
server.bs -> server.cap.link
server.cs -> server.cap.dl
server.cap.dl -> tidal: "get_urls_for_track\n+ HTTP GET stream"
server.bs -> disk.b
server.cs -> disk.c
cfs -> disk.c: "list + parse (read only)"
server.orch -> cfs: "/captures/..."
```

## Key flow: W on a Tidal album

```d2
shape: sequence_diagram
tui: TUI
rpc: gRPC
orch: Orchestrator
cs: CaptureStore
tidal: Tidal

tui -> rpc: "CaptureLibraryNode(path, name, download=true)"
rpc -> orch: "ProviderCommand (spawned)"
orch -> cs: "capture(name)"
cs -> orch: "get_lib_node: root allows download?"
cs -> tidal: "per track: get_urls_for_track"
cs -> tidal: "HTTP GET (deadline, byte budget)"
cs -> cs: "audio + toml pair\n(toml after audio, file = relative)"
cs -> rpc: "tmp-and-swap captures/<name>/"
rpc -> tui: OK
```

## Risks and open questions

- **Disk usage**: 500 tracks of FLAC can be tens of GiB; the byte budget
  caps one capture, not the folder's total. Accepted — the user manages
  `captures/` like any local music folder (and can delete via `d`).
- **Stream URL churn**: Tidal URLs are short-lived; the download happens
  immediately after fetching each URL, so expiry only matters for very
  slow transfers, which the per-track deadline already bounds.
- **Licensing**: captures are personal-use copies of streams the account
  can already play; nothing here bypasses provider protection.
- Open (future): a progress event stream for long captures; retry with
  classification + jitter; per-provider download quality knobs.